justin/jsix_import
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

[docs] Update docs: kernel mem, process init

Browse Source 
Updated documentation: Added documentation on the kernel address space
layout, process initialization, and rebuilt the syscall docs.
This commit is contained in:
Justin C. Miller
2024-08-10 23:11:14 -07:00
parent d3f5db2479
commit ff64d1989f
7 changed files with 265 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
definitions/memory_layout.yaml
View File

@@ -1,29 +1,38 @@
--- ---
- name: linear - name: linear
desc: Linearly-mapped physical memory
size: 64T size: 64T
shared: true shared: true
- name: bitmap - name: bitmap
desc: Used/free page tracking bitmap
size: 1T size: 1T
shared: true shared: true
- name: heapmap - name: heapmap
desc: Kernel heap accounting structures
size: 32G size: 32G
- name: heap - name: heap
desc: Kernel heap
size: 32G size: 32G
- name: capsmap - name: capsmap
desc: Capabilities accounting structures
size: 32G size: 32G
- name: caps - name: caps
desc: Capabilities
size: 32G size: 32G
- name: stacks - name: stacks
desc: Kernel thread stacks
size: 64G size: 64G
- name: buffers - name: buffers
desc: Kernel buffers
size: 64G size: 64G
- name: logs - name: logs
desc: Kernel logs circular buffer
size: 2G size: 2G

1
docs/Makefile
View File

@@ -19,4 +19,5 @@ help:
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile %: Makefile
PYTHONPATH=$(ROOTDIR)/scripts cog -r -D definitions_path=$(ROOTDIR)/definitions -c syscall_interface.rst PYTHONPATH=$(ROOTDIR)/scripts cog -r -D definitions_path=$(ROOTDIR)/definitions -c syscall_interface.rst
PYTHONPATH=$(ROOTDIR)/scripts cog -r -D definitions_path=$(ROOTDIR)/definitions -c kernel_memory.rst
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

2
docs/index.rst
View File

@@ -71,6 +71,8 @@ The jsix kernel is quite far along now, but the userland systems are still lacki
:caption: Site Contents: :caption: Site Contents:
syscall_interface syscall_interface
kernel_memory
process_initialization
* :ref:`genindex` * :ref:`genindex`

178
docs/kernel_memory.rst Normal file
View File

@@ -0,0 +1,178 @@
.. jsix syscall interface.
.. Automatically updated from the definition files using cog!
.. [[[cog code generation
.. from os.path import join
.. from memory import Layout, unit
..
.. layout = Layout(join(definitions_path, "memory_layout.yaml"))
.. l = max([len(r.name) for r in layout.regions])
.. ]]]
.. [[[end]]] (checksum: d41d8cd98f00b204e9800998ecf8427e)
Kernel memory
=============
While jsix probably should eventually use KASLR to randomize its memory layout,
currently the layout is mostly fixed. (Kernel code locations are not consistent
but aren't explicitly randomized.)
.. [[[cog code generation
.. line_size = 128 * 1024**3 # Each line represents up to 32 GiB
.. max_lines = 32
.. totals = sum([r.size for r in layout.regions])
.. remain = unit((128 * 1024**4) - totals)
..
.. def split(val):
.. return f"0x {val >> 48:04x} {(val >> 32) & 0xffff:04x} {(val >> 16) & 0xffff:04x} {val & 0xffff:04x}"
..
.. cog.outl()
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl(f"| | Address | Size | Use |")
.. cog.outl(f"+=+=============================+==========+=======================================+")
..
.. for region in layout.regions:
.. cog.outl(f"| | ``{split(region.start)}`` | {unit(region.size):>8} | {region.desc:37} |")
.. lines = min(max_lines, region.size // line_size)
.. for i in range(1, lines):
.. cog.outl(f"+-+ | | |")
.. cog.outl(f"| | | | |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
..
.. cog.outl(f"| | ... | | |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl(f"| | ``0x ffff 0000 0000 0000`` | | Kernel code / headers |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl("")
.. cog.outl("")
.. cog.outl(f"Un-reserved virtual memory address space in the higher half: {remain}")
.. cog.outl("")
..
.. ]]]
+-+-----------------------------+----------+---------------------------------------+
| | Address | Size | Use |
+=+=============================+==========+=======================================+
| | ``0x ffff c000 0000 0000`` | 64 TiB | Linearly-mapped physical memory |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bf00 0000 0000`` | 1 TiB | Used/free page tracking bitmap |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff be00 0000 0000`` | 1 TiB | Per-page state tracking structures |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdf8 0000 0000`` | 32 GiB | Kernel heap accounting structures |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdf0 0000 0000`` | 32 GiB | Kernel heap |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bde8 0000 0000`` | 32 GiB | Capabilities accounting structures |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bde0 0000 0000`` | 32 GiB | Capabilities |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdd0 0000 0000`` | 64 GiB | Kernel thread stacks |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdc0 0000 0000`` | 64 GiB | Kernel buffers |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdbf 8000 0000`` | 2 GiB | Kernel logs circular buffer |
+-+-----------------------------+----------+---------------------------------------+
| | ... | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff 0000 0000 0000`` | | Kernel code / headers |
+-+-----------------------------+----------+---------------------------------------+
Un-reserved virtual memory address space in the higher half: 61 TiB
.. [[[end]]] (checksum: 8c336cc8151beba1a79c8d3b653f1109)
* :ref:`genindex`
* :ref:`search`

42
docs/process_initialization.rst Normal file
View File

@@ -0,0 +1,42 @@
.. jsix process initialization in userspace
Process Initialization
======================
jsix follows the `System V ABI`_ on the ``amd64`` architecture. All arguments
needed for program initialization are passed to the program's initial thread on
the stack.
Note that jsix adds a number of additional auxiliary vector entry types for
passing jsix-specific data to a program. The jsix-specific auxiliary vector type
codes (what the ABI document refers to as ``a_type``) start from ``0xf000``. See
the header file ``<j6/init.h>`` for more detail.
.. _System V ABI: https://gitlab.com/x86-psABIs/x86-64-ABI
The initial stack frame
-----------------------
============== ==================== ============ =======
Address Value Bytes Notes
============== ==================== ============ =======
``top`` Stack top (out of stack bounds)
``top`` - 16 0 16 Stack sentinel
\ ``envp`` string data ?
\ ``argv`` string data ?
\ ... ? Possible padding
\ 0, 0 (``AT_NULL``) 16 Aux vector sentinel
\ Aux vectors 16 * `m` ``AT_NULL``-terminated array of Aux vectors
\ 0 8 Environment sentinel
\ ``envp`` 8 * `n` 0-terminated array of environment
string pointers
\ 0 8 Args sentinel
\ ``argv`` 8 * ``argc`` Pointers to argument strings
``rsp`` ``argc`` 8 Number of elements in argv
============== ==================== ============ =======
* :ref:`genindex`
* :ref:`search`

30
docs/syscall_interface.rst
View File

@@ -1,5 +1,5 @@
.. jsix syscall interface. .. jsix syscall interface.
.. Automatically update from the definition files using cog! .. Automatically updated from the definition files using cog!
.. [[[cog code generation .. [[[cog code generation
.. from textwrap import indent .. from textwrap import indent
@@ -14,11 +14,11 @@
.. ]]] .. ]]]
.. [[[end]]] (checksum: d41d8cd98f00b204e9800998ecf8427e) .. [[[end]]] (checksum: d41d8cd98f00b204e9800998ecf8427e)
The jsix syscall interface Syscall interface
========================== =================
The jsix kernel's syscall design is based around object handles. Object handles The jsix kernel's syscall design is based around object handles. Object handles
are also a collections capabilities, encoding certain rights over the object are also a collection of capabilities, encoding certain rights over the object
they reference. they reference.
Very few syscalls in jsix can be made without some handle, and most of them are Very few syscalls in jsix can be made without some handle, and most of them are
@@ -133,7 +133,7 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object :param self: Handle to the mailbox object
.. cpp:function:: j6_result_t j6_mailbox_call (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_in_len, j6_handle_t * handles, size_t * handles_count) .. cpp:function:: j6_result_t j6_mailbox_call (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_size, j6_handle_t * handles, size_t * handles_count, size_t handles_size)
Send a message to the reciever, and block until a response is Send a message to the reciever, and block until a response is
sent. Note that getting this response does not require the sent. Note that getting this response does not require the
@@ -144,10 +144,11 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object :param self: Handle to the mailbox object
:param tag: *[inout]* Undocumented :param tag: *[inout]* Undocumented
:param data: *[optional, inout]* Undocumented :param data: *[optional, inout]* Undocumented
:param data_in_len: number of bytes in data used for input :param data_size: number of total bytes in data buffer
:param handles: *[optional, inout, handle, list]* Undocumented :param handles: *[optional, inout, handle, list]* Undocumented
:param handles_size: total size of handles buffer
.. cpp:function:: j6_result_t j6_mailbox_respond (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_in_len, j6_handle_t * handles, size_t * handles_count, uint64_t * reply_tag, uint64_t flags) .. cpp:function:: j6_result_t j6_mailbox_respond (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_size, j6_handle_t * handles, size_t * handles_count, size_t handles_size, uint64_t * reply_tag, uint64_t flags)
Respond to a message sent using call, and wait for another Respond to a message sent using call, and wait for another
message to arrive. Note that this does not require the send message to arrive. Note that this does not require the send
@@ -159,8 +160,9 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object :param self: Handle to the mailbox object
:param tag: *[inout]* Undocumented :param tag: *[inout]* Undocumented
:param data: *[optional, inout]* Undocumented :param data: *[optional, inout]* Undocumented
:param data_in_len: number of bytes in data used for input :param data_size: number of total bytes in data buffer
:param handles: *[optional, inout, handle, list]* Undocumented :param handles: *[optional, inout, handle, list]* Undocumented
:param handles_size: total size of handles buffer
:param reply_tag: *[inout]* Undocumented :param reply_tag: *[inout]* Undocumented
:param flags: Undocumented :param flags: Undocumented
@@ -185,7 +187,7 @@ control over the threads, handles, and virtual memory space of that process.
:param self: Handle to the process object :param self: Handle to the process object
.. cpp:function:: j6_result_t j6_process_exit (int32_t result) .. cpp:function:: j6_result_t j6_process_exit (int64_t result)
Stop all threads and exit the current process Stop all threads and exit the current process
@@ -353,7 +355,7 @@ necessarily mean that it is mapped into that process' virtual memory space.
:param self: Handle to the vma object :param self: Handle to the vma object
:param size: *[inout]* New size for the VMA, or 0 to query the current size without changing :param size: *[inout]* New size for the VMA, or 0 to query the current size without changing
.. [[[end]]] (checksum: fe448e541c009a1bcf8bdc44f4760e32) .. [[[end]]] (checksum: cb17f54e443d1d3b85995870f3e8dbf2)
Non-object syscalls Non-object syscalls
------------------- -------------------
@@ -411,6 +413,12 @@ either do not require an object handle, or operate generically on handles.
:param clone: *[out]* The new handle :param clone: *[out]* The new handle
:param mask: The capability bitmask :param mask: The capability bitmask
.. cpp:function:: j6_result_t j6_handle_close (j6_handle_t hnd)
Close the handle to an object
:param hnd: *[handle]* The handle to close
.. cpp:function:: j6_result_t j6_futex_wait (const uint32_t * address, uint32_t current, uint64_t timeout) .. cpp:function:: j6_result_t j6_futex_wait (const uint32_t * address, uint32_t current, uint64_t timeout)
Block waiting on a futex Block waiting on a futex
@@ -432,5 +440,5 @@ either do not require an object handle, or operate generically on handles.
:param exit_code: Undocumented :param exit_code: Undocumented
.. [[[end]]] (checksum: b8b12e99a4a00c99b3859f05000a7bfd) .. [[[end]]] (checksum: 0b9d051972abcbb6de408f411331785f)

16
scripts/memory.py
View File

@@ -1,6 +1,18 @@
def unit(size):
units = ['B', 'KiB', 'MiB', 'GiB', 'TiB', 'PiB']
o = 0
while size >= 1024 and o < len(units):
o += 1
size = size >> 10
return f"{size} {units[o]}"
class Layout: class Layout:
from collections import namedtuple from collections import namedtuple
Region = namedtuple("Region", ("name", "start", "size", "shared")) Region = namedtuple("Region", ("name", "desc", "start", "size", "shared"))
sizes = {'G': 1024 ** 3, 'T': 1024 ** 4} sizes = {'G': 1024 ** 3, 'T': 1024 ** 4}
@@ -26,7 +38,7 @@ class Layout:
for r in data: for r in data:
size = Layout.get_size(r["size"]) size = Layout.get_size(r["size"])
addr -= size addr -= size
regions.append(Layout.Region(r["name"], addr, size, regions.append(Layout.Region(r["name"], r["desc"], addr, size,
r.get("shared", False))) r.get("shared", False)))
self.regions = tuple(regions) self.regions = tuple(regions)