justin/jsix
1
1
Fork 0
mirror of https://github.com/justinian/jsix.git synced 2025-12-09 16:04:32 -08:00
Code Issues Packages Projects Releases Wiki Activity

[docs] Update docs: kernel mem, process init

Browse Source 
Updated documentation: Added documentation on the kernel address space
layout, process initialization, and rebuilt the syscall docs.
This commit is contained in:
Justin C. Miller
2024-08-10 23:11:14 -07:00
parent d3f5db2479
commit ff64d1989f
7 changed files with 265 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
definitions/memory_layout.yaml
View File

@@ -1,29 +1,38 @@
---
- name: linear
desc: Linearly-mapped physical memory
size: 64T
shared: true
- name: bitmap
desc: Used/free page tracking bitmap
size: 1T
shared: true
- name: heapmap
desc: Kernel heap accounting structures
size: 32G
- name: heap
desc: Kernel heap
size: 32G
- name: capsmap
desc: Capabilities accounting structures
size: 32G
- name: caps
desc: Capabilities
size: 32G
- name: stacks
desc: Kernel thread stacks
size: 64G
- name: buffers
desc: Kernel buffers
size: 64G
- name: logs
desc: Kernel logs circular buffer
size: 2G

1
docs/Makefile
View File

@@ -19,4 +19,5 @@ help:
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
PYTHONPATH=$(ROOTDIR)/scripts cog -r -D definitions_path=$(ROOTDIR)/definitions -c syscall_interface.rst
PYTHONPATH=$(ROOTDIR)/scripts cog -r -D definitions_path=$(ROOTDIR)/definitions -c kernel_memory.rst
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

2
docs/index.rst
View File

@@ -71,6 +71,8 @@ The jsix kernel is quite far along now, but the userland systems are still lacki
:caption: Site Contents:
syscall_interface
kernel_memory
process_initialization
* :ref:`genindex`

178
docs/kernel_memory.rst Normal file
View File

@@ -0,0 +1,178 @@
.. jsix syscall interface.
.. Automatically updated from the definition files using cog!
.. [[[cog code generation
.. from os.path import join
.. from memory import Layout, unit
..
.. layout = Layout(join(definitions_path, "memory_layout.yaml"))
.. l = max([len(r.name) for r in layout.regions])
.. ]]]
.. [[[end]]] (checksum: d41d8cd98f00b204e9800998ecf8427e)
Kernel memory
=============
While jsix probably should eventually use KASLR to randomize its memory layout,
currently the layout is mostly fixed. (Kernel code locations are not consistent
but aren't explicitly randomized.)
.. [[[cog code generation
.. line_size = 128 * 1024**3 # Each line represents up to 32 GiB
.. max_lines = 32
.. totals = sum([r.size for r in layout.regions])
.. remain = unit((128 * 1024**4) - totals)
..
.. def split(val):
.. return f"0x {val >> 48:04x} {(val >> 32) & 0xffff:04x} {(val >> 16) & 0xffff:04x} {val & 0xffff:04x}"
..
.. cog.outl()
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl(f"| | Address | Size | Use |")
.. cog.outl(f"+=+=============================+==========+=======================================+")
..
.. for region in layout.regions:
.. cog.outl(f"| | ``{split(region.start)}`` | {unit(region.size):>8} | {region.desc:37} |")
.. lines = min(max_lines, region.size // line_size)
.. for i in range(1, lines):
.. cog.outl(f"+-+ | | |")
.. cog.outl(f"| | | | |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
..
.. cog.outl(f"| | ... | | |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl(f"| | ``0x ffff 0000 0000 0000`` | | Kernel code / headers |")
.. cog.outl(f"+-+-----------------------------+----------+---------------------------------------+")
.. cog.outl("")
.. cog.outl("")
.. cog.outl(f"Un-reserved virtual memory address space in the higher half: {remain}")
.. cog.outl("")
..
.. ]]]
+-+-----------------------------+----------+---------------------------------------+
| | Address | Size | Use |
+=+=============================+==========+=======================================+
| | ``0x ffff c000 0000 0000`` | 64 TiB | Linearly-mapped physical memory |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bf00 0000 0000`` | 1 TiB | Used/free page tracking bitmap |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff be00 0000 0000`` | 1 TiB | Per-page state tracking structures |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+ | | |
| | | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdf8 0000 0000`` | 32 GiB | Kernel heap accounting structures |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdf0 0000 0000`` | 32 GiB | Kernel heap |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bde8 0000 0000`` | 32 GiB | Capabilities accounting structures |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bde0 0000 0000`` | 32 GiB | Capabilities |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdd0 0000 0000`` | 64 GiB | Kernel thread stacks |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdc0 0000 0000`` | 64 GiB | Kernel buffers |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff bdbf 8000 0000`` | 2 GiB | Kernel logs circular buffer |
+-+-----------------------------+----------+---------------------------------------+
| | ... | | |
+-+-----------------------------+----------+---------------------------------------+
| | ``0x ffff 0000 0000 0000`` | | Kernel code / headers |
+-+-----------------------------+----------+---------------------------------------+
Un-reserved virtual memory address space in the higher half: 61 TiB
.. [[[end]]] (checksum: 8c336cc8151beba1a79c8d3b653f1109)
* :ref:`genindex`
* :ref:`search`

42
docs/process_initialization.rst Normal file
View File

@@ -0,0 +1,42 @@
.. jsix process initialization in userspace
Process Initialization
======================
jsix follows the `System V ABI`_ on the ``amd64`` architecture. All arguments
needed for program initialization are passed to the program's initial thread on
the stack.
Note that jsix adds a number of additional auxiliary vector entry types for
passing jsix-specific data to a program. The jsix-specific auxiliary vector type
codes (what the ABI document refers to as ``a_type``) start from ``0xf000``. See
the header file ``<j6/init.h>`` for more detail.
.. _System V ABI: https://gitlab.com/x86-psABIs/x86-64-ABI
The initial stack frame
-----------------------
============== ==================== ============ =======
Address Value Bytes Notes
============== ==================== ============ =======
``top`` Stack top (out of stack bounds)
``top`` - 16 0 16 Stack sentinel
\ ``envp`` string data ?
\ ``argv`` string data ?
\ ... ? Possible padding
\ 0, 0 (``AT_NULL``) 16 Aux vector sentinel
\ Aux vectors 16 * `m` ``AT_NULL``-terminated array of Aux vectors
\ 0 8 Environment sentinel
\ ``envp`` 8 * `n` 0-terminated array of environment
string pointers
\ 0 8 Args sentinel
\ ``argv`` 8 * ``argc`` Pointers to argument strings
``rsp`` ``argc`` 8 Number of elements in argv
============== ==================== ============ =======
* :ref:`genindex`
* :ref:`search`

30
docs/syscall_interface.rst
View File

@@ -1,5 +1,5 @@
.. jsix syscall interface.
.. Automatically update from the definition files using cog!
.. Automatically updated from the definition files using cog!
.. [[[cog code generation
.. from textwrap import indent
@@ -14,11 +14,11 @@
.. ]]]
.. [[[end]]] (checksum: d41d8cd98f00b204e9800998ecf8427e)
The jsix syscall interface
==========================
Syscall interface
=================
The jsix kernel's syscall design is based around object handles. Object handles
are also a collections capabilities, encoding certain rights over the object
are also a collection of capabilities, encoding certain rights over the object
they reference.
Very few syscalls in jsix can be made without some handle, and most of them are
@@ -133,7 +133,7 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object
.. cpp:function:: j6_result_t j6_mailbox_call (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_in_len, j6_handle_t * handles, size_t * handles_count)
.. cpp:function:: j6_result_t j6_mailbox_call (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_size, j6_handle_t * handles, size_t * handles_count, size_t handles_size)
Send a message to the reciever, and block until a response is
sent. Note that getting this response does not require the
@@ -144,10 +144,11 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object
:param tag: *[inout]* Undocumented
:param data: *[optional, inout]* Undocumented
:param data_in_len: number of bytes in data used for input
:param data_size: number of total bytes in data buffer
:param handles: *[optional, inout, handle, list]* Undocumented
:param handles_size: total size of handles buffer
.. cpp:function:: j6_result_t j6_mailbox_respond (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_in_len, j6_handle_t * handles, size_t * handles_count, uint64_t * reply_tag, uint64_t flags)
.. cpp:function:: j6_result_t j6_mailbox_respond (j6_handle_t self, uint64_t * tag, void * data, size_t * data_len, size_t data_size, j6_handle_t * handles, size_t * handles_count, size_t handles_size, uint64_t * reply_tag, uint64_t flags)
Respond to a message sent using call, and wait for another
message to arrive. Note that this does not require the send
@@ -159,8 +160,9 @@ as shared memory channels, but more flexible.
:param self: Handle to the mailbox object
:param tag: *[inout]* Undocumented
:param data: *[optional, inout]* Undocumented
:param data_in_len: number of bytes in data used for input
:param data_size: number of total bytes in data buffer
:param handles: *[optional, inout, handle, list]* Undocumented
:param handles_size: total size of handles buffer
:param reply_tag: *[inout]* Undocumented
:param flags: Undocumented
@@ -185,7 +187,7 @@ control over the threads, handles, and virtual memory space of that process.
:param self: Handle to the process object
.. cpp:function:: j6_result_t j6_process_exit (int32_t result)
.. cpp:function:: j6_result_t j6_process_exit (int64_t result)
Stop all threads and exit the current process
@@ -353,7 +355,7 @@ necessarily mean that it is mapped into that process' virtual memory space.
:param self: Handle to the vma object
:param size: *[inout]* New size for the VMA, or 0 to query the current size without changing
.. [[[end]]] (checksum: fe448e541c009a1bcf8bdc44f4760e32)
.. [[[end]]] (checksum: cb17f54e443d1d3b85995870f3e8dbf2)
Non-object syscalls
-------------------
@@ -411,6 +413,12 @@ either do not require an object handle, or operate generically on handles.
:param clone: *[out]* The new handle
:param mask: The capability bitmask
.. cpp:function:: j6_result_t j6_handle_close (j6_handle_t hnd)
Close the handle to an object
:param hnd: *[handle]* The handle to close
.. cpp:function:: j6_result_t j6_futex_wait (const uint32_t * address, uint32_t current, uint64_t timeout)
Block waiting on a futex
@@ -432,5 +440,5 @@ either do not require an object handle, or operate generically on handles.
:param exit_code: Undocumented
.. [[[end]]] (checksum: b8b12e99a4a00c99b3859f05000a7bfd)
.. [[[end]]] (checksum: 0b9d051972abcbb6de408f411331785f)

16
scripts/memory.py
View File

@@ -1,6 +1,18 @@
def unit(size):
units = ['B', 'KiB', 'MiB', 'GiB', 'TiB', 'PiB']
o = 0
while size >= 1024 and o < len(units):
o += 1
size = size >> 10
return f"{size} {units[o]}"
class Layout:
from collections import namedtuple
Region = namedtuple("Region", ("name", "start", "size", "shared"))
Region = namedtuple("Region", ("name", "desc", "start", "size", "shared"))
sizes = {'G': 1024 ** 3, 'T': 1024 ** 4}
@@ -26,7 +38,7 @@ class Layout:
for r in data:
size = Layout.get_size(r["size"])
addr -= size
regions.append(Layout.Region(r["name"], addr, size,
regions.append(Layout.Region(r["name"], r["desc"], addr, size,
r.get("shared", False)))
self.regions = tuple(regions)