The syscall helpers.h get_handle functions should be returing
j6_err_invalid_arg if the handle they're given is j6_handle_invalid,
unless explicitly set to optional.
Make build_symbol_table.py output statistics on the symbol table it
builds, and emit warnings for zero-length symbols. Also added lengths to
several functions defined in asm that this uncovered.
Restructuring paging into an object that carries its page cache with it
and makes for simpler code. Program loading is also changed to not copy
the pages loaded from the file into new pages - we can impose a new
constraint that anything loaded by boot have a simple, page-aligned
layout so that we can just map the existing pages into the right
addresses. Also included are some linker script changes to help
accommodate this.
The symbol table needs to be passed to the panic handler very early in
the kernel, loading it in init is far less useful. Return it to the boot
directory and remove it from the initrd.
I added util::format as a replacement for other printf implementations
last year, but it sat there only being used by the kernel all this time.
Now I've templated it so that it can be used by the bootloader, and
removed printf from panic.serial as well.
Previously, to add a custom linker script, a module would need to modify
its variables after the fact to add to ldflags. Now module constructors
take a new keyword `ld_script` and handle the ldflags and dependencies
properly.
Previously, status_line would show 'ok' until something went wrong. Now,
show an empty box until either a warning or error happens, or the
cleanup happens without an issue.
Load drv.uart.elf and srv.logger.elf from the initrd and start them.
It's extremely manual and hard-coded at the moment, but it works and
they run, getting us back to where we were pre-initrd branch.
Using `-fvisibility=hidden` when building the kernel, and then
`--discard-all` when stripping it, we shave almost 100KiB off of the
resulting ELF file.
Also dropped some unused symbols from the linker script, and rearranged
the sections so that the file is able to be mapped directly into memory
instead of having each section copied.
It felt clunky to have zstd.module in src/libraries/zstd by itself, and
doesn't make much sense in src/libraries as it's an external library
anyway.
Now the ./configure script will pick up .module files in the top-level
external directory as well.
CDB seemed to be too simple for the needs of init, and squashfs is too
laden with design choices to work around Linux's APIs. This commit adds
creation of an initrd image of a new format I've called `j6romfs`.
Note that this commit currently does not work! The initrd-reading code
still needs to be added.
I've been using `const util::buffer` a lot of places where that's not
really what I mean, because I was avoiding having a separate type for
buffers with const pointers. But really this needed to happen.
The initrd image is now created by the build system, loaded by the
bootloader, and passed to srv.init, which loads it (but doesn't do
anything with it yet, so this is actually a functional regression).
This simplifies a lot of the modules code between boot and init as well:
Gone are the many subclasses of module and all the data being inline
with the module structs, except for any loaded files. Now the only
modules loaded and passed will be the initrd, and any devices only the
bootloader has knowledge of, like the UEFI framebuffer.
A new compressed initrd format for srv.init to load drivers, services,
and data from, instead of every file getting loaded by the bootloader.
This will allow for less memory allocated by the bootloader and passed
to init if not every driver or data file is loaded.
Loading, passing, and using the new initrd will be done in a coming
commit.
I discovered that QEMU has a "remote" option to the `-vnc` switch, which
causes it to reverse-connect to a VNC client on start. I've added this
into qemu.sh as the -r option.
Having main.cpp in the kernel and in the application being debugged is
annoying when setting breakpoints, so just like with main() vs
kernel_main(), kernel/main.cpp is now kernel/kernel_main.cpp.
The clang __builtin_* functions cannot be relied upon, as they may just
emit a call to the stdlib version. So this commit adds an implementation
for ceil and frexpr, as well as their float versions.
In preparation for futexes, I wanted to make kobjects a bit lighter.
Storing 32 bits of object id, and 8 bits of type (and not ending the
class in a ushort for handle count, which meant all kobjects were likely
to have a bunch of pad bytes), the kobject class data is now just one 8
byte word.
Also from this, change logs that mention threads or processes from
printing the full koid to just 2 bytes of object id from both process
and thread, which makes following the logs much easier.
This new class makes it easier for user programs to spawn threads. This
change also includes support for .hh files in modules, to differentiate
headers that are C++-only in system libraries.
The status code from thread exit had too many issues, (eg, how does it
relate to process exit code? what happens when different threads exit
with different exit codes?) and not enough value, so I'm getting rid of
it.
In the new mailbox structure, passing a j6_handle_invalid with a message
would result in a permission denied result, as the process did not have
a handle "0".
When the bootloader loads srv.init's program sections into memory, it
needed to page-align them if they weren't. srv.init's loader itself
handles this case, but the bootloader's did not.
heap_allocator::reallocate relies on the allocate and free methods so
mostly doesn't need locking, but it does touch the tracking map, so
needs to protect that with a lock.
A number of simplifications of mailboxes now that the interface is much
simpler, and synchronous.
* call and respond can now only transfer one handle at a time
* mailbox objects got rid of the message queue, and just have
wait_queues of blocked threads, and a reply_to map.
* threads now have a message_data struct on them for use by mailboxes
Instead of handles / capabilities having numeric ids that are only valid
for the owning process, they are now global in a system capabilities
table. This will allow for specifying capabilities in IPC that doesn't
need to be kernel-controlled.
Processes will still need to be granted access to given capabilities,
but that can become a simpler system call than the current method of
sending them through mailbox messages (and worse, having to translate
every one into a new capability like was the case before). In order to
track which handles a process has access to, a new node_set based on
node_map allows for an efficient storage and lookup of handles.
When node_map grew, it was not properly applying the fixup routine to
non-moved elements. This fixes the grow algorithm to:
1. Realloc the array and set all new slots to empty/invalid
2. Check each old slot and remove/reinsert the item if it exists and its
optimal slot is later in the array than its current slot
3. Reverse-iterate the original slots and call fixup() on empty slots to
keep items from being located after a more-optimal empty slot
Also fixed the fixup() function to not need to be called in a loop
anymore, as it's only used the one way - on a given empty slot, looping
until it hits an empty slot or optimally-placed item.
The allocator is a interface for types that expose allocator functions
for use in container templates like node_map (usage coming soon).
Also added an implementation for the kernel heap allocator.
The heap_allocator::get_free(order) function returns a reference to the
head pointer of the given freelist, so that it can be manipulated.
However, split_off was also taking a reference to a pointer for an out
param - passing the freelist pointer in here caused split_off to modify
the freelist.
I cleaned up a bunch of the places the freelist pointers were being
touched to make the usage more explicit.
Adding pretty printers to aid in debugging:
* For the cap_table type so that `p g_cap_table` displays a neat table
* For node_sets of handles to easily see what handles a process owns
* For util::vector to include its contents in the output
Created a new util/node_map.h that implements a map that grows in-place.
Now this is used for tracking blocks' size orders, instead of a header
at the start of the memory block. This allows the whole buddy block to
be allocated, allowing for page-aligned (or greater) blocks to be
requested from the heap.
The kernel log levels are now numerically reversed so that more-verbose
levels can be added to the end. Replaced 'debug' with 'verbose', and
added new 'spam' level.
