Clang will complain if main() is not declared with 0, 2, or 3 arguments.
In order to allow an extra 4th parameter, a new weak main() symbol which
just jumps to driver_main is defined, and _start passes the extra init
pointer to main.
Additionally, libc's crt0.s _start is made weak, and a matching
_libc_crt0_start symbol is defined for implementations that wish to
override _start but still call libc's _start. (Will be used by init.)
Previously processes and threads would be deleted by the scheduler. Now,
only delete them based on refcounts - this allows joining an
already-exited thread, for instance.
Previously event tried to read its value in event::wake_observer, which
required jumping through some hoops in how wait_queue was designed, so
that a value wouldn't be wasted if the wait_queue was empty. Now, read
the event value in event::wait after returning from the thread::block
call instead, which simplifies the whole process and lets us simplify
the wait_queue API as well.
For the coming switch to cap/handle ref-counting being the main lifetime
determiner of objects, get rid of self handles for threads and processes
to avoid circular references. Instead, passing 0 to syscalls expecting a
thread or process handle signifies "this process/thread".
There was an inverted boolean logic in determining how many consecutive
pages were available.
Also adding some memory debugging tools I added to track down the recent
memory bugs:
- A direct debugcon::write call, for logging to the debugcon without the
possible page faults with the logger.
- A new vm_space::lock call, to make a page not fillable in memory
debugging mode
- A mode in heap_allocator to always alloc new pages, and lock freed
pages to cause page faults for use-after-free bugs.
- Logging in kobject on creation and deletion
- Page table cache structs are now page-sized for easy pointer math
Yet again burned by the fack that integer literals are assumed to be of
type int, so `1 << n` is 0 for any n >= 32. This burned me in the frame
allocator, but I also grepped for all instances of `1 <<` and fixed
those too.
Add a version of thread::block() that takes a lock and releases it after
marking the thread as unready, but before calling the scheduler.
Use this version of block() in the wait_queue.
Split out different constants for scheduler::idle_priority and
scheduler::max_priority, so that threads never fall to the same priority
level as the idle threads.
Previously process::exit() was going through the threads in order
calling thread::exit() - which blocks and never wakes if called on the
current thread. Since the current thread likely belongs to the process
which is exiting, and the current thread wasn't guaranteed to be last in
the list, this could leave threads not cleaned up.
Worse, no matter what, this caused the m_threads_lock to always be held
forever on exit, keeping the scheduler from ever finishing a call to
process::thread_exited() on its threads.
The `s_cache_count` counter had the potential to get out of sync with
the cache itself. Since we only call `fill_table_page_cache()` when the
cache is empty, the counter was not useful. After chasing the bug for
hours to figure out how they were getting out of sync, I just ripped it
out.
Another spot I meant to go back and clean up with a lock - found it when
a process with threads running on two CPUs exited, and the scheduler
tried to delete the process on both CPUs.
Several minor changes related to debug output.
- Colorize the debugcon logger like the userspace one.
- Display the process and thread for each cpu in the panic display
- Skip the panic() frame in panic back traces
- Don't try to follow obviously bad (non-canonical) stack frame pointers
Now that the log ring buffer is at a fixed known address, and entries
persist even after being read, it's easy to add a command to see what's
in the buffer from GDB. Useful if there's some log messages that hadn't
yet been printed at the time of a panic.
Two minor debugging helpers:
- the GDB script was still referencing m_koid on objects, switched to
the replacement m_obj_id instead.
- finally gave in and made panic print 1-based CPU ids like GDB uses
instead of 0-based like the hardware and LITERALLY EVERYTHING ELSE
In order to pass along arguments like the framebuffer, it's far simpler
to have that data stored along with the modules than mapping new pages
for every structure. Also now optionally pass a module's data to a
driver as init starts it.
This commit fixes the mailbox tests in test_runner, which broke when
mailbox was simplified to just use call and respond. It also fixes a
bug the tests uncovered: if the mailbox is closed while a caller is in
the reply map (ie, when its call data has been passed on to a thread
calling respond, but has yet to be responded to itself), that caller is
never awoken.
This commit changes the add_user_thunk to point to a new routine,
initialize_user_cpu, which sets all the registers that were previously
unset when starting a new user thread. The values for rdi and rsi are
popped off the initial stack values that add_user_thunk sets up, so that
user thread procs can take up to two arguments.
To suppor this, j6_thread_create gained two new arguments, which are
passed on to the thread.
This also let me finally get rid of the hack of passing an argument in
rsp when starting init.
This commit re-adds testapp to the default manifest and does some
housecleaning on the module:
- Remove the old serial.* and io.*
- Update it to use current syscall APIs
- Update it to use libj6's higher-level thread API
The debugcon logger is now separate from logger::output, and is instead
a kernel-internal thread that watches for logs and prints them to the
deubcon device.
A bip-buffer is good for producer/consumer systems, but ideally logs
will stay in the buffer until they're ousted because they need to be
overwritten. Now they're a regular ring buffer and every entry has an
incremental id. Consumers pass in the last id they've seen, and will get
the next log in the sequence.
This ended up being unused, but still probably useful: Refactor out the
"find" logic of page_tree::find_or_add (note that this is different than
the "find" logic of page_tree::find, as it potentially modifies the tree
to add a location to accommodate the page being searched for) into a new
page_tree::get_entry method. That was then used to add an add_existing
method for inserting pages into the page_tree.
In prep for the coming change to keep log entries as a true ring buffer,
move the log buffer from bss into its own memory section.
Related changes in this commit:
- New vm_area_ring, which maps a set of pages twice to allow easy linear
reading of data from a ring buffer when it wraps around the end.
- logger_init() went away, and the logger ctor is called from
mem::initialize()
- Instead of an event object, the logger just has a bare wait_queue
- util::counted::from template type changed slightly to allow easy
conversion from an intptr_t as well as a pointer
- Previously added debugcon_logger code removed - this will be added in
a separate file in a followup commit
Previously we were hard-coding loading specific files (the UART driver
and logging server) from the initrd. Now j6romfs has a for_each() method
to allow iterating all files in a directory, and init loads all programs
from /jsix/drivers and /jsix/services. Eventually this will need more
dynamic loading decisions for drivers but for now it's fine.
There have been a number of incidents lately where I've needed to see
logs but have been working on init, and broken the log output of
srv.logger. This commit adds a debug console output to io port 0x6600
if enabled at the top of logger.cpp.
I often want to use util::counted as a bool, like with a regular
pointer. There was some basic support for that, but it didn't cover
every case - now it should.
The syscall helpers.h get_handle functions should be returing
j6_err_invalid_arg if the handle they're given is j6_handle_invalid,
unless explicitly set to optional.
Make build_symbol_table.py output statistics on the symbol table it
builds, and emit warnings for zero-length symbols. Also added lengths to
several functions defined in asm that this uncovered.
Restructuring paging into an object that carries its page cache with it
and makes for simpler code. Program loading is also changed to not copy
the pages loaded from the file into new pages - we can impose a new
constraint that anything loaded by boot have a simple, page-aligned
layout so that we can just map the existing pages into the right
addresses. Also included are some linker script changes to help
accommodate this.
The symbol table needs to be passed to the panic handler very early in
the kernel, loading it in init is far less useful. Return it to the boot
directory and remove it from the initrd.
I added util::format as a replacement for other printf implementations
last year, but it sat there only being used by the kernel all this time.
Now I've templated it so that it can be used by the bootloader, and
removed printf from panic.serial as well.
Previously, to add a custom linker script, a module would need to modify
its variables after the fact to add to ldflags. Now module constructors
take a new keyword `ld_script` and handle the ldflags and dependencies
properly.
Previously, status_line would show 'ok' until something went wrong. Now,
show an empty box until either a warning or error happens, or the
cleanup happens without an issue.
Load drv.uart.elf and srv.logger.elf from the initrd and start them.
It's extremely manual and hard-coded at the moment, but it works and
they run, getting us back to where we were pre-initrd branch.
Using `-fvisibility=hidden` when building the kernel, and then
`--discard-all` when stripping it, we shave almost 100KiB off of the
resulting ELF file.
Also dropped some unused symbols from the linker script, and rearranged
the sections so that the file is able to be mapped directly into memory
instead of having each section copied.
It felt clunky to have zstd.module in src/libraries/zstd by itself, and
doesn't make much sense in src/libraries as it's an external library
anyway.
Now the ./configure script will pick up .module files in the top-level
external directory as well.
CDB seemed to be too simple for the needs of init, and squashfs is too
laden with design choices to work around Linux's APIs. This commit adds
creation of an initrd image of a new format I've called `j6romfs`.
Note that this commit currently does not work! The initrd-reading code
still needs to be added.
I've been using `const util::buffer` a lot of places where that's not
really what I mean, because I was avoiding having a separate type for
buffers with const pointers. But really this needed to happen.
