The `driver_main` sinature was an alternate signature for `main`
implemented with weak symbols, but it causes linking issues when not
statically linked, and drivers are going to work differently soon
anyway. Just get rid of it for now.
Bonnibel will now build dynamic libraries when they're dependencies for
non-statically linked modules. It will also copy those shared libraries
into the initrd image for programs being copied into the image.
This commit includes a number of changes to enable loading of PIE
executables:
- The loader in srv.init checks for a `PT_INTERP` segment in the program
its loading, and if it exists, loads the specified interpreter and
passes control to it instead of the program itself.
- Added ld.so the dynamic linker executable and set it as the
interpreter for all user-target programs.
- Program initial stack changed again to now contain a number of
possible tagged structures, including a new one for ld.so's arguments,
and for passing handles tagged with protocol ids.
- Added a stub for a new VFS protocol. Unused so far, but srv.init will
need to serve VFS requests from ld.so once I transition libraries to
shared libs for user-target programs. (Right now all executables are
PIE but statically linked, so they only need internal relocations.)
- Added 16 and 8 bit variants of `util::bitset`. This ended up not being
used, but could be useful.
Going back to letting mailboxes use variable-length data. Note that this
requires extra copies, so shared memory channels should be used for
anything in the hot path. But this allows better RPC over mailboxes and
other flexibility.
Other changes:
- added a j6::proto::sl::client class to act as a service locator
client, instead of duplicating that code in every program.
- moved protocol ids into j6/tables/protocols.inc so that C++ clients
can easily have their own API
This change allows for parameters in definition files to have the "list"
option while also needing the handle verification. The verify function
will now iterate through the list checking capabilities and types on
every valid handle in the list.
Add extra info to the NASM `global` directive to specify the sizes of
these symbols, mostly so they look right in `nm` or `readelf` and don't
trick me into thinking something is wrong.
My `REP.MOVSB` `memcpy` implementation had marked its C++ variable
constraints as output instead of input, causing the compiler to emit
code to copy the values of `$rsi` and `$rdi` back into the `src` and
`dst` pointers, so after the copy `dst` pointed to the memory just
beyond what had been copied.
Very few places actually used the return value from `memcpy`, so this
went unnoticed for a bit..
This is the second of two big changes to clean up includes throughout
the project. Since I've started using clangd with Neovim and using
VSCode's intellisense, my former strategy of copying all header files
into place in `build/include` means that the real files don't show up in
`compile_commands.json` and so display many include errors when viewing
those header files in those tools.
That setup was mostly predicated on a desire to keep directory depths
small, but really I don't think paths like `src/libraries/j6/j6` are
much better than `src/libraries/j6/include/j6`, and the latter doesn't
have the aforementioned issues, and is clearer to the casual observer as
well.
Some additional changes:
- Added a new module flag `copy_headers` for behavior similar to the old
style, but placing headers in `$module_dir/include` instead of the
global `build/include`. This was needed for external projects that
don't follow the same source/headers folder structure - in this case,
`zstd`.
- There is no longer an associated `headers.*.ninja` for each
`module.*.ninja` file, as only parsed headers need to be listed; this
functionality has been moved back into the module's ninja file.
This is the first of two rather big changes to clean up includes
throughout the project. In this commit, the implicit semi-dependency on
libc that bonnibel adds to every module is removed. Previously, I was
sloppy with includes of libc headers and include directory order. Now,
the freestanding headers from libc are split out into libc_free, and an
implicit real dependency is added onto this module, unless `no_libc` is
set to `True`. The full libc needs to be explicitly specified as a
dependency to be used.
Several things needed to change in order to do this:
- Many places use `memset` or `memcpy` that cannot depend on libc. The
kernel has basic implementations of them itself for this reason. Now
those functions are moved into the lower-level `j6/memutils.h`, and
libc merely references them. Other modules are now free to reference
those functions from libj6 instead.
- The kernel's `assert.h` was renamed kassert.h (matching its `kassert`
function) so that the new `util/assert.h` can use `__has_include` to
detect it and make sure the `assert` macro is usable in libutil code.
- Several implementation header files under `__libj6/` also moved under
the new libc_free.
- A new `include_phase` property has been added to modules for Bonnibel,
which can be "normal" (default) or "late" which uses `-idirafter`
instead of `-I` for includes.
- Since `<utility>` and `<new>` are not freestanding, implementations of
`remove_reference`, `forward`, `move`, and `swap` were added to the
`util` namespace to replace those from `std`, and `util/new.h` was
added to declare `operator new` and `operator delete`.
When keeping track of addresses to give to the kernel for channel shared
memory, double the channel's size since the kernel will double-map the
area as a ring buffer.
Now threads inherit their MXCSR (sans exception state bits) SIMD
settings from their creator. By default, all exceptions are masked, and
both "to zero" flags are set.
Well god damnit, when i converted the `cpu::cpu_id::regs` struct to a
union, i was super sloppy and forgot to wrap the existing fields in
their own anonymous struct. I have been wrong about CPUID vales for
ages.
The libc CRT _start function had a stray pop left in it, which was
causing the stack to never be 16-byte aligned and thus causing crashes
when SSE instructions were called.
When used in kernel vs. non-kernel code the assert macros were not
working as expected. Other util code does not use assert like this, so
I'm just dropping it from bitset.
This commit does a number of things to start the transition of channels
from kernel to user space:
- Remove channel objects / syscalls from the kernel
- Add mutex type in libj6
- Add condition type in libj6
- Add a `ring` type flag for VMA syscalls to create ring buffers
- Implement a rudimentary shared memory channel using all of the above
Add the syscalls j6_futex_wait and j6_futex_wake. Currently marking this
as WIP as they need more testing.
Added to support futexes:
- vm_area and vm_space support for looking up physical address for a
virtual address
- libj6 mutex implementation using futex system calls
There are some SSE instructions (moveaps, moveups) in userland code that
QEMU software emulation seems to be fine with but generate `#UD` on KVM.
So let's finally get floating-point support working. This is the first
step, just setting the control regs to try to fix that error.
This was kept in the kernel as a way to keep exercising the code, but it
doesn't belong there. This moves it to init, which doesn't do anything
but probe for devices currently - but at least it's executing the code
in userspace now.
Clang will complain if main() is not declared with 0, 2, or 3 arguments.
In order to allow an extra 4th parameter, a new weak main() symbol which
just jumps to driver_main is defined, and _start passes the extra init
pointer to main.
Additionally, libc's crt0.s _start is made weak, and a matching
_libc_crt0_start symbol is defined for implementations that wish to
override _start but still call libc's _start. (Will be used by init.)
For the coming switch to cap/handle ref-counting being the main lifetime
determiner of objects, get rid of self handles for threads and processes
to avoid circular references. Instead, passing 0 to syscalls expecting a
thread or process handle signifies "this process/thread".
Yet again burned by the fack that integer literals are assumed to be of
type int, so `1 << n` is 0 for any n >= 32. This burned me in the frame
allocator, but I also grepped for all instances of `1 <<` and fixed
those too.
Previously process::exit() was going through the threads in order
calling thread::exit() - which blocks and never wakes if called on the
current thread. Since the current thread likely belongs to the process
which is exiting, and the current thread wasn't guaranteed to be last in
the list, this could leave threads not cleaned up.
Worse, no matter what, this caused the m_threads_lock to always be held
forever on exit, keeping the scheduler from ever finishing a call to
process::thread_exited() on its threads.
In order to pass along arguments like the framebuffer, it's far simpler
to have that data stored along with the modules than mapping new pages
for every structure. Also now optionally pass a module's data to a
driver as init starts it.
This commit changes the add_user_thunk to point to a new routine,
initialize_user_cpu, which sets all the registers that were previously
unset when starting a new user thread. The values for rdi and rsi are
popped off the initial stack values that add_user_thunk sets up, so that
user thread procs can take up to two arguments.
To suppor this, j6_thread_create gained two new arguments, which are
passed on to the thread.
This also let me finally get rid of the hack of passing an argument in
rsp when starting init.
This commit re-adds testapp to the default manifest and does some
housecleaning on the module:
- Remove the old serial.* and io.*
- Update it to use current syscall APIs
- Update it to use libj6's higher-level thread API
The debugcon logger is now separate from logger::output, and is instead
a kernel-internal thread that watches for logs and prints them to the
deubcon device.
In prep for the coming change to keep log entries as a true ring buffer,
move the log buffer from bss into its own memory section.
Related changes in this commit:
- New vm_area_ring, which maps a set of pages twice to allow easy linear
reading of data from a ring buffer when it wraps around the end.
- logger_init() went away, and the logger ctor is called from
mem::initialize()
- Instead of an event object, the logger just has a bare wait_queue
- util::counted::from template type changed slightly to allow easy
conversion from an intptr_t as well as a pointer
- Previously added debugcon_logger code removed - this will be added in
a separate file in a followup commit
I often want to use util::counted as a bool, like with a regular
pointer. There was some basic support for that, but it didn't cover
every case - now it should.
Restructuring paging into an object that carries its page cache with it
and makes for simpler code. Program loading is also changed to not copy
the pages loaded from the file into new pages - we can impose a new
constraint that anything loaded by boot have a simple, page-aligned
layout so that we can just map the existing pages into the right
addresses. Also included are some linker script changes to help
accommodate this.
The symbol table needs to be passed to the panic handler very early in
the kernel, loading it in init is far less useful. Return it to the boot
directory and remove it from the initrd.
I added util::format as a replacement for other printf implementations
last year, but it sat there only being used by the kernel all this time.
Now I've templated it so that it can be used by the bootloader, and
removed printf from panic.serial as well.
Using `-fvisibility=hidden` when building the kernel, and then
`--discard-all` when stripping it, we shave almost 100KiB off of the
resulting ELF file.
Also dropped some unused symbols from the linker script, and rearranged
the sections so that the file is able to be mapped directly into memory
instead of having each section copied.
