Update init to use the new xoroshiro RNG to create a random load address for dynamic executables.
Also pass an address past the end of the loader in the loader_arg to use when loading dynamic
dependencies.
ld.so will now go through all DT_NEEDED entries in the dynamic table and load and relocate
those shared libraries as well. Lazy linking of functions via the PLT is not yet supported,
all PLT entries are looked up ahead of time by ld.so.
Passing a size of 0 in to vma_resize will now not attempt to alter the VMA size, but will
still put the size into the passed-in pointer. Using this allows querying the size of a
VMA without changing it.
Stacks were not 16-byte aligned when coming into j6::thread::init_proc,
make sure it aligns stacks for SSE by adding the `force_align_arg_pointer`
attribute.
While waiting for the main panicing CPU to finish, don't spin the CPU
without using `asm("pause")` to lower power consumption. Some panics can
get stuck in this state and oh man do my fans spin up.
In the CRT startup code, when linked in a PIC executable, jumps to
`__init_libj6`, `__init_libc`, `main`, and `exit` were not linked
correctly. They needed a bit more support for looking up the GOT, and
getting the symbol address out of it.
Now libutil has a `got.inc` file for inclusion in asm code that needs to
reference symbols from the GOT.
This change allows the `vma_map` and `vma_create_map` syscalls to map to
addresses other than the one specified, and therefore makes the address
parameter to those syscalls `inout` in order to return the mapped
address.
Also add the `exact` flag for specifying that mapping needs to be done
at the exact address given. If the mapping collides with another, the
new `j6_err_collision` error is returned.
Added an `API` macro in `j6/api.h` that expands to mark the given
declaration as a default-visible symbol. Also change `format` and
`vformat` to non-template functions, and make calls to `main`, `exit`,
and the library init functions in `_start` GOT-relative.
The `driver_main` sinature was an alternate signature for `main`
implemented with weak symbols, but it causes linking issues when not
statically linked, and drivers are going to work differently soon
anyway. Just get rid of it for now.
Bonnibel will now build dynamic libraries when they're dependencies for
non-statically linked modules. It will also copy those shared libraries
into the initrd image for programs being copied into the image.
This commit includes a number of changes to enable loading of PIE
executables:
- The loader in srv.init checks for a `PT_INTERP` segment in the program
its loading, and if it exists, loads the specified interpreter and
passes control to it instead of the program itself.
- Added ld.so the dynamic linker executable and set it as the
interpreter for all user-target programs.
- Program initial stack changed again to now contain a number of
possible tagged structures, including a new one for ld.so's arguments,
and for passing handles tagged with protocol ids.
- Added a stub for a new VFS protocol. Unused so far, but srv.init will
need to serve VFS requests from ld.so once I transition libraries to
shared libs for user-target programs. (Right now all executables are
PIE but statically linked, so they only need internal relocations.)
- Added 16 and 8 bit variants of `util::bitset`. This ended up not being
used, but could be useful.
Going back to letting mailboxes use variable-length data. Note that this
requires extra copies, so shared memory channels should be used for
anything in the hot path. But this allows better RPC over mailboxes and
other flexibility.
Other changes:
- added a j6::proto::sl::client class to act as a service locator
client, instead of duplicating that code in every program.
- moved protocol ids into j6/tables/protocols.inc so that C++ clients
can easily have their own API
This change allows for parameters in definition files to have the "list"
option while also needing the handle verification. The verify function
will now iterate through the list checking capabilities and types on
every valid handle in the list.
Remove the `-lc++`, `-lc++abi`, and `-lunwind` options from the user
target, where they should be handled automatically. (ie, we're not using
`-nostdlib` or its bretheren.)
Add extra info to the NASM `global` directive to specify the sizes of
these symbols, mostly so they look right in `nm` or `readelf` and don't
trick me into thinking something is wrong.
In order to allow -fpic and -fpie in the user target, move init to it's
own target -- it needs its own special build rules to make it loadable
by boot.
My `REP.MOVSB` `memcpy` implementation had marked its C++ variable
constraints as output instead of input, causing the compiler to emit
code to copy the values of `$rsi` and `$rdi` back into the `src` and
`dst` pointers, so after the copy `dst` pointed to the memory just
beyond what had been copied.
Very few places actually used the return value from `memcpy`, so this
went unnoticed for a bit..
This is the second of two big changes to clean up includes throughout
the project. Since I've started using clangd with Neovim and using
VSCode's intellisense, my former strategy of copying all header files
into place in `build/include` means that the real files don't show up in
`compile_commands.json` and so display many include errors when viewing
those header files in those tools.
That setup was mostly predicated on a desire to keep directory depths
small, but really I don't think paths like `src/libraries/j6/j6` are
much better than `src/libraries/j6/include/j6`, and the latter doesn't
have the aforementioned issues, and is clearer to the casual observer as
well.
Some additional changes:
- Added a new module flag `copy_headers` for behavior similar to the old
style, but placing headers in `$module_dir/include` instead of the
global `build/include`. This was needed for external projects that
don't follow the same source/headers folder structure - in this case,
`zstd`.
- There is no longer an associated `headers.*.ninja` for each
`module.*.ninja` file, as only parsed headers need to be listed; this
functionality has been moved back into the module's ninja file.
This is the first of two rather big changes to clean up includes
throughout the project. In this commit, the implicit semi-dependency on
libc that bonnibel adds to every module is removed. Previously, I was
sloppy with includes of libc headers and include directory order. Now,
the freestanding headers from libc are split out into libc_free, and an
implicit real dependency is added onto this module, unless `no_libc` is
set to `True`. The full libc needs to be explicitly specified as a
dependency to be used.
Several things needed to change in order to do this:
- Many places use `memset` or `memcpy` that cannot depend on libc. The
kernel has basic implementations of them itself for this reason. Now
those functions are moved into the lower-level `j6/memutils.h`, and
libc merely references them. Other modules are now free to reference
those functions from libj6 instead.
- The kernel's `assert.h` was renamed kassert.h (matching its `kassert`
function) so that the new `util/assert.h` can use `__has_include` to
detect it and make sure the `assert` macro is usable in libutil code.
- Several implementation header files under `__libj6/` also moved under
the new libc_free.
- A new `include_phase` property has been added to modules for Bonnibel,
which can be "normal" (default) or "late" which uses `-idirafter`
instead of `-I` for includes.
- Since `<utility>` and `<new>` are not freestanding, implementations of
`remove_reference`, `forward`, `move`, and `swap` were added to the
`util` namespace to replace those from `std`, and `util/new.h` was
added to declare `operator new` and `operator delete`.
When keeping track of addresses to give to the kernel for channel shared
memory, double the channel's size since the kernel will double-map the
area as a ring buffer.
These are some changes I made to debug tooling while tracking down the
bugfix in the previous commit.
Each `scripts/debug_*_alloc.gdb` script has gdb output a `*_allocs.txt`
file, which in turn can be parsed by the `scripts/parse_*_allocs.py`
script to find errors.
In the heap allocator, new blocks allocated directly for an allocate
request (instead of indirectly as part of a block split) would only set
their order in the tracking map, not their free flag. This left
uninitialized data in the block info map, which thus meant it was marked
as free for looking up for merges. (Not for allocations, since the block
didn't actually appear in the free list.)
I spent some time getting VSCode debugging working. Now I can use VSCode
on windows to work on jsix in Linux (WSL) and launch and debug it within
QEMU. So many layers but it works pretty nicely!
Now threads inherit their MXCSR (sans exception state bits) SIMD
settings from their creator. By default, all exceptions are masked, and
both "to zero" flags are set.
Initial support for XSAVE, but not XSAVEOPT or XSAVEC:
- Enable XSAVE and set up xcr0 for all CPUs
- Allocate XSAVE area for all non-kernel threads
- Call XSAVE and XRSTOR on task switch
Well god damnit, when i converted the `cpu::cpu_id::regs` struct to a
union, i was super sloppy and forgot to wrap the existing fields in
their own anonymous struct. I have been wrong about CPUID vales for
ages.
The libc CRT _start function had a stray pop left in it, which was
causing the stack to never be 16-byte aligned and thus causing crashes
when SSE instructions were called.
