This makes the job of the kernel easier when marking module pages as
used in the frame allocator. This will also help when sending modules
over to the init process.
The previous frame allocator involved a lot of splitting and merging
linked lists and lost all information about frames while they were
allocated. The new allocator is based on an array of descriptor
structures and a bitmap. Each memory map region of allocatable memory
becomes one or more descriptors, each mapping up to 1GiB of physical
memory. The descriptors implement two levels of a bitmap tree, and have
a pointer into the large contiguous bitmap to track individual pages.
To enable setting sections as NX or read-only, the boot program loader
now loads programs as lists of sections, and the kernel args are updated
accordingly. The kernel's loader now just takes a program pointer to
iterate the sections. Also enable NX in IA32_EFER in the bootloader.
There was previously no good way to block log-display tasks, either the
fb driver or the kernel log task. Now the system object has a signal
(j6_signal_system_has_log) that gets asserted when the log is written
to.
Now that the spinwait bug is fixed, the raised time for APIC calibration
can be put back to a lower value. It was previously raised thinking more
time would get a more accurate result -- but accuracy was not the issue.
The endpoint syscalls endpoint_recv and endpoint_sendrecv gained new
local stack variables for calling into possibly blocking endpoint
functions, but the len variable was being initialized to 0 instead of
the incoming buffer size.
The amount of spurious IRQ activity on real hardware severely slows down
the system (minutes per frame instead of frames per second). There's no
reason to unmask all of them from the get-go before they're set up to be
handled.
In order to allow the bootloader to do preliminary CPUID validation
while UEFI is still handling displaying information to the user, split
most of the kernel's CPUID handling into a library to be used by both
kernel and boot.
Several changes were needed to make this work:
- Update the page_table::flags to understand memory caching types
- Set up the PAT MSR to add the WC option
- Make page-offset area mapped as WT
- Add all the MTRR and PAT MSRs, and log the MTRRs for verification
- Add a vm_area flag for write_combining
The UEFI spec specifically calls out memory types with the high bit set
as being available for OS loaders' custom use. However, it seems many
UEFI firmware implementations don't handle this well. (Virtualbox, and
the firmware on my Intel NUC and Dell XPS laptop to name a few.)
So sadly since we can't rely on this feature of UEFI in all cases, we
can't use it at all. Instead, treat _all_ memory tagged as EfiLoaderData
as possibly containing data that's been passed to the OS by the
bootloader and don't free it yet.
This will need to be followed up with a change that copies anything we
need to save and frees this memory.
See: https://github.com/kiznit/rainbow-os/blob/master/boot/machine/efi/README.md
The endpoint receive syscalls can block and then write to userspace
memory. Since the current address space may be different after blocking,
make sure to only actually write to the user memory after returning to
the syscall handler - pass values that are on the syscall handler stack
deeper into the kernel.
It was not consistent how processes got handles to themselves or their
threads, ending up with double entries. Now make such handles automatic
and expose them with new self_handle() methods.
If there's no video, do as we did before, otherwise route logs to the fb
driver instead. (Need to clean this up to just have a log consumer
general interface?) Also added a "scrollback" class to fb driver and
updated the system_get_log syscall.
Moved old PSF parsing code from kernel, and switched to embedding whole
PSF instead of just glyph data to make font class the same code paths
for both cases.
Move process init from each process needing a main.s with _start to
crt0.s in libc. Also change to a sysv-like initial stack with a
j6-specific array of initialization values after the program arguments.
Create a new framebuffer driver. Also hackily passing frame buffer size
in the list of init handles to all processes and mapping the framebuffer
into all processes. Changed bootloader passing frame buffer as a module
to its own struct.
The console's putc() was looking for CRs and if it saw one, appending an
LF. The output was only writing LFs, though, so instead what's needed is
to look for LFs, and if it sees one, insert a CR first.
In order to implement capabilities on system resources like IRQs so that
they may be restricted to drivers only, add a new 'system' kobject type,
and move the bind_irq functionality from endpoint to system.
Also fix some stack bugs passing the initial handles to a program.
- Add a tag field to all endpoint messages, which doubles as a
notification field
- Add a endpoint_bind_irq syscall to enable an endpoint to listen for
interrupt notifications. This mechanism needs to change.
- Add a temporary copy of the serial port code to nulldrv, and let it
take responsibility for COM2
Remove ELF and initrd loading from the kernel. The bootloader now loads
the initial programs, as it does with the kernel. Other files that were
in the initrd are now on the ESP, and non-program files are just passed
as modules.
The vm_area_shared type of VMA used to track mappings in a separate
array, which doubled information and wasted space. This was no longer
used, and is now removed.
Instead of making every callsite that may make a thread do a blocking
operation also invoke the scheduler, move that logic into thread
implementation - if the thread is blocking and is the current thread,
call schedule().
Related changes in this commit:
- Also make exiting threads and processes call the scheduler when
blocking.
- Threads start blocked, and get automatically added to the scheduler's
blocked list.
The allowed flag was janky and easy to get lost when doing page table
manipulation. All allocation goes throug vm_area now, so 'allowed' can
be dropped.
The rcx register is used by the function call ABI for the 4th argument,
but is also clobbered by SYSCALL to hold the IP. The r10 register is
caller-saved but not part of the ABI, so stash rcx there when crossing
the syscall boundary.
The vm_space allow() functionality was a bit janky; using VMAs for all
regions would be a lot cleaner. To that end, this change:
- Adds a "static array" ctor to kutil::vector for setting the kernel
address space's VMA list. This way a kernel heap VMA can be created
without the heap already existing.
- Splits vm_area into different subclasses depending on desired behavior
- Splits out the concept of vm_mapper which maps vm_areas to vm_spaces,
so that some kinds of VMA can be inherently single-space
- Implements VMA resizing so that userspace can grow allocations.
- Obsolete page_table_indices is removed
Also, the following bugs were fixed:
- kutil::map iterators on empty maps no longer break
- memory::page_count was doing page-align, not page-count
See: Github bug #242
See: [frobozz blog post](https://jsix.dev/posts/frobozz/)
Tags:
Finished the VMA kobject and added the related syscalls. Processes can
now allocate memory! Other changes in this commit:
- stop using g_frame_allocator and add frame_allocator::get()
- make sure to release all handles in the process dtor
- fix kutil::map::iterator never comparing to end()
Added the syscalls/helpers.h file to templatize common kobject syscall
operations. Also moved most syscall implementations to using
process::current() and thread::current() instead of asking the
scheduler.
A check was added in scheduler::prune() which defers deleting threads
and processes if they're the current ones. However, they were still
getting removed from the block list, so they were being leaked.
vm_space no longer relies on page_manager to map pages during a page
fault. Other changes that come with this commit:
- C++ standard has been changed to C++17
- enum bitfield operators became constexpr
- enum bifrield operators can take a mix of ints and enum arguments
- added page table flags enum instead of relying on ints
- remove page_table::unmap_table and page_table::unmap_pages
