In order to allow the bootloader to do preliminary CPUID validation
while UEFI is still handling displaying information to the user, split
most of the kernel's CPUID handling into a library to be used by both
kernel and boot.
Several changes were needed to make this work:
- Update the page_table::flags to understand memory caching types
- Set up the PAT MSR to add the WC option
- Make page-offset area mapped as WT
- Add all the MTRR and PAT MSRs, and log the MTRRs for verification
- Add a vm_area flag for write_combining
The UEFI spec specifically calls out memory types with the high bit set
as being available for OS loaders' custom use. However, it seems many
UEFI firmware implementations don't handle this well. (Virtualbox, and
the firmware on my Intel NUC and Dell XPS laptop to name a few.)
So sadly since we can't rely on this feature of UEFI in all cases, we
can't use it at all. Instead, treat _all_ memory tagged as EfiLoaderData
as possibly containing data that's been passed to the OS by the
bootloader and don't free it yet.
This will need to be followed up with a change that copies anything we
need to save and frees this memory.
See: https://github.com/kiznit/rainbow-os/blob/master/boot/machine/efi/README.md
The endpoint receive syscalls can block and then write to userspace
memory. Since the current address space may be different after blocking,
make sure to only actually write to the user memory after returning to
the syscall handler - pass values that are on the syscall handler stack
deeper into the kernel.
It was not consistent how processes got handles to themselves or their
threads, ending up with double entries. Now make such handles automatic
and expose them with new self_handle() methods.
If there's no video, do as we did before, otherwise route logs to the fb
driver instead. (Need to clean this up to just have a log consumer
general interface?) Also added a "scrollback" class to fb driver and
updated the system_get_log syscall.
Moved old PSF parsing code from kernel, and switched to embedding whole
PSF instead of just glyph data to make font class the same code paths
for both cases.
Move process init from each process needing a main.s with _start to
crt0.s in libc. Also change to a sysv-like initial stack with a
j6-specific array of initialization values after the program arguments.
Create a new framebuffer driver. Also hackily passing frame buffer size
in the list of init handles to all processes and mapping the framebuffer
into all processes. Changed bootloader passing frame buffer as a module
to its own struct.
The console's putc() was looking for CRs and if it saw one, appending an
LF. The output was only writing LFs, though, so instead what's needed is
to look for LFs, and if it sees one, insert a CR first.
In order to implement capabilities on system resources like IRQs so that
they may be restricted to drivers only, add a new 'system' kobject type,
and move the bind_irq functionality from endpoint to system.
Also fix some stack bugs passing the initial handles to a program.
- Add a tag field to all endpoint messages, which doubles as a
notification field
- Add a endpoint_bind_irq syscall to enable an endpoint to listen for
interrupt notifications. This mechanism needs to change.
- Add a temporary copy of the serial port code to nulldrv, and let it
take responsibility for COM2
Remove ELF and initrd loading from the kernel. The bootloader now loads
the initial programs, as it does with the kernel. Other files that were
in the initrd are now on the ESP, and non-program files are just passed
as modules.
The vm_area_shared type of VMA used to track mappings in a separate
array, which doubled information and wasted space. This was no longer
used, and is now removed.
Instead of making every callsite that may make a thread do a blocking
operation also invoke the scheduler, move that logic into thread
implementation - if the thread is blocking and is the current thread,
call schedule().
Related changes in this commit:
- Also make exiting threads and processes call the scheduler when
blocking.
- Threads start blocked, and get automatically added to the scheduler's
blocked list.
The allowed flag was janky and easy to get lost when doing page table
manipulation. All allocation goes throug vm_area now, so 'allowed' can
be dropped.
The rcx register is used by the function call ABI for the 4th argument,
but is also clobbered by SYSCALL to hold the IP. The r10 register is
caller-saved but not part of the ABI, so stash rcx there when crossing
the syscall boundary.
The vm_space allow() functionality was a bit janky; using VMAs for all
regions would be a lot cleaner. To that end, this change:
- Adds a "static array" ctor to kutil::vector for setting the kernel
address space's VMA list. This way a kernel heap VMA can be created
without the heap already existing.
- Splits vm_area into different subclasses depending on desired behavior
- Splits out the concept of vm_mapper which maps vm_areas to vm_spaces,
so that some kinds of VMA can be inherently single-space
- Implements VMA resizing so that userspace can grow allocations.
- Obsolete page_table_indices is removed
Also, the following bugs were fixed:
- kutil::map iterators on empty maps no longer break
- memory::page_count was doing page-align, not page-count
See: Github bug #242
See: [frobozz blog post](https://jsix.dev/posts/frobozz/)
Tags:
Finished the VMA kobject and added the related syscalls. Processes can
now allocate memory! Other changes in this commit:
- stop using g_frame_allocator and add frame_allocator::get()
- make sure to release all handles in the process dtor
- fix kutil::map::iterator never comparing to end()
Added the syscalls/helpers.h file to templatize common kobject syscall
operations. Also moved most syscall implementations to using
process::current() and thread::current() instead of asking the
scheduler.
A check was added in scheduler::prune() which defers deleting threads
and processes if they're the current ones. However, they were still
getting removed from the block list, so they were being leaked.
vm_space no longer relies on page_manager to map pages during a page
fault. Other changes that come with this commit:
- C++ standard has been changed to C++17
- enum bitfield operators became constexpr
- enum bifrield operators can take a mix of ints and enum arguments
- added page table flags enum instead of relying on ints
- remove page_table::unmap_table and page_table::unmap_pages
As mentioned in the last commit, with processes owning spaces, there was
a weird extra space in the "kernel" process that owns the kernel
threads. Now we use that space as the global kernel space, and don't
create a separate one.
vm_space and page_table continue to take over duties from
page_manager:
- creation and deletion of address spaces / pml4s
- cross-address-space copies for endpoints
- taking over pml4 ownership from process
Also fixed the bug where the wrong process was being set in the cpu
data.
To solve: now the kernel process has its own vm_space which is not
g_kernel_space.
This is the first commit of several reworking the VM system. The main
focus is replacing page_manager's global functionality with objects
representing individual VM spaces. The main changes in this commit were:
- Adding the (as yet unused) vm_area object, which will be the main
point of control for programs to allocate or share memory.
- Replace the old vm_space with a new one based on state in its page
tables. They will also be containers for vm_areas.
- vm_space takes over from page_manager as the page fault handler
- Commented out the page walking in memory_bootstrap; I'll probably need
to recreate this functionality, but it was broken as it was.
- Split out the page_table.h implementations from page_manager.cpp into
the new page_table.cpp, updated it, and added page_table::iterator as
well.
Add an iterator type to kutil::map, and allow for each loops. Also
unify the compare() signature expected by sorting containers, and fixes
to adding and sorting in kutil::vector.
Replace linearly-indexed vector of handles with new kutil::map. Also
provide thread::current() and process::current() accessors so that every
syscall doesn't need to include the scheduler to deduce the current
process.
The buffer cache will try to clean up a returned buffer by unmapping it,
but it may have only been committed without ever getting mapped. Allow
for page_out to handle non-fully mapped regions.
Defer from calling process::thread_exited() in scheduler::prune() if the
thread in question is the currently-executing thread, so that we don't
blow away the stack we're executing on. The next call to prune will pick
up the exited thread.
The "fake" stdout channel is now being passed in the new j6_process_init
structure to processes, and nulldrv now uses it to print a message to
the console.
Multiple changes regarding channels. Mainly channels are now stream
based and can handle partial reads or writes. Channels now use the
kernel buffers area with the related buffer_cache. Added a fake stdout
stream channel and kernel task to read its contents to the screen in
preparation for handing channels as stdin/stdout to processes.
Renamed and genericized the stack_cache class to manage any address
range area of buffers or memory regions. Removed singleton and created
some globals that map to different address regions (kernel stacks,
kernel buffers).
Tags: vmem virtual memeory
