vm_space no longer relies on page_manager to map pages during a page
fault. Other changes that come with this commit:
- C++ standard has been changed to C++17
- enum bitfield operators became constexpr
- enum bifrield operators can take a mix of ints and enum arguments
- added page table flags enum instead of relying on ints
- remove page_table::unmap_table and page_table::unmap_pages
As mentioned in the last commit, with processes owning spaces, there was
a weird extra space in the "kernel" process that owns the kernel
threads. Now we use that space as the global kernel space, and don't
create a separate one.
vm_space and page_table continue to take over duties from
page_manager:
- creation and deletion of address spaces / pml4s
- cross-address-space copies for endpoints
- taking over pml4 ownership from process
Also fixed the bug where the wrong process was being set in the cpu
data.
To solve: now the kernel process has its own vm_space which is not
g_kernel_space.
This is the first commit of several reworking the VM system. The main
focus is replacing page_manager's global functionality with objects
representing individual VM spaces. The main changes in this commit were:
- Adding the (as yet unused) vm_area object, which will be the main
point of control for programs to allocate or share memory.
- Replace the old vm_space with a new one based on state in its page
tables. They will also be containers for vm_areas.
- vm_space takes over from page_manager as the page fault handler
- Commented out the page walking in memory_bootstrap; I'll probably need
to recreate this functionality, but it was broken as it was.
- Split out the page_table.h implementations from page_manager.cpp into
the new page_table.cpp, updated it, and added page_table::iterator as
well.
Add an iterator type to kutil::map, and allow for each loops. Also
unify the compare() signature expected by sorting containers, and fixes
to adding and sorting in kutil::vector.
Replace linearly-indexed vector of handles with new kutil::map. Also
provide thread::current() and process::current() accessors so that every
syscall doesn't need to include the scheduler to deduce the current
process.
Added the kutil::map collection, an open addressing, robinhood hash map
with backwards shift deletes. Also added hash.h with templated
implementations of the FNV-1a 64 bit hash function, and pulled the log2
function out of the heap_allocator code into the new util.h.
The tests clearly haven't even been built in a while. I've added a
helper script to the project root. Also added a kassert() handler that
will allow tests to catch or fail on asserts.
The buffer cache will try to clean up a returned buffer by unmapping it,
but it may have only been committed without ever getting mapped. Allow
for page_out to handle non-fully mapped regions.
Defer from calling process::thread_exited() in scheduler::prune() if the
thread in question is the currently-executing thread, so that we don't
blow away the stack we're executing on. The next call to prune will pick
up the exited thread.
The "fake" stdout channel is now being passed in the new j6_process_init
structure to processes, and nulldrv now uses it to print a message to
the console.
Multiple changes regarding channels. Mainly channels are now stream
based and can handle partial reads or writes. Channels now use the
kernel buffers area with the related buffer_cache. Added a fake stdout
stream channel and kernel task to read its contents to the screen in
preparation for handing channels as stdin/stdout to processes.
Renamed and genericized the stack_cache class to manage any address
range area of buffers or memory regions. Removed singleton and created
some globals that map to different address regions (kernel stacks,
kernel buffers).
Tags: vmem virtual memeory
When committing an area of vmem and splitting from a larger block, the
block that is returned was set to the unknown state, and the leading
block was incorrectly set to the desired state.
Also remove extra unused thread ctor.
Added the functionality for unreseve and uncommit to vm_space. Also
added the documented but not implemented ability to pass a 0 address to
get any address region with the given size.
The bootloader now creates all PD tables in kernel space, so remove
memory_bootstrap.cpp code that dealt with cases where there was no PD
for a given range, and kassert that all PDs exist.
Also deal with the case where the final PD exists, which never committed
the last address range.
We were previously allocating kernel stacks as large objects on the
heap. Now keep track of areas of the kernel stack area that are in use,
and allocate them from there. Also required actually implementing
vm_space::commit(). This still needs more work.
The scheduler singleton was getting constructed twice, once at static
time and then again in main(). Make the singleton a pointer so we only
construct it once.
Process PML4s all point their high (kernelspace) entries at the same set
of PDs, but that copying only happens on process creation. New PDs added
to the kernel PML4 won't get shared among other PML4s. This change
instantiates empty PDs for all PML4 entries in the higher half to make
sure this can't happen.
When moving heap allocation order calculation to use __builtin_clz, we
based that calculation off of the length requested, not the totla length
including the memory header size.
Previously we were using a less efficient loop method of finding the
appropriate block size order, now we use __builtin_clz, which should use
the CPU's clz instruction if it's available.
Heap allocator is a buddy allocator that deals with power-of-two block
sizes. Previously it referred to both a number of bytes and an order of
magnitude as a 'size'. Rename functions and variables referring to
orders of magnitude to 'order'.
Tags: pedantry
Previously we added startup_bonus to work around a segfault happening
when we preemted a newly created process instead of letting it give up
the CPU. Bug is not longer occuring, though that makes me nervous.
Add a system call to assert signals on a given object, only within the
range of user-settable signals. Also made object_wait return
immediately if any of the given signals are already set.
Add the channel object for sending messages between threads. Currently
no good of passing channels to other threads, but global variables in a
single process work. Currently channels are slow and do double copies,
need to refine more.
Tags: ipc
Implement the syscalls necessary for threads to create other threads in
their same process. This involved rearranging a number of syscalls, as
well as implementing object_wait and a basic implementation of a
process' list of handles.
The TCB is always stored at a constant offset within the thread object.
So instead of carrying an extra pointer, just implement thread::from_tcb
to get the thread.
Re-implent the concept of processes as separate from threads, and as a
kobject API object. Also improve scheduler::prune which was doing some
unnecessary iterations.
Calling `spinwait()` was hanging due to improper computation of the
clock rate because justin did a dumb at math. Also the period can be
greater than 1ns, so the clock's units were updated to microseconds.
When loading ELF headers (as opposed to sections), the `file_size` of
the data may be smaller than the `mem_size` of the section to be loaded
in memory. Don't blindly copy `mem_size` bytes from the ELF file, but
instead only `file_size`, then zero the rest.
Tags: elf loader
