Finished the VMA kobject and added the related syscalls. Processes can
now allocate memory! Other changes in this commit:
- stop using g_frame_allocator and add frame_allocator::get()
- make sure to release all handles in the process dtor
- fix kutil::map::iterator never comparing to end()
vm_space and page_table continue to take over duties from
page_manager:
- creation and deletion of address spaces / pml4s
- cross-address-space copies for endpoints
- taking over pml4 ownership from process
Also fixed the bug where the wrong process was being set in the cpu
data.
To solve: now the kernel process has its own vm_space which is not
g_kernel_space.
The "fake" stdout channel is now being passed in the new j6_process_init
structure to processes, and nulldrv now uses it to print a message to
the console.
Multiple changes regarding channels. Mainly channels are now stream
based and can handle partial reads or writes. Channels now use the
kernel buffers area with the related buffer_cache. Added a fake stdout
stream channel and kernel task to read its contents to the screen in
preparation for handing channels as stdin/stdout to processes.
We were previously allocating kernel stacks as large objects on the
heap. Now keep track of areas of the kernel stack area that are in use,
and allocate them from there. Also required actually implementing
vm_space::commit(). This still needs more work.
Add a system call to assert signals on a given object, only within the
range of user-settable signals. Also made object_wait return
immediately if any of the given signals are already set.
Add the channel object for sending messages between threads. Currently
no good of passing channels to other threads, but global variables in a
single process work. Currently channels are slow and do double copies,
need to refine more.
Tags: ipc
Implement the syscalls necessary for threads to create other threads in
their same process. This involved rearranging a number of syscalls, as
well as implementing object_wait and a basic implementation of a
process' list of handles.
Re-implent the concept of processes as separate from threads, and as a
kobject API object. Also improve scheduler::prune which was doing some
unnecessary iterations.
Create a clock class which can be queried for current timestamp in
nanoseconds. Also implements a simple HPET class as one possible clock
source.
Tags: time
The scheduler again tracks remaining timeslice. Timeslices are bigger,
but once a process uses all of its timeslice, it's demoted and
replenished at the next priority. The scheduler also tracks the last
time a process ran, and promotes it if it's been starved for twice its
full timeslice.
TODO: replenish a small amount of timeslice each time a process is run,
so that more interactive processes keep their priorities.
There were a few lingering bugs due to places where 510/511 were
hard-coded as the kernel-space PML4 entries. These are now constants
defined in kernel_memory.h instead.
Tags: boot memory paging
The bootloader was previously just passing the memory map as a module,
but the memory map is important enough to want a direct pointer, instead
of having to search the modules.
The `kernel_offset` and `page_offset` had already been updated with
previous bootloader changes, but `kernel_max_heap` had not. Also, make
all the constants `constexpr` instead of `static const` that would live
in multiple TUs.
Set up initial page tables for both the offset-mapped area and the
loaded kernel code and data.
* Got rid of the `loaded_elf` struct - the loader now runs after the
initial PML4 is created and maps the ELF sections itself.
* Copied in the `page_table` and `page_table_indices` from the kernel,
still need to clean this up and extract it into shared code.
* Added `page_table_cache` to the kernel args to pass along free pages
that can be used for initial page tables.
Tags: paging
Adding this now because I'm sure I'll forget later. Make sure to
annotate the entrypoint function pointer as `__attribute__((sysv_abi))`
so that it's not called via ms abi like the rest of the loader.
- The old kernel_args structure is now mostly represented as a series of
'modules' or memory ranges, tagged with a type. An arbitrary number
can be passed to the kernel
- Update bootloader to allocate space for the args header and 10 module
descriptors
This commit makes several fundamental changes to memory handling:
- the frame allocator is now only an allocator for free frames, and does
not track used frames.
- the frame allocator now stores its free list inside the free frames
themselves, as a hybrid stack/span model.
- This has the implication that all frames must currently fit within
the offset area.
- kutil has a new allocator interface, which is the only allowed way for
any code outside of src/kernel to allocate. Code under src/kernel
_may_ use new/delete, but should prefer the allocator interface.
- the heap manager has become heap_allocator, which is merely an
implementation of kutil::allocator which doles out sections of a given
address range.
- the heap manager now only writes block headers when necessary,
avoiding page faults until they're actually needed
- page_manager now has a page fault handler, which checks with the
address_manager to see if the address is known, and provides a frame
mapping if it is, allowing heap manager to work with its entire
address size from the start. (Currently 32GiB.)
Additionally, there were several bug fixes needed to allow this:
- frame_allocator was allocating its frame_blocks from the heap, causing
a circular dependency. Now it gives itself a page on its own when
needed.
- frame_allocator::free was putting any trailing pages in a block back
into the list after the current block, so they would be the next block
iterated to.
- frame_allocator::free was updating the address it was looking for
after freeing some pages, but not the count it was looking for, so it
would eventually free all pages after the initial address.
Removed the frame allocation logic from page_manager and replaced it
with using an instance of frame_allocator instead. This had several
major ripple effects:
- memory_initalize() had to change to support this new world
- Where to map used blocks is now passed as a flag, since blocks don't
track their virtual address anymore
- Instead of the complicated "find N contiguous pages that can be
mapped in with one page table", we now just have the bootloader give
us some (currently 64) pages to use both for tables and scratch
space.
- frame_allocator initialization was split into two steps to allow
mapping used blocks before std::move()ing them over
Added the cpptoml library (and license), and moved to using that for
the initrd manifest. It's now possible to specify the `executable`
flag for files, and the kernel correctly only launches new processes
for the initrd files marked `executable`.
- Create initrd library to support definitions and loading
- Allow tools compiled for the host machine to be built by wscript
- Create makerd tool to build initrd from manifest
- Move screenfont to initrd, so don't load framebuffer initially
