Remove ELF and initrd loading from the kernel. The bootloader now loads
the initial programs, as it does with the kernel. Other files that were
in the initrd are now on the ESP, and non-program files are just passed
as modules.
Instead of making every callsite that may make a thread do a blocking
operation also invoke the scheduler, move that logic into thread
implementation - if the thread is blocking and is the current thread,
call schedule().
Related changes in this commit:
- Also make exiting threads and processes call the scheduler when
blocking.
- Threads start blocked, and get automatically added to the scheduler's
blocked list.
A check was added in scheduler::prune() which defers deleting threads
and processes if they're the current ones. However, they were still
getting removed from the block list, so they were being leaked.
As mentioned in the last commit, with processes owning spaces, there was
a weird extra space in the "kernel" process that owns the kernel
threads. Now we use that space as the global kernel space, and don't
create a separate one.
vm_space and page_table continue to take over duties from
page_manager:
- creation and deletion of address spaces / pml4s
- cross-address-space copies for endpoints
- taking over pml4 ownership from process
Also fixed the bug where the wrong process was being set in the cpu
data.
To solve: now the kernel process has its own vm_space which is not
g_kernel_space.
Defer from calling process::thread_exited() in scheduler::prune() if the
thread in question is the currently-executing thread, so that we don't
blow away the stack we're executing on. The next call to prune will pick
up the exited thread.
The "fake" stdout channel is now being passed in the new j6_process_init
structure to processes, and nulldrv now uses it to print a message to
the console.
The scheduler singleton was getting constructed twice, once at static
time and then again in main(). Make the singleton a pointer so we only
construct it once.
Previously we added startup_bonus to work around a segfault happening
when we preemted a newly created process instead of letting it give up
the CPU. Bug is not longer occuring, though that makes me nervous.
Implement the syscalls necessary for threads to create other threads in
their same process. This involved rearranging a number of syscalls, as
well as implementing object_wait and a basic implementation of a
process' list of handles.
The TCB is always stored at a constant offset within the thread object.
So instead of carrying an extra pointer, just implement thread::from_tcb
to get the thread.
Re-implent the concept of processes as separate from threads, and as a
kobject API object. Also improve scheduler::prune which was doing some
unnecessary iterations.
A few updates to scheduler policies:
* Grant processes a startup timeslice bonus for time spent loading the
process
* Grant processes a small fraction of a timeslice for yielding the CPU
with time left
Create a clock class which can be queried for current timestamp in
nanoseconds. Also implements a simple HPET class as one possible clock
source.
Tags: time
The scheduler again tracks remaining timeslice. Timeslices are bigger,
but once a process uses all of its timeslice, it's demoted and
replenished at the next priority. The scheduler also tracks the last
time a process ran, and promotes it if it's been starved for twice its
full timeslice.
TODO: replenish a small amount of timeslice each time a process is run,
so that more interactive processes keep their priorities.
Instead of many timer interrupts and decrementing a process' remaining
quanta, change to setting a single timer for when a process should be
preempted. If it uses its whole timeslice, demote it. If it uses less
than half before blocking, promote it. Determine timeslice based on
priority as well.
This change also required changing the apic timer interface to be purely
interval (in microseconds) based instead of its previous interval/tick
hybrid.
Many kernel objects had to keep a hold of refrences to allocators in
order to pass them on down the call chain. Remove those explicit
refrences and use `operator new`, `operator delete`, and define new
`kalloc` and `kfree`.
Also remove `slab_allocator` and replace it with a new mixin for slab
allocation, `slab_allocated`, that overrides `operator new` and
`operator free` for its subclass.
Remove some no longer used related headers, `buddy_allocator.h` and
`address_manager.h`
Tags: memory
This commit makes several fundamental changes to memory handling:
- the frame allocator is now only an allocator for free frames, and does
not track used frames.
- the frame allocator now stores its free list inside the free frames
themselves, as a hybrid stack/span model.
- This has the implication that all frames must currently fit within
the offset area.
- kutil has a new allocator interface, which is the only allowed way for
any code outside of src/kernel to allocate. Code under src/kernel
_may_ use new/delete, but should prefer the allocator interface.
- the heap manager has become heap_allocator, which is merely an
implementation of kutil::allocator which doles out sections of a given
address range.
- the heap manager now only writes block headers when necessary,
avoiding page faults until they're actually needed
- page_manager now has a page fault handler, which checks with the
address_manager to see if the address is known, and provides a frame
mapping if it is, allowing heap manager to work with its entire
address size from the start. (Currently 32GiB.)
There are a lot of under the hood changes here:
- Move syscalls to be a dispatch table, defined by syscalls.inc
- Don't need a full process state (push_all) in syscalls now
- In push_all, define REGS instead of using offsets
- Save TWO stack pointers as well as current saved stack pointer in TCB:
- rsp0 is the base of the kernel stack for interrupts
- rsp3 is the saved user stack from cpu_data
- Update syscall numbers in nulldrv
- Some asm-debugging enhancements to the gdb script
- fork() still not working
- More sensible stack tracer, in C++ (no symbols yet)
- Was forgetting to add null frame to new kernel stacks
- __kernel_assert was using an old vector
- A GP fault will only print its associated table entry
Processes can now wait on signals/children/time. There is no clock
currently so "time" is just a monotonically increating tick count. Added
a SLEEP syscall to test this waiting/waking.
The syscall/sysret instructions don't swap stacks. This was bad but
passable until syscalls caused the scheduler to run, and scheduling a
task that paused due to interrupt.
Adding a new (hopefully temporary) syscall interrupt `int 0xee` to allow
me to test syscalls without stack issues before I tackle the
syscall/sysret issue.
Also implemented a basic `pause` syscall that causes the calling process
to become unready. Because nothing can wake a process yet, it never
returns.
- Scheduler now has multiple linked_lists of processes at different
priorities
- Process structure improvements
- scheduler::tick() and scheduler::schedule() separation
Now any initrd file is treated like a program image and passed to the
loader to load as a process. Very rudimentary elf loading just allocates
pages, copies sections, and sets the ELF's entrypoint as the RIP to
iretq to.
The scheduler's create_process now sets up the stack to iretq into a
load_process function, which will load the process image into memory
from within the process' own virtual memory space. Currently this
loading is just copying the old 'taskA' function from kernel space.
More work on process page tables, including only mapping the last 2 pml4
entries (the highest 1TiB of the address space, ie, kernel space) into a
new table.
Includes the work of actually moving the kernel there, which I had
apparently done in name only previously. Oops.
* It looks like UEFI enables SSE, so we need to tell clang -mno-sse for
now to not use XMM* until we're ready to save them.
* SYSCALL is working from ring3 tasks, calling console printf!
