Create a script that builds a simple-to-read symbol table from the
output of `nm`. Include running that script over the kernel in the
build, and including that output in the initrd.
Tags: callstack debugging
The bootloader now creates all PD tables in kernel space, so remove
memory_bootstrap.cpp code that dealt with cases where there was no PD
for a given range, and kassert that all PDs exist.
Also deal with the case where the final PD exists, which never committed
the last address range.
We were previously allocating kernel stacks as large objects on the
heap. Now keep track of areas of the kernel stack area that are in use,
and allocate them from there. Also required actually implementing
vm_space::commit(). This still needs more work.
The scheduler singleton was getting constructed twice, once at static
time and then again in main(). Make the singleton a pointer so we only
construct it once.
Process PML4s all point their high (kernelspace) entries at the same set
of PDs, but that copying only happens on process creation. New PDs added
to the kernel PML4 won't get shared among other PML4s. This change
instantiates empty PDs for all PML4 entries in the higher half to make
sure this can't happen.
When moving heap allocation order calculation to use __builtin_clz, we
based that calculation off of the length requested, not the totla length
including the memory header size.
Previously we were using a less efficient loop method of finding the
appropriate block size order, now we use __builtin_clz, which should use
the CPU's clz instruction if it's available.
Heap allocator is a buddy allocator that deals with power-of-two block
sizes. Previously it referred to both a number of bytes and an order of
magnitude as a 'size'. Rename functions and variables referring to
orders of magnitude to 'order'.
Tags: pedantry
Previously we added startup_bonus to work around a segfault happening
when we preemted a newly created process instead of letting it give up
the CPU. Bug is not longer occuring, though that makes me nervous.
Add a system call to assert signals on a given object, only within the
range of user-settable signals. Also made object_wait return
immediately if any of the given signals are already set.
Add the channel object for sending messages between threads. Currently
no good of passing channels to other threads, but global variables in a
single process work. Currently channels are slow and do double copies,
need to refine more.
Tags: ipc
Implement the syscalls necessary for threads to create other threads in
their same process. This involved rearranging a number of syscalls, as
well as implementing object_wait and a basic implementation of a
process' list of handles.
The TCB is always stored at a constant offset within the thread object.
So instead of carrying an extra pointer, just implement thread::from_tcb
to get the thread.
Re-implent the concept of processes as separate from threads, and as a
kobject API object. Also improve scheduler::prune which was doing some
unnecessary iterations.
Calling `spinwait()` was hanging due to improper computation of the
clock rate because justin did a dumb at math. Also the period can be
greater than 1ns, so the clock's units were updated to microseconds.
When loading ELF headers (as opposed to sections), the `file_size` of
the data may be smaller than the `mem_size` of the section to be loaded
in memory. Don't blindly copy `mem_size` bytes from the ELF file, but
instead only `file_size`, then zero the rest.
Tags: elf loader
A few updates to scheduler policies:
* Grant processes a startup timeslice bonus for time spent loading the
process
* Grant processes a small fraction of a timeslice for yielding the CPU
with time left
Create a clock class which can be queried for current timestamp in
nanoseconds. Also implements a simple HPET class as one possible clock
source.
Tags: time
The scheduler again tracks remaining timeslice. Timeslices are bigger,
but once a process uses all of its timeslice, it's demoted and
replenished at the next priority. The scheduler also tracks the last
time a process ran, and promotes it if it's been starved for twice its
full timeslice.
TODO: replenish a small amount of timeslice each time a process is run,
so that more interactive processes keep their priorities.
Instead of many timer interrupts and decrementing a process' remaining
quanta, change to setting a single timer for when a process should be
preempted. If it uses its whole timeslice, demote it. If it uses less
than half before blocking, promote it. Determine timeslice based on
priority as well.
This change also required changing the apic timer interface to be purely
interval (in microseconds) based instead of its previous interval/tick
hybrid.
The bip buffer implementation was not initializing it's write-state
member, which was causing logs to always fail when the logger was not in
immediate mode.
Removing the `allocator.h` file defining the `kutil::allocator`
interface, now that explicit allocators are not being passed around.
Also removed the unused `frame_allocator::raw_allocator` class and
`kutil::invalid_allocator` object.
Tags: memory
Reformat the cpu_features.inc file and add the `in_hv` feature that is
supposedly set by hypervisors when running in emulation. QEMU does not
set it.
Tags: cpuid
Before global constructors were working, I had added a hack to zero out
the static vector member of in `slab_allocated`. Now that they are
working, this can be removed.
Back when the bootloader could only mirror higher half memory linearly
to the lower half, the kernel couldn't be loaded at the beginning of
kernel space because it was unlikely to find enough pages there, so it
was loaded at +1MiB in kernel space. Now the bootloader can map any
pages to the necessary locations, the offset can be removed.
Tags: elf linker virtual memory
Look up the global constructor list that the linker outputs, and run
them all. Required creation of the `kutil::no_construct` template for
objects that are constructed before the global constructors are run.
Also split the `memory_initialize` function into two - one for just
those objects that need to happen before the global ctors, and one
after.
Tags: memory c++
Many kernel objects had to keep a hold of refrences to allocators in
order to pass them on down the call chain. Remove those explicit
refrences and use `operator new`, `operator delete`, and define new
`kalloc` and `kfree`.
Also remove `slab_allocator` and replace it with a new mixin for slab
allocation, `slab_allocated`, that overrides `operator new` and
`operator free` for its subclass.
Remove some no longer used related headers, `buddy_allocator.h` and
`address_manager.h`
Tags: memory
With the new bootloader changes that use clang to directly build an EFI
application, the last piece of GNU-EFI that was used (the linker script)
is no longer necessary.
Thanks for being a great starting point, GNU-EFI!
Eventually the UEFI headers should be brought in from their own project,
but for now, like the other projects under external/, these are being
copied into this repository.
Tags: boot uefi
The last bug getting back to par with master - looks like there might be
threading issues with the logger task at the moment. Turning it off for
now.
Tags: log bug todo
There were a few lingering bugs due to places where 510/511 were
hard-coded as the kernel-space PML4 entries. These are now constants
defined in kernel_memory.h instead.
Tags: boot memory paging
GDB works far better now with QEMU's `-S` flag. No longer does it
complain about changing the target from 32 to 64 bits. Get rid of the
old `waiting` loop and `sleep` call in the GDB config for the kernel.
Tags: debugging
`kutil::vector` was calling `operator delete []` on memory that had not
been allocated with `operator new []`, and so was deleting the wrong
pointer.
Tags: bug memory allocator
The `kernel_main()` had a lot change out from under it with the
bootloader changes. This change brings most of it back in line with the
new kernel arguments.
Tags: pml4 paging boot
Created a new `memory_initialize()` function that uses the new-style
kernel args structure from the new bootloader.
Additionally:
* Fixed a hard-coded interrupt EOI address that didn't work with new
memory locations
* Make the `page_manager::fault_handler()` automatically grant pages
in the kernel heap
Tags: boot page fault
At some point, `init_console()` ended up not being before the first
usage of some `log::` functions, which were jumping off into garbage.
Tags: initialization boot
The bootloader was previously just passing the memory map as a module,
but the memory map is important enough to want a direct pointer, instead
of having to search the modules.
The `kernel_offset` and `page_offset` had already been updated with
previous bootloader changes, but `kernel_max_heap` had not. Also, make
all the constants `constexpr` instead of `static const` that would live
in multiple TUs.
When `page_entry_iterator` became a template and changed its static shifts
translating virtual address to table indices into a for loop, that loop
was getting the indices backwards (ie, PML4E index was really the PTE
index, and so on).
Tags: paging
