Create a new usermode program, srv.init, and have it read the initial
module_page args sent to it by the bootloader. Doesn't yet do anything
useful but sets up the way for loading the rest of the programs from
srv.init.
Other (mostly) related changes:
- bootloader: The allocator now has a function for allocating init
modules out of a modules_page slab. Also changed how the allocator is
initialized and passes the allocation register and modules_page list
to efi_main().
- bootloader: Expose the simple wstrlen() to the rest of the program
- bootloader: Move check_cpu_supported() to hardware.cpp
- bootloader: Moved program_desc to loader.h and made the loader
functions take it as an argument instead of paths.
- kernel: Rename the system_map_mmio syscall to system_map_phys, and
stop having it default those VMAs to having the vm_flags::mmio flag.
Added a new flag mask, vm_flags::driver_mask, so that drivers can be
allowed to ask for the MMIO flag.
- kernel: Rename load_simple_process() to load_init_server() and got rid
of all the stack setup routines in memory_bootstrap.cpp and task.s
- Fixed formatting in config/debug.toml, undefined __linux and other
linux-specific defines, and got rid of _LIBCPP_HAS_THREAD_API_EXTERNAL
because that's just not true.
The bootloader's load_program was reproducing all loadable program
header sections into new pages. Now only do that for sections containing
BSS sections (eg, where file size and mem size do not match).
The bootloader relied on the kernel to know which parts of memory to not
allocate over. For the future shift of having the init process load
other processes instead of the kernel, the bootloader needs a mechanism
to just hand the kernel a list of allocations. This is now done through
the new bootloader allocator, which all allocation goes through. Pool
memory will not be tracked, and so can be overwritten - this means the
args structure and its other structures like programs need to be handled
right away, or copied by the kernel.
- Add bootloader allocator
- Implement a new linked-list based set of pages that act as allocation
registers
- Allow for operator new in the bootloader, which goes through the
global allocator for pool memory
- Split memory map and frame accouting code in the bootloader into
separate memory_map.* files
- Remove many includes that could be replaced by forward declaration in
the bootloader
- Add a new global template type, `counted`, which replaces the
bootloader's `buffer` type, and updated kernel args structure to use it.
- Move bootloader's pointer_manipulation.h to the global include dir
- Make offset_iterator try to return references instead of pointers to
make it more consistent with static array iteration
- Implement a stub atexit() in the bootloader to satisfy clang
The kernel's file header has not been verified for a long time. This
change returns file verification to the bootloader to make sure the ELF
loaded in position 0 is actually the kernel.
The kernel::args namespace is really the protocol for initializing the
kernel from the bootloader. Also, the header struct in that namespace
isn't actually a header, but a collection of parameters. This change
renames the namespace to kernel::init and the struct to args.
The previous frame allocator involved a lot of splitting and merging
linked lists and lost all information about frames while they were
allocated. The new allocator is based on an array of descriptor
structures and a bitmap. Each memory map region of allocatable memory
becomes one or more descriptors, each mapping up to 1GiB of physical
memory. The descriptors implement two levels of a bitmap tree, and have
a pointer into the large contiguous bitmap to track individual pages.
To enable setting sections as NX or read-only, the boot program loader
now loads programs as lists of sections, and the kernel args are updated
accordingly. The kernel's loader now just takes a program pointer to
iterate the sections. Also enable NX in IA32_EFER in the bootloader.
The UEFI spec specifically calls out memory types with the high bit set
as being available for OS loaders' custom use. However, it seems many
UEFI firmware implementations don't handle this well. (Virtualbox, and
the firmware on my Intel NUC and Dell XPS laptop to name a few.)
So sadly since we can't rely on this feature of UEFI in all cases, we
can't use it at all. Instead, treat _all_ memory tagged as EfiLoaderData
as possibly containing data that's been passed to the OS by the
bootloader and don't free it yet.
This will need to be followed up with a change that copies anything we
need to save and frees this memory.
See: https://github.com/kiznit/rainbow-os/blob/master/boot/machine/efi/README.md
After exiting UEFI, the bootloader had no way of displaying status to
the user. Now it will display a series of small boxes as a progress bar
along the bottom of the screen if a framebuffer exists. Errors or
warnings during a step will cause that step's box to turn red or orange,
and display bars above it to signal the error code.
This caused the simplification of the error handling system (which was
mostly just calling status_line::fail) and added different types of
status objects.
Now that the ELF loader is known to be working correctly, remove its
extra print statements about section loading to keep the bootloader
output to one screen.
Remove ELF and initrd loading from the kernel. The bootloader now loads
the initial programs, as it does with the kernel. Other files that were
in the initrd are now on the ESP, and non-program files are just passed
as modules.
When loading ELF headers (as opposed to sections), the `file_size` of
the data may be smaller than the `mem_size` of the section to be loaded
in memory. Don't blindly copy `mem_size` bytes from the ELF file, but
instead only `file_size`, then zero the rest.
Tags: elf loader
When `page_entry_iterator` became a template and changed its static shifts
translating virtual address to table indices into a for loop, that loop
was getting the indices backwards (ie, PML4E index was really the PTE
index, and so on).
Tags: paging
* When using the non-allocating version of `get_uefi_mappings` the
length was not getting set. Reworked this function.
* Having `build_kernel_mem_map` from `bootloader_main_uefi` caused it to
get an out of date map key. Moved this function into `efi_main` right
before exiting boot services.
The page table code had been copied mostly verbatim from the kernel, and
was a dense mess. I abstraced the `page_table_indices` class and the old
loop behavior of `map_in` into a new `page_entry_iterator` class, making
both `map_pages` and the initial offset mapping code much cleaner.
Tags: vmem paging
Set up initial page tables for both the offset-mapped area and the
loaded kernel code and data.
* Got rid of the `loaded_elf` struct - the loader now runs after the
initial PML4 is created and maps the ELF sections itself.
* Copied in the `page_table` and `page_table_indices` from the kernel,
still need to clean this up and extract it into shared code.
* Added `page_table_cache` to the kernel args to pass along free pages
that can be used for initial page tables.
Tags: paging
* Non-blocksize-aligned regions could fail to be found. Have the
bootloader load them aligned.
* Consolidating used frame blocks in the bootstrap means these would
have been impossible to free as address space
* mark_permanent wasn't actually removing blocks from the free list
Removed the frame allocation logic from page_manager and replaced it
with using an instance of frame_allocator instead. This had several
major ripple effects:
- memory_initalize() had to change to support this new world
- Where to map used blocks is now passed as a flag, since blocks don't
track their virtual address anymore
- Instead of the complicated "find N contiguous pages that can be
mapped in with one page table", we now just have the bootloader give
us some (currently 64) pages to use both for tables and scratch
space.
- frame_allocator initialization was split into two steps to allow
mapping used blocks before std::move()ing them over
