justin/jsix_import
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: justin:v0.5.0
Branches Tags
justin:main justin:feature/multi-arch justin:feature/dynamic_linking justin:framebuffer-issue justin:fb-driver justin:vdso
justin:v0.8.0 justin:v0.7.0 justin:v0.6.0 justin:v0.5.0 justin:v0.4.0 justin:v0.3.0 justin:v0.2.2 justin:v0.2.1 justin:v0.2.0 justin:v0.1.3 justin:v0.1.2 justin:v0.1.1 justin:v0.1.0
..
compare: justin:fb-driver
Branches Tags
justin:feature/multi-arch justin:main justin:feature/dynamic_linking justin:framebuffer-issue justin:fb-driver justin:vdso
justin:v0.8.0 justin:v0.7.0 justin:v0.6.0 justin:v0.5.0 justin:v0.4.0 justin:v0.3.0 justin:v0.2.2 justin:v0.2.1 justin:v0.2.0 justin:v0.1.3 justin:v0.1.2 justin:v0.1.1 justin:v0.1.0

31 Commits
v0.5.0 ... fb-driver

Author SHA1 Message Date
Justin C. Miller
2be70976ca Merge branch 'real-hardware' into fb-driver 2021-01-18 13:38:51 -08:00
Justin C. Miller
699fc57e0a [boot] Log address of new table pages 
Since it's often needed when debugging between the bootloader and
kernel, log the address of the table pages the bootloader allocated.
 2021-01-18 13:36:05 -08:00
Justin C. Miller
f3bb2e5259 [boot] Save commented-out mem map dumping code 
This is often needed, so I'm commiting it commented out.
 2021-01-18 13:35:01 -08:00
Justin C. Miller
34120fc4c1 [cpu] Split cpuid validation into separate lib 
In order to allow the bootloader to do preliminary CPUID validation
while UEFI is still handling displaying information to the user, split
most of the kernel's CPUID handling into a library to be used by both
kernel and boot.
 2021-01-18 13:26:45 -08:00
Justin C. Miller
e52fd8eacf [libc] Attempt to speed up memcpy for aligned mem 
Copy long-by-long instead of byte-by-byte if both pointers are similarly
aligned.
 2021-01-17 20:54:38 -08:00
Justin C. Miller
a97073848c [fb] Use double-buffering in fb driver 
Allocate and use a back buffer, so that draws to the screen are always a
single memcpy()
 2021-01-17 20:53:41 -08:00
Justin C. Miller
03b2d0dac7 [kernel] Set framebuffer to write-combining 
Several changes were needed to make this work:

- Update the page_table::flags to understand memory caching types
- Set up the PAT MSR to add the WC option
- Make page-offset area mapped as WT
- Add all the MTRR and PAT MSRs, and log the MTRRs for verification
- Add a vm_area flag for write_combining
 2021-01-17 20:49:47 -08:00
Justin C. Miller
cfeeba4400 [kenrel] Ensure page tables are zeroed before use 
I forgot to zero out pages used for page tables, which didn't come back
to bite me until testing on physical hardware..
 2021-01-17 10:35:19 -08:00
Justin C. Miller
45b52633bb [boot] Add scanline size to fb boot message 
Scanline size can differ from horizontal resolution in some
framebuffers. This isn't currently handled, but at least log it so
it's visible if this lack of handling is a potential error.
 2021-01-17 10:28:54 -08:00
Justin C. Miller
b9af081a44 [boot] Don't use custom UEFI memory types 
The UEFI spec specifically calls out memory types with the high bit set
as being available for OS loaders' custom use. However, it seems many
UEFI firmware implementations don't handle this well. (Virtualbox, and
the firmware on my Intel NUC and Dell XPS laptop to name a few.)

So sadly since we can't rely on this feature of UEFI in all cases, we
can't use it at all. Instead, treat _all_ memory tagged as EfiLoaderData
as possibly containing data that's been passed to the OS by the
bootloader and don't free it yet.

This will need to be followed up with a change that copies anything we
need to save and frees this memory.

See: https://github.com/kiznit/rainbow-os/blob/master/boot/machine/efi/README.md
 2021-01-17 10:26:24 -08:00
Justin C. Miller
d75b9c22d4 [boot] Don't use custom UEFI memory types 
The UEFI spec specifically calls out memory types with the high bit set
as being available for OS loaders' custom use. However, it seems many
UEFI firmware implementations don't handle this well. (Virtualbox, and
the firmware on my Intel NUC and Dell XPS laptop to name a few.)

So sadly since we can't rely on this feature of UEFI in all cases, we
can't use it at all. Instead, treat _all_ memory tagged as EfiLoaderData
as possibly containing data that's been passed to the OS by the
bootloader and don't free it yet.

This will need to be followed up with a change that copies anything we
need to save and frees this memory.

See: https://github.com/kiznit/rainbow-os/blob/master/boot/machine/efi/README.md
 2021-01-08 22:40:30 -08:00
Justin C. Miller
e20c53f193 [boot] Add framebuffer progress bar 
After exiting UEFI, the bootloader had no way of displaying status to
the user. Now it will display a series of small boxes as a progress bar
along the bottom of the screen if a framebuffer exists. Errors or
warnings during a step will cause that step's box to turn red or orange,
and display bars above it to signal the error code.

This caused the simplification of the error handling system (which was
mostly just calling status_line::fail) and added different types of
status objects.
 2021-01-08 22:25:37 -08:00
Justin C. Miller
6d4a32b6e8 [boot] List the detected framebuffer in boot log 
List information about the detected framebuffer device (or lack thereof)
in the bootloader log messages.
 2021-01-06 23:29:31 -08:00
Justin C. Miller
63a5c2da00 Merge branch 'fb-driver' of github.com:justinian/jsix into fb-driver 2021-01-06 23:26:26 -08:00
Justin C. Miller
bdcd0c2fda [kernel] Clean up process::exit 
Make process::exit slightly more resilient to being called again.
 2021-01-06 23:25:48 -08:00
Justin C. Miller
1be929b9d5 [fb] Simplify scrollback line counting 
Using a start and a count was redundant when we know how many lines are
in the buffer already.
 2021-01-06 23:20:29 -08:00
Justin C. Miller
d11dd0c3f9 [kernel] Fix memory clobbering from endpoint 
The endpoint receive syscalls can block and then write to userspace
memory. Since the current address space may be different after blocking,
make sure to only actually write to the user memory after returning to
the syscall handler - pass values that are on the syscall handler stack
deeper into the kernel.
 2021-01-06 23:16:16 -08:00
Justin C. Miller
e08e00790f [kernel] Give processes and threads self handles 
It was not consistent how processes got handles to themselves or their
threads, ending up with double entries. Now make such handles automatic
and expose them with new self_handle() methods.
 2021-01-06 23:14:39 -08:00
Justin C. Miller
8b3356e9d8 [kutil] Remove uint64_t hash_node specialization 
Using a hash of zero to signal an empty slot doesn't play nice with the
hash_node specialization that uses the key for the hash, when 0 is a
common key.

I thought it would be ok, that it'd just be something to remember. But
then I used 0 as a key anyway, so clearly it was a bad idea.
 2021-01-06 23:09:50 -08:00
Justin C. Miller
cd30126f17 [boot] Reduce loader spam 
Now that the ELF loader is known to be working correctly, remove its
extra print statements about section loading to keep the bootloader
output to one screen.
 2021-01-06 11:34:34 -08:00
Justin C. Miller
2c444dccd6 [build] Remove fake terminal.elf 
A fake terminal.elf (copy of nulldrv.elf) was added to test the loader.
Now that there actually are multiple programs to load, remove the fake
one.
 2021-01-06 11:32:45 -08:00
Justin C. Miller
bd490c4c7b [scripts] Ignore demangle errors building sym table 
For some reason, cxxfilt fails to demangle some names on some systems.
Instead of failing the build process, just skip those symbols.
 2021-01-06 11:30:40 -08:00
Justin C. Miller
3a67e461de [scripts] Allow qemu.sh to not remove VGA device 
Added --vga option to qemu.sh to stop it from passing "-vga none" to
qemu. This allows the console version to act like it has a video device.
 2021-01-04 01:00:03 -08:00
Justin C. Miller
972ef39295 [fb] Output klog to fb if video exists 
If there's no video, do as we did before, otherwise route logs to the fb
driver instead. (Need to clean this up to just have a log consumer
general interface?) Also added a "scrollback" class to fb driver and
updated the system_get_log syscall.
 2021-01-03 18:13:41 -08:00
Justin C. Miller
4c41205e73 [fb] Change to embedding PSF file 
Moved old PSF parsing code from kernel, and switched to embedding whole
PSF instead of just glyph data to make font class the same code paths
for both cases.
 2021-01-03 00:08:20 -08:00
Justin C. Miller
f6b4a4a103 [fb] Add default hard-coded font 
For the fb driver to have a font before loading from disk is available,
create a hard-coded font as a byte array.

To create this, added a new scripts/psf_to_cpp.py which also refactored
out much of scripts/parse_font.py into a new shared module
scripts/fontpsf.py.
 2021-01-02 15:52:26 -08:00
Justin C. Miller
327cd93c04 [kernel] Fix vm_space extra deletion 
vm_space::clear() was freeing pages on process exit even when free was
false, and potentially double-freeing some pages.
 2020-12-31 00:59:48 -08:00
Justin C. Miller
0e6b27e741 [kernel] Improve process init 
Move process init from each process needing a main.s with _start to
crt0.s in libc. Also change to a sysv-like initial stack with a
j6-specific array of initialization values after the program arguments.
 2020-12-31 00:57:51 -08:00
Justin C. Miller
db8a14720b [kernel] Rename kernel entrypoint 
The kernel entrypoint being named _start conflicts with userspace
program entrypoints and makes debugging more difficult. Rename it to
_kernel_start.
 2020-12-30 17:56:37 -08:00
Justin C. Miller
87a7293ac1 [tools] Improve j6stack GDB command 
Improve the j6stack command in two ways: first, swap the order of the
arguments, as depth is much more likely to be changed. Second, on any
exception accessing memory in the stack, print the exception and
continue instead of failing the whole command.
 2020-12-30 17:49:27 -08:00
Justin C. Miller
5787aff866 [fb] Create fb driver 
Create a new framebuffer driver. Also hackily passing frame buffer size
in the list of init handles to all processes and mapping the framebuffer
into all processes. Changed bootloader passing frame buffer as a module
to its own struct.
 2020-12-27 18:49:38 -08:00
124 changed files with 1890 additions and 4103 deletions
Expand all files Collapse all files

2
.gitignore vendored
View File

@@ -3,10 +3,8 @@
*.bak
tags
jsix.log
*.out
*.o
*.a
sysroot
.gdb_history
.peru
__pycache__

94
README.md
View File

@@ -1,18 +1,15 @@
![jsix](assets/jsix.svg)
# jsix: A toy OS kernel
# The jsix operating system
**jsix** is a custom multi-core x64 operating system that I am building from
scratch. It's far from finished, or even being usable - see the *Status and
Roadmap* section, below.
**jsix** is the kernel for the hobby OS that I am currently building. It's
far from finished, or even being usable. Instead, it's a sandbox for me to play
with kernel-level code and explore architectures.
The design goals of the project are:
* Modernity - I'm not interested in designing for legacy systems, or running on
all hardware out there. My target is only 64 bit architecutres, and modern
commodity hardware. Currently that means x64 systems with Nehalem or newer
CPUs and UEFI firmware. (See [this list][cpu_features] for the currently
required CPU features.) Eventually I'd like to work on an AArch64 port,
CPUs and UEFI firmware. Eventually I'd like to work on an AArch64 port,
partly to force myself to factor out the architecture-dependent pieces of the
code base.
@@ -20,11 +17,13 @@ The design goals of the project are:
processes as possible, in the microkernel fashion. A sub-goal of this is to
explore where the bottlenecks of such a microkernel are now, and whether
eschewing legacy hardware will let me design a system that's less bogged down
by the traditional microkernel problems.
by the traditional microkernel problems. Given that there are no processes
yet, the kernel is monolithic by default.
* Exploration - I'm really mostly doing this to have fun learning and exploring
modern OS development. Initial feature implementations may temporarily throw
away modular design to allow for exploration of the related hardware.
modern OS development. Modular design may be tossed out (hopefully
temporarily) in some places to allow me to play around with the related
hardware.
A note on the name: This kernel was originally named Popcorn, but I have since
discovered that the Popcorn Linux project is also developing a kernel with that
@@ -32,69 +31,6 @@ name, started around the same time as this project. So I've renamed this kernel
jsix (Always styled _jsix_ or `j6`, never capitalized) as an homage to L4, xv6,
and my wonderful wife.
[cpu_features]: https://github.com/justinian/jsix/blob/master/src/libraries/cpu/include/cpu/features.inc
## Status and Roadmap
The following major feature areas are targets for jsix development:
#### UEFI boot loader
_Done._ The bootloader loads the kernel and initial userspace programs, and
sets up necessary kernel arguments about the memory map and EFI GOP
framebuffer. Possible future ideas:
- take over more init-time functions from the kernel
- rewrite it in Zig
#### Memory
_Virtual memory: Sufficient._ The kernel manages virtual memory with a number
of kinds of `vm_area` objects representing mapped areas, which can belong to
one or more `vm_space` objects which represent a whole virtual memory space.
(Each process has a `vm_space`, and so does the kernel itself.)
Remaining to do:
- TLB shootdowns
- Page swapping
_Physical page allocation: Sufficient._ The current physical page allocator
implementation suses a group of block representing up-to-1GiB areas of usable
memory as defined by the bootloader. Each block has a three-level bitmap
denoting free/used pages.
#### Multitasking
_Sufficient._ The global scheduler object keeps separate ready/blocked lists
per core. Cores periodically attempt to balance load via work stealing.
User-space tasks are able to create threads as well as other processes.
Several kernel-only tasks exist, though I'm trying to reduce that. Eventually
only the timekeeping task should be a separate kernel-only thread.
#### API
_In progress._ User-space tasks are able to make syscalls to the kernel via
fast SYSCALL/SYSRET instructions.
Major tasks still to do:
- The process initialization protocol needs to be re-built entirely.
- Processes' handles to kernel objects need the ability to check capabilities
#### Hardware Support
* Framebuffer driver: _In progress._ Currently on machines with a video
device accessible by UEFI, jsix starts a user-space framebuffer driver that
only prints out kernel logs.
* Serial driver: _To do._ Machines without a video device should have a
user-space log output task like the framebuffer driver, but currently this
is done inside the kernel.
* USB driver: _To do_
* AHCI (SATA) driver: _To do_
## Building
jsix uses the [Ninja][] build tool, and generates the build files for it with a
@@ -102,7 +38,7 @@ custom tool called [Bonnibel][]. Bonnibel can be installed with [Cargo][], or
downloaded as a prebuilt binary from its Github repository.
[Ninja]: https://ninja-build.org
[Bonnibel]: https://github.com/justinian/bonnibel_rs
[Bonnibel]: https://github.com/justinian/bonnibel
[Cargo]: https://crates.io/crates/bonnibel
Requrirements:
@@ -135,8 +71,8 @@ I personally run this either from a real debian amd64 testing/buster machine or
a windows WSL debian testing/buster installation. The following should be
enough to set up such a system to build the kernel:
sudo apt install qemu-system-x86 nasm clang-10 mtools curl ninja-build
sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-10 1000
sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-10 1000
curl -L -o pb https://github.com/justinian/bonnibel_rs/releases/download/v2.3.0/pb-linux-amd64 && chmod a+x pb
sudo apt install qemu-system-x86 nasm clang-6.0 mtools curl
sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-6.0 1000
sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-6.0 1000
curl -L -o pb https://github.com/justinian/bonnibel/releases/download/2.0.0/pb_linux_amd64 && chmod a+x pb

69
assets/debugging/jsix.elf-gdb.py
View File

@@ -56,77 +56,8 @@ class PrintBacktraceCommand(gdb.Command):
return
class TableWalkCommand(gdb.Command):
def __init__(self):
super().__init__("j6tw", gdb.COMMAND_DATA)
def invoke(self, arg, from_tty):
args = gdb.string_to_argv(arg)
if len(args) < 2:
raise Exception("Must be: j6tw <pml4> <addr>")
pml4 = int(gdb.parse_and_eval(args[0]))
addr = int(gdb.parse_and_eval(args[1]))
indices = [
(addr >> 39) & 0x1ff,
(addr >> 30) & 0x1ff,
(addr >> 21) & 0x1ff,
(addr >> 12) & 0x1ff,
]
names = ["PML4", "PDP", "PD", "PT"]
table_flags = [
(0x0001, "present"),
(0x0002, "write"),
(0x0004, "user"),
(0x0008, "pwt"),
(0x0010, "pcd"),
(0x0020, "accessed"),
(0x0040, "dirty"),
(0x0080, "largepage"),
(0x0100, "global"),
(0x1080, "pat"),
((1<<63), "xd"),
]
page_flags = [
(0x0001, "present"),
(0x0002, "write"),
(0x0004, "user"),
(0x0008, "pwt"),
(0x0010, "pcd"),
(0x0020, "accessed"),
(0x0040, "dirty"),
(0x0080, "pat"),
(0x0100, "global"),
((1<<63), "xd"),
]
flagsets = [table_flags, table_flags, table_flags, page_flags]
table = pml4
entry = 0
for i in range(len(indices)):
entry = int(gdb.parse_and_eval(f'((uint64_t*){table})[{indices[i]}]'))
flagset = flagsets[i]
flag_names = " | ".join([f[1] for f in flagset if (entry & f[0]) == f[0]])
print(f"{names[i]:>4}: {table:016x}")
print(f" index: {indices[i]:3} {entry:016x}")
print(f" flags: {flag_names}")
if (entry & 1) == 0 or (i < 3 and (entry & 0x80)):
break
table = (entry & 0x7ffffffffffffe00) | 0xffffc00000000000
PrintStackCommand()
PrintBacktraceCommand()
TableWalkCommand()
gdb.execute("target remote :1234")
gdb.execute("display/i $rip")

1
assets/jsix.svg
View File

@@ -1 +0,0 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?><!-- Generator: Gravit.io --><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" style="isolation:isolate" viewBox="176.562 356.069 211.11 113" width="211.11pt" height="113pt"><g><g><rect x="176.562" y="356.069" width="211.11" height="113" transform="matrix(1,0,0,1,0,0)" fill="rgb(255,255,255)"/><g><path d=" M 212.981 372.36 L 219.564 376.16 L 226.147 379.961 L 226.147 387.563 L 226.147 395.164 L 219.564 398.965 L 212.981 402.766 L 206.398 398.965 L 199.815 395.164 L 199.815 387.563 L 199.815 379.961 L 206.398 376.16 L 212.981 372.36 L 212.981 372.36 L 212.981 372.36 Z M 256.292 397.366 L 262.875 401.166 L 269.458 404.967 L 269.458 412.569 L 269.458 420.17 L 262.875 423.971 L 256.292 427.772 L 249.709 423.971 L 243.126 420.17 L 243.126 412.569 L 243.126 404.967 L 249.709 401.166 L 256.292 397.366 L 256.292 397.366 Z M 183.622 387.283 L 205.52 374.64 L 227.418 361.997 L 249.316 374.64 L 271.214 387.283 L 271.214 412.569 L 271.214 437.854 L 249.316 450.497 L 227.418 463.14 L 205.52 450.497 L 183.622 437.854 L 183.622 412.569 L 183.622 387.283 L 183.622 387.283 L 183.622 387.283 Z M 241.855 372.36 L 248.438 376.16 L 255.021 379.961 L 255.021 387.563 L 255.021 395.164 L 248.438 398.965 L 241.855 402.766 L 235.272 398.965 L 228.689 395.164 L 228.689 387.563 L 228.689 379.961 L 235.272 376.16 L 241.855 372.36 Z " fill-rule="evenodd" fill="rgb(49,79,128)"/><path d=" M 298.642 379.579 L 291.621 379.579 L 291.621 372.558 L 298.642 372.558 L 298.642 379.579 Z M 285.214 446.718 L 285.214 441.452 L 287.32 441.452 L 287.32 441.452 Q 289.339 441.452 290.524 440.092 L 290.524 440.092 L 290.524 440.092 Q 291.708 438.731 291.708 436.625 L 291.708 436.625 L 291.708 387.039 L 298.729 387.039 L 298.729 436.011 L 298.729 436.011 Q 298.729 440.925 295.921 443.822 L 295.921 443.822 L 295.921 443.822 Q 293.113 446.718 288.286 446.718 L 288.286 446.718 L 285.214 446.718 Z M 306.628 432.676 L 306.628 427.41 L 314.088 427.41 L 314.088 427.41 Q 317.862 427.41 319.573 425.347 L 319.573 425.347 L 319.573 425.347 Q 321.285 423.285 321.285 419.95 L 321.285 419.95 L 321.285 419.95 Q 321.285 417.317 319.705 415.474 L 319.705 415.474 L 319.705 415.474 Q 318.125 413.631 314.966 411.174 L 314.966 411.174 L 314.966 411.174 Q 312.245 408.98 310.621 407.356 L 310.621 407.356 L 310.621 407.356 Q 308.998 405.732 307.813 403.319 L 307.813 403.319 L 307.813 403.319 Q 306.628 400.905 306.628 397.746 L 306.628 397.746 L 306.628 397.746 Q 306.628 393.095 309.744 390.067 L 309.744 390.067 L 309.744 390.067 Q 312.859 387.039 318.125 387.039 L 318.125 387.039 L 325.76 387.039 L 325.76 392.305 L 319.441 392.305 L 319.441 392.305 Q 313.21 392.305 313.21 398.185 L 313.21 398.185 L 313.21 398.185 Q 313.21 400.467 314.615 402.134 L 314.615 402.134 L 314.615 402.134 Q 316.019 403.802 319.003 406.083 L 319.003 406.083 L 319.003 406.083 Q 321.723 408.19 323.479 409.901 L 323.479 409.901 L 323.479 409.901 Q 325.234 411.613 326.463 414.202 L 326.463 414.202 L 326.463 414.202 Q 327.691 416.791 327.691 420.301 L 327.691 420.301 L 327.691 420.301 Q 327.691 426.532 324.4 429.604 L 324.4 429.604 L 324.4 429.604 Q 321.109 432.676 315.141 432.676 L 315.141 432.676 L 306.628 432.676 Z M 342.611 379.579 L 335.59 379.579 L 335.59 372.558 L 342.611 372.558 L 342.611 379.579 Z M 342.611 432.676 L 335.59 432.676 L 335.59 387.039 L 342.611 387.039 L 342.611 432.676 Z M 356.126 432.676 L 348.754 432.676 L 361.392 409.77 L 349.632 387.039 L 356.39 387.039 L 364.639 403.187 L 372.977 387.039 L 379.735 387.039 L 367.974 409.77 L 380.612 432.676 L 373.24 432.676 L 364.639 416.001 L 356.126 432.676 Z " fill="rgb(49,79,128)"/></g></g></g></svg>
Side by Side

Before

Width:  |  Height:  |  Size: 3.6 KiB

6
external/uefi/boot_services.h vendored
View File

@@ -14,10 +14,8 @@
namespace uefi {
namespace bs_impl {
using allocate_pages = status (*)(allocate_type, memory_type, size_t, void**);
using free_pages = status (*)(void*, size_t);
using get_memory_map = status (*)(size_t*, memory_descriptor*, size_t*, size_t*, uint32_t*);
using allocate_pool = status (*)(memory_type, uint64_t, void**);
using free_pool = status (*)(void*);
using handle_protocol = status (*)(handle, const guid*, void**);
using create_event = status (*)(evt, tpl, event_notify, void*, event*);
using exit_boot_services = status (*)(handle, size_t);
@@ -37,10 +35,10 @@ struct boot_services {
// Memory Services
bs_impl::allocate_pages allocate_pages;
bs_impl::free_pages free_pages;
void *free_pages;
bs_impl::get_memory_map get_memory_map;
bs_impl::allocate_pool allocate_pool;
bs_impl::free_pool free_pool;
void *free_pool;
// Event & Timer Services
bs_impl::create_event create_event;

390
external/uefi/networking.h vendored
View File

@@ -1,390 +0,0 @@
#pragma once
#ifndef _uefi_networking_h_
#define _uefi_networking_h_
// This Source Code Form is part of the j6-uefi-headers project and is subject
// to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was
// not distributed with this file, You can obtain one at
// http://mozilla.org/MPL/2.0/.
#include <stdint.h>
#include <uefi/types.h>
namespace uefi {
//
// IPv4 definitions
//
struct ipv4_address
{
uint8_t addr[4];
};
//
// IPv6 definitions
//
struct ipv6_address
{
uint8_t addr[16];
};
struct ip6_address_info
{
ipv6_address address;
uint8_t prefix_length;
};
struct ip6_route_table
{
ipv6_address gateway;
ipv6_address destination;
uint8_t prefix_length;
};
enum class ip6_neighbor_state : int {
incomplete,
reachable,
stale,
delay,
probe,
}
struct ip6_neighbor_cache
{
ipv6_address neighbor;
mac_address link_address;
ip6_neighbor_state state;
};
enum class icmpv6_type : uint8_t
{
dest_unreachable = 0x1,
packet_too_big = 0x2,
time_exceeded = 0x3,
parameter_problem = 0x4,
echo_request = 0x80,
echo_reply = 0x81,
listener_query = 0x82,
listener_report = 0x83,
listener_done = 0x84,
router_solicit = 0x85,
router_advertise = 0x86,
neighbor_solicit = 0x87,
neighbor_advertise = 0x88,
redirect = 0x89,
listener_report_2 = 0x8f,
};
enum class icmpv6_code : uint8_t
{
// codes for icmpv6_type::dest_unreachable
no_route_to_dest = 0x0,
comm_prohibited = 0x1,
beyond_scope = 0x2,
addr_unreachable = 0x3,
port_unreachable = 0x4,
source_addr_failed = 0x5,
route_rejected = 0x6,
// codes for icmpv6_type::time_exceeded
timeout_hop_limit = 0x0,
timeout_reassemble = 0x1,
// codes for icmpv6_type::parameter_problem
erroneous_header = 0x0,
unrecognize_next_hdr = 0x1,
unrecognize_option = 0x2,
};
struct ip6_icmp_type
{
icmpv6_type type;
icmpv6_code code;
};
struct ip6_config_data
{
uint8_t default_protocol;
bool accept_any_protocol;
bool accept_icmp_errors;
bool accept_promiscuous;
ipv6_address destination_address;
ipv6_address station_address;
uint8_t traffic_class;
uint8_t hop_limit;
uint32_t flow_label;
uint32_t receive_timeout;
uint32_t transmit_timeout;
};
struct ip6_mode_data
{
bool is_started;
uint32_t max_packet_size;
ip6_config_data config_data;
bool is_configured;
uint32_t address_count;
ip6_address_info * address_list;
uint32_t group_count;
ipv6_address * group_table;
uint32_t route_count;
ip6_route_table * route_table;
uint32_t neighbor_count;
ip6_neighbor_cache * neighbor_cache;
uint32_t prefix_count;
ip6_address_info * prefix_table;
uint32_t icmp_type_count;
* icmp_type_list;
};
struct ip6_header
{
uint8_t traffic_class_h : 4;
uint8_t version : 4;
uint8_t flow_label_h : 4;
uint8_t traffic_class_l : 4;
uint16_t flow_label_l;
uint16_t payload_length;
uint8_t next_header;
uint8_t hop_limit;
ipv6_address source_address;
ipv6_address destination_address;
} __attribute__ ((packed));
struct ip6_fragment_data
{
uint32_t fragment_length;
void *fragment_buffer;
};
struct ip6_override_data
{
uint8_t protocol;
uint8_t hop_limit;
uint32_t flow_label;
};
struct ip6_receive_data
{
time time_stamp;
event recycle_signal;
uint32_t header_length;
ip6_header *header;
uint32_t data_length;
uint32_t fragment_count;
ip6_fragment_data fragment_table[1];
};
struct ip6_transmit_data
{
ipv6_address destination_address;
ip6_override_data *override_data;
uint32_t ext_hdrs_length;
void *ext_hdrs;
uint8_t next_header;
uint32_t data_length;
uint32_t fragment_count;
ip6_fragment_data fragment_table[1];
};
struct ip6_completion_token
{
event event;
status status;
union {
ip6_receive_data *rx_data;
ip6_transmit_data *tx_data;
} packet;
};
enum class ip6_config_data_type : int
{
interface_info,
alt_interface_id,
policy,
dup_addr_detect_transmits,
manual_address,
gateway,
dns_server,
maximum
};
struct ip6_config_interface_info
{
wchar_t name[32];
uint8_t if_type;
uint32_t hw_address_size;
mac_address hw_address;
uint32_t address_info_count;
ip6_address_info *address_info;
uint32_t route_count;
ip6_route_table *route_table;
};
struct ip6_config_interface_id
{
uint8_t id[8];
};
enum class ip6_config_policy : int
{
manual,
automatic
};
struct ip6_config_dup_addr_detect_transmits
{
uint32_t dup_addr_detect_transmits;
};
struct ip6_config_manual_address
{
ipv6_address address;
bool is_anycast;
uint8_t prefix_length;
};
//
// IP definitions
//
union ip_address
{
uint8_t addr[4];
ipv4_address v4;
ipv6_address v6;
};
//
// HTTP definitions
//
struct httpv4_access_point
{
bool use_default_address;
ipv4_address local_address;
ipv4_address local_subnet;
uint16_t local_port;
};
struct httpv6_access_point
{
ipv6_address local_address;
uint16_t local_port;
};
enum class http_version : int {
v10,
v11,
unsupported,
};
struct http_config_data
{
http_version http_version;
uint32_t time_out_millisec;
bool local_address_is_ipv6;
union {
httpv4_access_point *ipv4_node;
httpv6_access_point *ipv6_node;
} access_point;
};
enum class http_method : int {
get,
post,
patch,
options,
connect,
head,
put,
delete_,
trace,
};
struct http_request_data
{
http_method method;
wchar_t *url;
};
enum class http_status_code : int {
unsupported,
continue_,
switching_protocols,
ok,
created,
accepted,
non_authoritative_information,
no_content,
reset_content,
partial_content,
multiple_choices,
moved_permanently,
found,
see_other,
not_modified,
use_proxy,
temporary_redirect,
bad_request,
unauthorized,
payment_required,
forbidden,
not_found,
method_not_allowed,
not_acceptable,
proxy_authentication_required,
request_time_out,
conflict,
gone,
length_required,
precondition_failed,
request_entity_too_large,
request_uri_too_large,
unsupported_media_type,
requested_range_not_satisfied,
expectation_failed,
internal_server_error,
not_implemented,
bad_gateway,
service_unavailable,
gateway_timeout,
http_version_not_supported,
permanent_redirect, // I hate your decisions, uefi.
};
struct http_response_data
{
http_status_code status_code;
};
struct http_header
{
char *field_name;
char *field_value;
};
struct http_message
{
union {
http_request_data *request;
http_response_data *response;
} data;
size_t header_count;
http_header *headers;
size_t body_length;
void *body;
};
struct http_token
{
event event;
status status;
http_message *message;
};
} // namespace uefi
#endif

3
external/uefi/protos/device_path.h vendored
View File

@@ -16,7 +16,6 @@ struct device_path
{
static constexpr uefi::guid guid{ 0x09576e91,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
uint8_t type;
uint8_t sub_type;
uint16_t length;
@@ -28,4 +27,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_device_path_h_
#endif // _uefi_protos_device_path_h_

4
external/uefi/protos/file.h vendored
View File

@@ -14,7 +14,7 @@ struct file;
struct file
{
inline uefi::status open(file ** new_handle, const wchar_t * file_name, file_mode open_mode, file_attr attributes) {
return _open(this, new_handle, file_name, open_mode, attributes);
@@ -123,4 +123,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_file_h_
#endif // _uefi_protos_file_h_

3
external/uefi/protos/file_info.h vendored
View File

@@ -16,7 +16,6 @@ struct file_info
{
static constexpr uefi::guid guid{ 0x09576e92,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
uint64_t size;
uint64_t file_size;
uint64_t physical_size;
@@ -33,4 +32,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_file_info_h_
#endif // _uefi_protos_file_info_h_

3
external/uefi/protos/graphics_output.h vendored
View File

@@ -17,7 +17,6 @@ struct graphics_output
{
static constexpr uefi::guid guid{ 0x9042a9de,0x23dc,0x4a38,{0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a} };
inline uefi::status query_mode(uint32_t mode_number, uint64_t * size_of_info, uefi::graphics_output_mode_info ** info) {
return _query_mode(this, mode_number, size_of_info, info);
}
@@ -48,4 +47,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_graphics_output_h_
#endif // _uefi_protos_graphics_output_h_

72
external/uefi/protos/http.h vendored
View File

@@ -1,72 +0,0 @@
#pragma once
#ifndef _uefi_protos_http_h_
#define _uefi_protos_http_h_
// This file was auto generated by the j6-uefi-headers project. Please see
// https://github.com/justinian/j6-uefi-headers for more information.
#include <uefi/guid.h>
#include <uefi/types.h>
#include <uefi/networking.h>
namespace uefi {
namespace protos {
struct http;
struct http
{
static constexpr uefi::guid guid{ 0x7a59b29b,0x910b,0x4171,{0x82,0x42,0xa8,0x5a,0x0d,0xf2,0x5b,0x5b} };
static constexpr uefi::guid service_binding{ 0xbdc8e6af,0xd9bc,0x4379,{0xa7,0x2a,0xe0,0xc4,0xe7,0x5d,0xae,0x1c} };
inline uefi::status get_mode_data(uefi::http_config_data * http_config_data) {
return _get_mode_data(this, http_config_data);
}
inline uefi::status configure(uefi::http_config_data * http_config_data) {
return _configure(this, http_config_data);
}
inline uefi::status request(uefi::http_token * token) {
return _request(this, token);
}
inline uefi::status cancel(uefi::http_token * token) {
return _cancel(this, token);
}
inline uefi::status response(uefi::http_token * token) {
return _response(this, token);
}
inline uefi::status poll() {
return _poll(this);
}
protected:
using _get_mode_data_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
_get_mode_data_def _get_mode_data;
using _configure_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
_configure_def _configure;
using _request_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
_request_def _request;
using _cancel_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
_cancel_def _cancel;
using _response_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
_response_def _response;
using _poll_def = uefi::status (*)(uefi::protos::http *);
_poll_def _poll;
public:
};
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_http_h_

93
external/uefi/protos/ip6.h vendored
View File

@@ -1,93 +0,0 @@
#pragma once
#ifndef _uefi_protos_ip6_h_
#define _uefi_protos_ip6_h_
// This file was auto generated by the j6-uefi-headers project. Please see
// https://github.com/justinian/j6-uefi-headers for more information.
#include <uefi/guid.h>
#include <uefi/types.h>
#include <uefi/networking.h>
namespace uefi {
namespace protos {
struct ip6;
struct ip6
{
static constexpr uefi::guid guid{ 0x2c8759d5,0x5c2d,0x66ef,{0x92,0x5f,0xb6,0x6c,0x10,0x19,0x57,0xe2} };
static constexpr uefi::guid service_binding{ 0xec835dd3,0xfe0f,0x617b,{0xa6,0x21,0xb3,0x50,0xc3,0xe1,0x33,0x88} };
inline uefi::status get_mode_data(uefi::ip6_mode_data * ip6_mode_data, uefi::managed_network_config_data * mnp_config_data, uefi::simple_network_mode * snp_config_data) {
return _get_mode_data(this, ip6_mode_data, mnp_config_data, snp_config_data);
}
inline uefi::status configure(uefi::ip6_config_data * ip6_config_data) {
return _configure(this, ip6_config_data);
}
inline uefi::status groups(bool join_flag, uefi::ipv6_address * group_address) {
return _groups(this, join_flag, group_address);
}
inline uefi::status routes(bool delete_route, uefi::ipv6_address * destination, uint8_t prefix_length, uefi::ipv6_address * gateway_address) {
return _routes(this, delete_route, destination, prefix_length, gateway_address);
}
inline uefi::status neighbors(bool delete_flag, uefi::ipv6_address * target_ip6_address, uefi::mac_address * target_link_address, uint32_t timeout, bool override) {
return _neighbors(this, delete_flag, target_ip6_address, target_link_address, timeout, override);
}
inline uefi::status transmit(uefi::ip6_completion_token * token) {
return _transmit(this, token);
}
inline uefi::status receive(uefi::ip6_completion_token * token) {
return _receive(this, token);
}
inline uefi::status cancel(uefi::ip6_completion_token * token) {
return _cancel(this, token);
}
inline uefi::status poll() {
return _poll(this);
}
protected:
using _get_mode_data_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_mode_data *, uefi::managed_network_config_data *, uefi::simple_network_mode *);
_get_mode_data_def _get_mode_data;
using _configure_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_config_data *);
_configure_def _configure;
using _groups_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *);
_groups_def _groups;
using _routes_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uint8_t, uefi::ipv6_address *);
_routes_def _routes;
using _neighbors_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uefi::mac_address *, uint32_t, bool);
_neighbors_def _neighbors;
using _transmit_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
_transmit_def _transmit;
using _receive_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
_receive_def _receive;
using _cancel_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
_cancel_def _cancel;
using _poll_def = uefi::status (*)(uefi::protos::ip6 *);
_poll_def _poll;
public:
};
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_ip6_h_

57
external/uefi/protos/ip6_config.h vendored
View File

@@ -1,57 +0,0 @@
#pragma once
#ifndef _uefi_protos_ip6_config_h_
#define _uefi_protos_ip6_config_h_
// This file was auto generated by the j6-uefi-headers project. Please see
// https://github.com/justinian/j6-uefi-headers for more information.
#include <uefi/guid.h>
#include <uefi/types.h>
#include <uefi/networking.h>
namespace uefi {
namespace protos {
struct ip6_config;
struct ip6_config
{
static constexpr uefi::guid guid{ 0x937fe521,0x95ae,0x4d1a,{0x89,0x29,0x48,0xbc,0xd9,0x0a,0xd3,0x1a} };
inline uefi::status set_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
return _set_data(this, data_type, data_size, data);
}
inline uefi::status get_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
return _get_data(this, data_type, data_size, data);
}
inline uefi::status register_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
return _register_data_notify(this, data_type, event);
}
inline uefi::status unregister_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
return _unregister_data_notify(this, data_type, event);
}
protected:
using _set_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
_set_data_def _set_data;
using _get_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
_get_data_def _get_data;
using _register_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
_register_data_notify_def _register_data_notify;
using _unregister_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
_unregister_data_notify_def _unregister_data_notify;
public:
};
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_ip6_config_h_

36
external/uefi/protos/load_file.h vendored
View File

@@ -1,36 +0,0 @@
#pragma once
#ifndef _uefi_protos_load_file_h_
#define _uefi_protos_load_file_h_
// This file was auto generated by the j6-uefi-headers project. Please see
// https://github.com/justinian/j6-uefi-headers for more information.
#include <uefi/guid.h>
#include <uefi/types.h>
#include <uefi/protos/device_path.h>
namespace uefi {
namespace protos {
struct load_file;
struct load_file
{
static constexpr uefi::guid guid{ {0x56ec3091,0x954c,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
inline uefi::status load_file(uefi::protos::device_path * file_path, bool boot_policy, size_t * buffer_size, void * buffer) {
return _load_file(this, file_path, boot_policy, buffer_size, buffer);
}
protected:
using _load_file_def = uefi::status (*)(uefi::protos::load_file *, uefi::protos::device_path *, bool, size_t *, void *);
_load_file_def _load_file;
public:
};
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_load_file_h_

3
external/uefi/protos/loaded_image.h vendored
View File

@@ -18,7 +18,6 @@ struct loaded_image
{
static constexpr uefi::guid guid{ 0x5b1b31a1,0x9562,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
inline uefi::status unload(uefi::handle image_handle) {
return _unload(image_handle);
}
@@ -46,4 +45,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_loaded_image_h_
#endif // _uefi_protos_loaded_image_h_

41
external/uefi/protos/service_binding.h vendored
View File

@@ -1,41 +0,0 @@
#pragma once
#ifndef _uefi_protos_service_binding_h_
#define _uefi_protos_service_binding_h_
// This file was auto generated by the j6-uefi-headers project. Please see
// https://github.com/justinian/j6-uefi-headers for more information.
#include <uefi/guid.h>
#include <uefi/types.h>
namespace uefi {
namespace protos {
struct service_binding;
struct service_binding
{
inline uefi::status create_child(uefi::handle * child_handle) {
return _create_child(this, child_handle);
}
inline uefi::status destroy_child(uefi::handle child_handle) {
return _destroy_child(this, child_handle);
}
protected:
using _create_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle *);
_create_child_def _create_child;
using _destroy_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle);
_destroy_child_def _destroy_child;
public:
};
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_service_binding_h_

3
external/uefi/protos/simple_file_system.h vendored
View File

@@ -17,7 +17,6 @@ struct simple_file_system
{
static constexpr uefi::guid guid{ 0x0964e5b22,0x6459,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
inline uefi::status open_volume(uefi::protos::file ** root) {
return _open_volume(this, root);
}
@@ -34,4 +33,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_simple_file_system_h_
#endif // _uefi_protos_simple_file_system_h_

3
external/uefi/protos/simple_text_output.h vendored
View File

@@ -17,7 +17,6 @@ struct simple_text_output
{
static constexpr uefi::guid guid{ 0x387477c2,0x69c7,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
inline uefi::status reset(bool extended_verification) {
return _reset(this, extended_verification);
}
@@ -90,4 +89,4 @@ public:
} // namespace protos
} // namespace uefi
#endif // _uefi_protos_simple_text_output_h_
#endif // _uefi_protos_simple_text_output_h_

23
modules.yaml
View File

@@ -12,7 +12,6 @@ modules:
- src/kernel
source:
- src/kernel/apic.cpp
- src/kernel/ap_startup.s
- src/kernel/assert.cpp
- src/kernel/boot.s
- src/kernel/clock.cpp
@@ -25,12 +24,12 @@ modules:
- src/kernel/frame_allocator.cpp
- src/kernel/fs/gpt.cpp
- src/kernel/gdt.cpp
- src/kernel/gdtidt.s
- src/kernel/gdt.s
- src/kernel/hpet.cpp
- src/kernel/idt.cpp
- src/kernel/interrupts.cpp
- src/kernel/interrupts.s
- src/kernel/io.cpp
- src/kernel/loader.s
- src/kernel/log.cpp
- src/kernel/main.cpp
- src/kernel/memory_bootstrap.cpp
@@ -43,7 +42,6 @@ modules:
- src/kernel/objects/system.cpp
- src/kernel/objects/vm_area.cpp
- src/kernel/page_table.cpp
- src/kernel/page_tree.cpp
- src/kernel/pci.cpp
- src/kernel/scheduler.cpp
- src/kernel/serial.cpp
@@ -58,7 +56,7 @@ modules:
- src/kernel/syscalls/thread.cpp
- src/kernel/syscalls/vm_area.cpp
- src/kernel/task.s
- src/kernel/tss.cpp
- src/kernel/vm_mapper.cpp
- src/kernel/vm_space.cpp
boot:
@@ -114,7 +112,6 @@ modules:
- src/libraries/kutil/logger.cpp
- src/libraries/kutil/memory.cpp
- src/libraries/kutil/printf.c
- src/libraries/kutil/spinlock.cpp
cpu:
kind: lib
@@ -122,24 +119,14 @@ modules:
includes:
- src/libraries/cpu/include
source:
- src/libraries/cpu/cpu_id.cpp
- src/libraries/cpu/cpu.cpp
j6:
kind: lib
output: libj6.a
includes:
- src/libraries/j6/include
target: user
source:
- src/libraries/j6/syscalls.s
libc:
kind: lib
output: libc.a
includes:
- src/libraries/libc/include
deps:
- j6
target: user
defines:
- DISABLE_SSE
@@ -158,6 +145,7 @@ modules:
- src/libraries/libc/arch/x86_64/_Exit.s
- src/libraries/libc/arch/x86_64/crt0.s
- src/libraries/libc/arch/x86_64/init_libc.c
- src/libraries/libc/arch/x86_64/syscalls.s
- src/libraries/libc/ctype/isalnum.c
- src/libraries/libc/ctype/isalpha.c
- src/libraries/libc/ctype/isblank.c
@@ -326,7 +314,6 @@ modules:
- src/tests/main.cpp
- src/tests/map.cpp
- src/tests/vector.cpp
overlays:
- url: https://f000.backblazeb2.com/file/jsix-os/sysroot-llvm8-20190706.tar.bz2
path: sysroot

2
src/boot/console.cpp
View File

@@ -55,7 +55,7 @@ console::console(uefi::boot_services *bs, uefi::protos::simple_text_output *out)
m_out->output_string(GIT_VERSION_WIDE);
m_out->set_attribute(uefi::attribute::light_gray);
m_out->output_string(L" booting...\r\n");
m_out->output_string(L" booting...\r\n\n");
if (m_fb.type != kernel::args::fb_type::none) {
wchar_t const * type = nullptr;

9
src/boot/error.cpp
View File

@@ -35,28 +35,27 @@ message(uefi::status status)
}
[[ noreturn ]] static void
cpu_assert(uefi::status s, const wchar_t *message, size_t line)
cpu_assert(uefi::status s, const wchar_t *message)
{
asm volatile (
"movq $0xeeeeeeebadbadbad, %%r8;"
"movq %0, %%r9;"
"movq %1, %%r10;"
"movq %2, %%r11;"
"movq $0, %%rdx;"
"divq %%rdx;"
:
: "r"((uint64_t)s), "r"(message), "r"(line)
: "r"((uint64_t)s), "r"(message)
: "rax", "rdx", "r8", "r9", "r10");
while (1) asm("hlt");
}
[[ noreturn ]] void
raise(uefi::status status, const wchar_t *message, size_t line)
raise(uefi::status status, const wchar_t *message)
{
if(status_line::fail(message, status))
while (1) asm("hlt");
else
cpu_assert(status, message, line);
cpu_assert(status, message);
}

4
src/boot/error.h
View File

@@ -12,7 +12,7 @@ class console;
namespace error {
/// Halt or exit the program with the given error status/message
[[ noreturn ]] void raise(uefi::status status, const wchar_t *message, size_t line = 0);
[[ noreturn ]] void raise(uefi::status status, const wchar_t *message);
const wchar_t * message(uefi::status status);
@@ -28,6 +28,6 @@ void debug_break();
#define try_or_raise(s, m) \
do { \
uefi::status _s = (s); \
if (uefi::is_error(_s)) ::boot::error::raise(_s, (m), __LINE__); \
if (uefi::is_error(_s)) ::boot::error::raise(_s, (m)); \
} while(0)

2
src/boot/fs.cpp
View File

@@ -78,7 +78,7 @@ file::load(uefi::memory_type mem_type)
m_file->read(&size, data),
L"Could not read from file");
return { .size = size, .data = data };
return { .data = data, .size = size };
}
file

13
src/boot/guids.cpp Normal file
View File

@@ -0,0 +1,13 @@
#include "guids.h"
#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) \
EFI_GUID name __attribute__((section(".rodata"))) = {dw, w1, w2, {b1, b2, b3, b4, b5, b6, b7, b8}}
#include "guids.inc"
#undef GUID
int is_guid(EFI_GUID *a, EFI_GUID *b)
{
uint64_t *ai = (uint64_t *)a;
uint64_t *bi = (uint64_t *)b;
return ai[0] == bi[0] && ai[1] == bi[1];
}

8
src/boot/guids.h Normal file
View File

@@ -0,0 +1,8 @@
#pragma once
#include <efi/efi.h>
int is_guid(EFI_GUID *a, EFI_GUID *b);
#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) extern EFI_GUID name
#include "guids.inc"
#undef GUID

11
src/boot/guids.inc Normal file
View File

@@ -0,0 +1,11 @@
GUID(0xeb9d2d30,0x2d88,0x11d3,0x9a,0x16,0x00,0x90,0x27,0x3f,0xc1,0x4d, guid_acpi1);
GUID(0x8868e871,0xe4f1,0x11d3,0xbc,0x22,0x00,0x80,0xc7,0x3c,0x88,0x81, guid_acpi2);
GUID(0x09576e92,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_file_info);
GUID(0x9042a9de,0x23dc,0x4a38,0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a, guid_gfx_out);
GUID(0x964e5b22,0x6459,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_simple_filesystem);
GUID(0x09576e91,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_device_path);
GUID(0x8b843e20,0x8132,0x4852,0x90,0xcc,0x55,0x1a,0x4e,0x4a,0x7f,0x1c, guid_device_path_to_text);
GUID(0x10d0669c,0x9ec6,0x4268,0xbc,0x48,0xff,0x74,0x75,0x21,0xfe,0x07, guid_jsix_vendor);
// vim: ft=c

28
src/boot/hardware.cpp
View File

@@ -37,25 +37,8 @@ find_acpi_table(uefi::system_table *st)
return reinterpret_cast<void*>(acpi1_table);
}
static uint64_t
rdmsr(uint32_t addr)
{
uint32_t low, high;
__asm__ __volatile__ ("rdmsr" : "=a"(low), "=d"(high) : "c"(addr));
return (static_cast<uint64_t>(high) << 32) | low;
}
static void
wrmsr(uint32_t addr, uint64_t value)
{
uint32_t low = value & 0xffffffff;
uint32_t high = value >> 32;
__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
}
void
setup_control_regs()
setup_cr4()
{
uint64_t cr4 = 0;
asm volatile ( "mov %%cr4, %0" : "=r" (cr4) );
@@ -66,15 +49,6 @@ setup_control_regs()
0x020000 | // Enable PCIDs
0;
asm volatile ( "mov %0, %%cr4" :: "r" (cr4) );
// Set up IA32_EFER
constexpr uint32_t IA32_EFER = 0xC0000080;
uint64_t efer = rdmsr(IA32_EFER);
efer |=
0x0001 | // Enable SYSCALL
0x0800 | // Enable NX bit
0;
wrmsr(IA32_EFER, efer);
}
} // namespace hw

4
src/boot/hardware.h
View File

@@ -13,8 +13,8 @@ namespace hw {
/// significant bit set to 1.
void * find_acpi_table(uefi::system_table *st);
/// Enable CPU options in CR4 etc for the kernel starting state.
void setup_control_regs();
/// Enable CPU options in CR4 for the kernel starting state.
void setup_cr4();
} // namespace hw
} // namespace boot

15
src/boot/loader.cpp
View File

@@ -27,7 +27,7 @@ load_file(
fs::file file = disk.open(path);
buffer b = file.load(type);
//console::print(L" Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
console::print(L" Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
return b;
}
@@ -87,9 +87,6 @@ load_program(
bs->set_mem(pages, total_size, 0);
program.base = prog_base;
program.total_size = total_size;
program.num_sections = 0;
for (int i = 0; i < header->ph_num; ++i) {
ptrdiff_t offset = header->ph_offset + i * header->ph_entsize;
const elf::program_header *pheader =
@@ -98,18 +95,14 @@ load_program(
if (pheader->type != elf::PT_LOAD)
continue;
args::program_section &section = program.sections[program.num_sections++];
void *src_start = offset_ptr<void>(data.data, pheader->offset);
void *dest_start = offset_ptr<void>(pages, pheader->vaddr - prog_base);
bs->copy_mem(dest_start, src_start, pheader->file_size);
section.phys_addr = reinterpret_cast<uintptr_t>(dest_start);
section.virt_addr = pheader->vaddr;
section.size = pheader->mem_size;
section.type = static_cast<args::section_flags>(pheader->flags);
}
program.phys_addr = reinterpret_cast<uintptr_t>(pages);
program.size = total_size;
program.virt_addr = prog_base;
program.entrypoint = header->entrypoint;
}

25
src/boot/main.cpp
View File

@@ -8,7 +8,7 @@
#include <stdint.h>
#include "console.h"
#include "cpu/cpu_id.h"
#include "cpu/cpu.h"
#include "error.h"
#include "fs.h"
#include "hardware.h"
@@ -93,8 +93,6 @@ add_module(args::header *args, args::mod_type type, buffer &data)
m.type = type;
m.location = data.data;
m.size = data.size;
change_pointer(m.location);
}
/// Check that all required cpu features are supported
@@ -154,13 +152,18 @@ uefi_preboot(uefi::handle image, uefi::system_table *st)
loader::load_program(program, desc.name, buf, bs);
}
for (unsigned i = 0; i < args->num_modules; ++i) {
args::module &mod = args->modules[i];
change_pointer(mod.location);
}
return args;
}
memory::efi_mem_map
uefi_exit(args::header *args, uefi::handle image, uefi::boot_services *bs)
{
status_line status {L"Exiting UEFI", nullptr, false};
status_line status {L"Exiting UEFI"};
memory::efi_mem_map map =
memory::build_kernel_mem_map(args, bs);
@@ -188,27 +191,17 @@ efi_main(uefi::handle image, uefi::system_table *st)
args->video = con.fb();
status_bar status {con.fb()}; // Switch to fb status display
// Map the kernel to the appropriate address
args::program &kernel = args->programs[0];
for (auto &section : kernel.sections)
if (section.size)
paging::map_section(args, section);
memory::fix_frame_blocks(args);
paging::map_pages(args, kernel.phys_addr, kernel.virt_addr, kernel.size);
kernel::entrypoint kentry =
reinterpret_cast<kernel::entrypoint>(kernel.entrypoint);
status.next();
hw::setup_control_regs();
memory::virtualize(args->pml4, map, st->runtime_services);
status.next();
change_pointer(args);
change_pointer(args->pml4);
change_pointer(args->modules);
change_pointer(args->programs);
hw::setup_cr4();
status.next();
kentry(args);

180
src/boot/memory.cpp
View File

@@ -14,8 +14,6 @@ namespace memory {
using mem_entry = kernel::args::mem_entry;
using mem_type = kernel::args::mem_type;
using frame_block = kernel::args::frame_block;
using kernel::args::frames_per_block;
size_t fixup_pointer_index = 0;
void **fixup_pointers[64];
@@ -109,143 +107,29 @@ can_merge(mem_entry &prev, mem_type type, uefi::memory_descriptor *next)
}
void
get_uefi_mappings(efi_mem_map &map, uefi::boot_services *bs)
get_uefi_mappings(efi_mem_map *map, bool allocate, uefi::boot_services *bs)
{
size_t length = map.total;
size_t length = 0;
uefi::status status = bs->get_memory_map(
&length, map.entries, &map.key, &map.size, &map.version);
map.length = length;
if (status == uefi::status::success)
return;
&length, nullptr, &map->key, &map->size, &map->version);
if (status != uefi::status::buffer_too_small)
error::raise(status, L"Error getting memory map size");
if (map.entries) {
map->length = length;
if (allocate) {
map->length += 10*map->size;
try_or_raise(
bs->free_pool(reinterpret_cast<void*>(map.entries)),
L"Freeing previous memory map space");
}
bs->allocate_pool(
uefi::memory_type::loader_data, map->length,
reinterpret_cast<void**>(&map->entries)),
L"Allocating space for memory map");
map.total = length + 10*map.size;
try_or_raise(
bs->allocate_pool(
uefi::memory_type::loader_data, map.total,
reinterpret_cast<void**>(&map.entries)),
L"Allocating space for memory map");
map.length = map.total;
try_or_raise(
bs->get_memory_map(&map.length, map.entries, &map.key, &map.size, &map.version),
L"Getting UEFI memory map");
}
inline size_t bitmap_size(size_t frames) { return (frames + 63) / 64; }
inline size_t num_blocks(size_t frames) { return (frames + (frames_per_block-1)) / frames_per_block; }
void
build_kernel_frame_blocks(const mem_entry *map, size_t nent, kernel::args::header *args, uefi::boot_services *bs)
{
status_line status {L"Creating kernel frame accounting map"};
size_t block_count = 0;
size_t total_bitmap_size = 0;
for (size_t i = 0; i < nent; ++i) {
const mem_entry &ent = map[i];
if (ent.type != mem_type::free)
continue;
block_count += num_blocks(ent.pages);
total_bitmap_size += bitmap_size(ent.pages) * sizeof(uint64_t);
}
size_t total_size = block_count * sizeof(frame_block) + total_bitmap_size;
frame_block *blocks = nullptr;
try_or_raise(
bs->allocate_pages(
uefi::allocate_type::any_pages,
uefi::memory_type::loader_data,
bytes_to_pages(total_size),
reinterpret_cast<void**>(&blocks)),
L"Error allocating kernel frame block space");
frame_block *next_block = blocks;
for (size_t i = 0; i < nent; ++i) {
const mem_entry &ent = map[i];
if (ent.type != mem_type::free)
continue;
size_t page_count = ent.pages;
uintptr_t base_addr = ent.start;
while (page_count) {
frame_block *blk = next_block++;
bs->set_mem(blk, sizeof(frame_block), 0);
blk->attrs = ent.attr;
blk->base = base_addr;
base_addr += frames_per_block * page_size;
if (page_count >= frames_per_block) {
page_count -= frames_per_block;
blk->count = frames_per_block;
blk->map1 = ~0ull;
bs->set_mem(blk->map2, sizeof(blk->map2), 0xff);
} else {
blk->count = page_count;
unsigned i = 0;
uint64_t b1 = (page_count + 4095) / 4096;
blk->map1 = (1 << b1) - 1;
uint64_t b2 = (page_count + 63) / 64;
uint64_t b2q = b2 / 64;
uint64_t b2r = b2 % 64;
bs->set_mem(blk->map2, b2q, 0xff);
blk->map2[b2q] = (1 << b2r) - 1;
break;
}
}
}
uint64_t *bitmap = reinterpret_cast<uint64_t*>(next_block);
bs->set_mem(bitmap, total_bitmap_size, 0);
for (unsigned i = 0; i < block_count; ++i) {
frame_block &blk = blocks[i];
blk.bitmap = bitmap;
size_t b = blk.count / 64;
size_t r = blk.count % 64;
bs->set_mem(blk.bitmap, b*8, 0xff);
blk.bitmap[b] = (1 << r) - 1;
bitmap += bitmap_size(blk.count);
}
args->frame_block_count = block_count;
args->frame_block_pages = bytes_to_pages(total_size);
args->frame_blocks = blocks;
}
void
fix_frame_blocks(kernel::args::header *args)
{
// Map the frame blocks to the appropriate address
paging::map_pages(args,
reinterpret_cast<uintptr_t>(args->frame_blocks),
::memory::bitmap_start,
args->frame_block_pages,
true, false);
uintptr_t offset = ::memory::bitmap_start -
reinterpret_cast<uintptr_t>(args->frame_blocks);
for (unsigned i = 0; i < args->frame_block_count; ++i) {
frame_block &blk = args->frame_blocks[i];
blk.bitmap = reinterpret_cast<uint64_t*>(
reinterpret_cast<uintptr_t>(blk.bitmap) + offset);
try_or_raise(
bs->get_memory_map(&map->length, map->entries, &map->key, &map->size, &map->version),
L"Getting UEFI memory map");
}
}
@@ -255,7 +139,7 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
status_line status {L"Creating kernel memory map"};
efi_mem_map map;
get_uefi_mappings(map, bs);
get_uefi_mappings(&map, false, bs);
size_t map_size = map.num_entries() * sizeof(mem_entry);
@@ -269,14 +153,14 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
L"Error allocating kernel memory map module space");
bs->set_mem(kernel_map, map_size, 0);
get_uefi_mappings(map, bs);
get_uefi_mappings(&map, true, bs);
size_t nent = 0;
size_t i = 0;
bool first = true;
for (auto desc : map) {
/*
// EFI map dump
console::print(L" eRange %lx (%lx) %x(%s) [%lu]\r\n",
console::print(L" Range %lx (%lx) %x(%s) [%lu]\r\n",
desc->physical_start, desc->attribute, desc->type, memory_type_name(desc->type), desc->number_of_pages);
*/
@@ -292,10 +176,13 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
case uefi::memory_type::boot_services_code:
case uefi::memory_type::boot_services_data:
case uefi::memory_type::conventional_memory:
case uefi::memory_type::loader_data:
type = mem_type::free;
break;
case uefi::memory_type::loader_data:
type = mem_type::pending;
break;
case uefi::memory_type::runtime_services_code:
case uefi::memory_type::runtime_services_data:
type = mem_type::uefi_runtime;
@@ -323,19 +210,18 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
// TODO: validate uefi's map is sorted
if (first) {
first = false;
mem_entry &ent = kernel_map[nent++];
ent.start = desc->physical_start;
ent.pages = desc->number_of_pages;
ent.type = type;
ent.attr = (desc->attribute & 0xffffffff);
kernel_map[i].start = desc->physical_start;
kernel_map[i].pages = desc->number_of_pages;
kernel_map[i].type = type;
kernel_map[i].attr = (desc->attribute & 0xffffffff);
continue;
}
mem_entry &prev = kernel_map[nent - 1];
mem_entry &prev = kernel_map[i];
if (can_merge(prev, type, desc)) {
prev.pages += desc->number_of_pages;
} else {
mem_entry &next = kernel_map[nent++];
mem_entry &next = kernel_map[++i];
next.start = desc->physical_start;
next.pages = desc->number_of_pages;
next.type = type;
@@ -345,19 +231,17 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
// Give just the actually-set entries in the header
args->mem_map = kernel_map;
args->map_count = nent;
args->map_count = i;
/*
// kernel map dump
for (unsigned i = 0; i < nent; ++i) {
for (unsigned i = 0; i < args->map_count; ++i) {
const kernel::args::mem_entry &e = kernel_map[i];
console::print(L" kRange %lx (%lx) %x(%s) [%lu]\r\n",
console::print(L" Range %lx (%lx) %x(%s) [%lu]\r\n",
e.start, e.attr, e.type, kernel_memory_type_name(e.type), e.pages);
}
*/
build_kernel_frame_blocks(kernel_map, nent, args, bs);
get_uefi_mappings(map, bs);
return map;
}

11
src/boot/memory.h
View File

@@ -41,13 +41,12 @@ struct efi_mem_map
using iterator = offset_iterator<desc>;
size_t length; ///< Total length of the map data
size_t total; ///< Total allocated space for map data
size_t size; ///< Size of an entry in the array
size_t key; ///< Key for detecting changes
uint32_t version; ///< Version of the `memory_descriptor` struct
desc *entries; ///< The array of UEFI descriptors
efi_mem_map() : length(0), total(0), size(0), key(0), version(0), entries(nullptr) {}
efi_mem_map() : length(0), size(0), key(0), version(0), entries(nullptr) {}
/// Get the count of entries in the array
inline size_t num_entries() const { return length / size; }
@@ -63,14 +62,6 @@ struct efi_mem_map
/// \returns The uefi memory map used to build the kernel map
efi_mem_map build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs);
/// Create the kernel frame allocation maps
void build_kernel_frame_blocks(
const kernel::args::mem_entry *map, size_t nent,
kernel::args::header *args, uefi::boot_services *bs);
/// Map the frame allocation maps to the right spot and fix up pointers
void fix_frame_blocks(kernel::args::header *args);
/// Activate the given memory mappings. Sets the given page tables live as well
/// as informs UEFI runtime services of the new mappings.
/// \arg pml4 The root page table for the new mappings

56
src/boot/paging.cpp
View File

@@ -15,7 +15,7 @@ using memory::page_size;
using ::memory::pml4e_kernel;
using ::memory::table_entries;
// Flags: 0 0 0 1 0 0 0 0 0 0 0 1 = 0x0101
// Flags: 0 0 0 1 0 0 0 0 0 0 1 1 = 0x0103
// IGN | | | | | | | | +- Present
// | | | | | | | +--- Writeable
// | | | | | | +----- Usermode access (supervisor only)
@@ -26,7 +26,7 @@ using ::memory::table_entries;
// | +---------------- PAT (determining memory type for page)
// +------------------- Global
/// Page table entry flags for entries pointing at a page
constexpr uint64_t page_flags = 0x101;
constexpr uint16_t page_flags = 0x103;
// Flags: 0 0 0 0 1 1 0 0 0 1 0 1 1 = 0x018b
// | IGN | | | | | | | | +- Present
@@ -40,7 +40,7 @@ constexpr uint64_t page_flags = 0x101;
// | +------------------- Global
// +---------------------------- PAT (determining memory type for page)
/// Page table entry flags for entries pointing at a huge page
constexpr uint64_t huge_page_flags = 0x18b;
constexpr uint16_t huge_page_flags = 0x183;
// Flags: 0 0 0 0 0 0 0 0 0 0 1 1 = 0x0003
// IGNORED | | | | | | | +- Present
@@ -52,7 +52,7 @@ constexpr uint64_t huge_page_flags = 0x18b;
// | +-------------- Ignored
// +---------------- Reserved 0 (Table pointer, not page)
/// Page table entry flags for entries pointing at another table
constexpr uint64_t table_flags = 0x003;
constexpr uint16_t table_flags = 0x003;
/// Iterator over page table entries.
template <unsigned D = 4>
@@ -118,7 +118,7 @@ private:
if (!(parent_ent & 1)) {
if (!m_cache_count--)
error::raise(uefi::status::out_of_resources, L"Page table cache empty", 0x7ab1e5);
error::raise(uefi::status::out_of_resources, L"Page table cache empty");
page_table *table = reinterpret_cast<page_table*>(m_page_cache);
m_page_cache = offset_ptr<void>(m_page_cache, page_size);
@@ -191,7 +191,7 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
status_line status(L"Allocating initial page tables");
static constexpr size_t pd_tables = 256; // number of pages for kernelspace PDs
static constexpr size_t extra_tables = 64; // number of extra pages
static constexpr size_t extra_tables = 49; // number of extra pages
// number of pages for kernelspace PDs + PML4
static constexpr size_t kernel_tables = pd_tables + 1;
@@ -212,7 +212,6 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
page_table *pml4 = reinterpret_cast<page_table*>(addr);
args->pml4 = pml4;
args->table_pages = tables_needed;
args->table_count = tables_needed - 1;
args->page_tables = offset_ptr<void>(addr, page_size);
@@ -221,42 +220,27 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
add_kernel_pds(pml4, args->page_tables, args->table_count);
add_offset_mappings(pml4, args->page_tables, args->table_count);
//console::print(L" Set up initial mappings, %d spare tables.\r\n", args->table_count);
}
template <typename E>
constexpr bool has_flag(E set, E flag) {
return
(static_cast<uint64_t>(set) & static_cast<uint64_t>(flag)) ==
static_cast<uint64_t>(flag);
console::print(L" Set up initial mappings, %d spare tables.\r\n", args->table_count);
}
void
map_pages(
kernel::args::header *args,
uintptr_t phys, uintptr_t virt,
size_t count, bool write_flag, bool exe_flag)
size_t size)
{
if (!count)
return;
paging::page_table *pml4 =
reinterpret_cast<paging::page_table*>(args->pml4);
size_t pages = memory::bytes_to_pages(size);
page_entry_iterator<4> iterator{
virt, pml4,
args->page_tables,
args->table_count};
uint64_t flags = page_flags;
if (!exe_flag)
flags |= (1ull << 63); // set NX bit
if (write_flag)
flags |= 2;
while (true) {
*iterator = phys | flags;
if (--count == 0)
*iterator = phys | page_flags;
if (--pages == 0)
break;
iterator.increment();
@@ -264,24 +248,6 @@ map_pages(
}
}
void
map_section(
kernel::args::header *args,
const kernel::args::program_section &section)
{
using kernel::args::section_flags;
size_t pages = memory::bytes_to_pages(section.size);
map_pages(
args,
section.phys_addr,
section.virt_addr,
pages,
has_flag(section.type, section_flags::write),
has_flag(section.type, section_flags::execute));
}
} // namespace paging
} // namespace boot

19
src/boot/paging.h
View File

@@ -38,22 +38,15 @@ void allocate_tables(
/// tables in the current PML4.
void add_current_mappings(page_table *new_pml4);
/// Map physical memory pages to virtual addresses in the given page tables.
/// \arg args The kernel args header, used for the page table cache and pml4
/// \arg section The program section to load
/// Map a physical address to a virtual address in the given page tables.
/// \arg args The kernel args header, used for the page table cache and pml4
/// \arg phys The phyiscal address to map in
/// \arg virt The virtual address to map in
/// \arg size The size in bytes of the mapping
void map_pages(
kernel::args::header *args,
uintptr_t phys, uintptr_t virt,
size_t count, bool write_flag, bool exe_flag);
/// Map a program section in physical memory to its virtual address in the
/// given page tables.
/// \arg args The kernel args header, used for the page table cache and pml4
/// \arg section The program section to load
void map_section(
kernel::args::header *args,
const kernel::args::program_section &section);
size_t bytes);
} // namespace paging
} // namespace boot

4
src/boot/status.cpp
View File

@@ -30,8 +30,7 @@ status *status::s_current = nullptr;
unsigned status::s_current_type = 0;
unsigned status_bar::s_count = 0;
status_line::status_line(const wchar_t *message, const wchar_t *context, bool fails_clean) :
status(fails_clean),
status_line::status_line(const wchar_t *message, const wchar_t *context) :
m_level(level_ok),
m_depth(0),
m_outer(nullptr)
@@ -149,7 +148,6 @@ status_line::do_fail(const wchar_t *message, uefi::status status)
status_bar::status_bar(kernel::args::framebuffer const &fb) :
status(fb.size),
m_outer(nullptr)
{
m_size = (fb.vertical / num_boxes) - 1;

15
src/boot/status.h
View File

@@ -17,8 +17,6 @@ namespace boot {
class status
{
public:
status(bool fails_clean = true) : m_fails_clean(fails_clean) {}
virtual void do_warn(const wchar_t *message, uefi::status status) = 0;
virtual void do_fail(const wchar_t *message, uefi::status status) = 0;
@@ -30,7 +28,7 @@ public:
inline static bool warn(const wchar_t *message, uefi::status status = uefi::status::success) {
if (!s_current) return false;
s_current->do_warn(message, status);
return s_current->m_fails_clean;
return true;
}
/// Set the state to error, and print a message. If the state is already at
@@ -41,15 +39,12 @@ public:
inline static bool fail(const wchar_t *message, uefi::status status) {
if (!s_current) return false;
s_current->do_fail(message, status);
return s_current->m_fails_clean;
return true;
}
protected:
static status *s_current;
static unsigned s_current_type;
private:
bool m_fails_clean;
};
/// Scoped status line reporter. Prints a message and an "OK" if no errors
@@ -64,11 +59,7 @@ public:
/// Constructor.
/// \arg message Description of the operation in progress
/// \arg context If non-null, printed after `message` and a colon
/// \arg fails_clean If true, this object can handle printing failure
status_line(
const wchar_t *message,
const wchar_t *context = nullptr,
bool fails_clean = true);
status_line(const wchar_t *message, const wchar_t *context = nullptr);
~status_line();
virtual void do_warn(const wchar_t *message, uefi::status status) override;

64
src/drivers/fb/main.cpp
View File

@@ -4,11 +4,11 @@
#include "j6/init.h"
#include "j6/errors.h"
#include "j6/flags.h"
#include "j6/signals.h"
#include "j6/syscalls.h"
#include "j6/types.h"
#include <j6libc/syscalls.h>
#include "font.h"
#include "screen.h"
#include "scrollback.h"
@@ -19,7 +19,6 @@ extern "C" {
}
extern j6_handle_t __handle_sys;
extern j6_handle_t __handle_self;
struct entry
{
@@ -33,7 +32,7 @@ struct entry
int
main(int argc, const char **argv)
{
j6_system_log("fb driver starting");
_syscall_system_log("fb driver starting");
size_t initc = 0;
j6_init_value *initv = nullptr;
@@ -42,40 +41,20 @@ main(int argc, const char **argv)
j6_init_framebuffer *fb = nullptr;
for (unsigned i = 0; i < initc; ++i) {
if (initv[i].type == j6_init_desc_framebuffer) {
fb = reinterpret_cast<j6_init_framebuffer*>(initv[i].data);
fb = reinterpret_cast<j6_init_framebuffer*>(initv[i].value);
break;
}
}
if (!fb || fb->addr == 0) {
j6_system_log("fb driver didn't find a framebuffer, exiting");
if (!fb || fb->addr == nullptr) {
_syscall_system_log("fb driver didn't find a framebuffer, exiting");
return 1;
}
j6_handle_t fb_handle = j6_handle_invalid;
uint32_t flags =
j6_vm_flag_write |
j6_vm_flag_write_combine;
j6_status_t s = j6_system_map_mmio(__handle_sys, &fb_handle, fb->addr, fb->size, flags);
if (s != j6_status_ok) {
return s;
}
s = j6_vma_map(fb_handle, __handle_self, fb->addr);
if (s != j6_status_ok) {
return s;
}
const screen::pixel_order order = (fb->flags & 1) ?
screen::pixel_order::bgr8 : screen::pixel_order::rgb8;
screen scr(
reinterpret_cast<void*>(fb->addr),
fb->horizontal,
fb->vertical,
fb->scanline,
order);
screen scr(fb->addr, fb->horizontal, fb->vertical, order);
font fnt;
screen::pixel_t fg = scr.color(0xb0, 0xb0, 0xb0);
@@ -92,28 +71,14 @@ main(int argc, const char **argv)
scrollback scroll(rows, cols);
int pending = 0;
constexpr int pending_threshold = 5;
j6_handle_t sys = __handle_sys;
size_t buffer_size = 0;
void *message_buffer = nullptr;
constexpr int pending_threshold = 10;
char message_buffer[256];
while (true) {
size_t size = buffer_size;
j6_status_t s = j6_system_get_log(sys, message_buffer, &size);
if (s == j6_err_insufficient) {
free(message_buffer);
message_buffer = malloc(size * 2);
buffer_size = size;
continue;
} else if (s != j6_status_ok) {
j6_system_log("fb driver got error from get_log, quitting");
return s;
}
if (size > 0) {
entry *e = reinterpret_cast<entry*>(message_buffer);
size_t size = sizeof(message_buffer);
_syscall_system_get_log(__handle_sys, message_buffer, &size);
if (size != 0) {
entry *e = reinterpret_cast<entry*>(&message_buffer);
size_t eom = e->bytes - sizeof(entry);
e->message[eom] = 0;
@@ -133,7 +98,8 @@ main(int argc, const char **argv)
}
}
j6_system_log("fb driver done, exiting");
_syscall_system_log("fb driver done, exiting");
return 0;
}

24
src/drivers/fb/screen.cpp
View File

@@ -2,15 +2,13 @@
#include <string.h>
#include "screen.h"
screen::screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order) :
m_fb(static_cast<volatile pixel_t *>(addr)),
screen::screen(void *addr, unsigned hres, unsigned vres, pixel_order order) :
m_fb(static_cast<pixel_t *>(addr)),
m_order(order),
m_scanline(scanline),
m_resx(hres),
m_resy(vres)
{
const size_t size = scanline * vres;
m_back = reinterpret_cast<pixel_t*>(malloc(size * sizeof(pixel_t)));
m_back = reinterpret_cast<pixel_t*>(malloc(hres*vres*sizeof(pixel_t)));
}
screen::pixel_t
@@ -34,15 +32,19 @@ screen::color(uint8_t r, uint8_t g, uint8_t b) const
void
screen::fill(pixel_t color)
{
const size_t len = m_scanline * m_resy;
asm volatile ( "rep stosl" : :
"a"(color), "c"(len), "D"(m_back) );
const size_t len = m_resx * m_resy;
for (size_t i = 0; i < len; ++i)
m_back[i] = color;
}
void
screen::draw_pixel(unsigned x, unsigned y, pixel_t color)
{
m_back[x + y * m_resx] = color;
}
void
screen::update()
{
const size_t len = m_scanline * m_resy * sizeof(pixel_t);
asm volatile ( "rep movsb" : :
"c"(len), "S"(m_back), "D"(m_fb) );
memcpy(m_fb, m_back, m_resx*m_resy*sizeof(pixel_t));
}

12
src/drivers/fb/screen.h
View File

@@ -9,7 +9,7 @@ public:
enum class pixel_order : uint8_t { bgr8, rgb8, };
screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order);
screen(void *addr, unsigned hres, unsigned vres, pixel_order order);
unsigned width() const { return m_resx; }
unsigned height() const { return m_resy; }
@@ -17,19 +17,13 @@ public:
pixel_t color(uint8_t r, uint8_t g, uint8_t b) const;
void fill(pixel_t color);
inline void draw_pixel(unsigned x, unsigned y, pixel_t color) {
const size_t index = x + y * m_scanline;
m_back[index] = color;
}
void draw_pixel(unsigned x, unsigned y, pixel_t color);
void update();
private:
volatile pixel_t *m_fb;
pixel_t *m_back;
pixel_t *m_fb, *m_back;
pixel_order m_order;
unsigned m_scanline;
unsigned m_resx, m_resy;
screen() = delete;

18
src/drivers/fb/scrollback.cpp
View File

@@ -12,6 +12,10 @@ scrollback::scrollback(unsigned lines, unsigned cols, unsigned margin) :
m_margin {margin}
{
m_data = reinterpret_cast<char*>(malloc(lines*cols));
m_lines = reinterpret_cast<char**>(malloc(lines*sizeof(char*)));
for (unsigned i = 0; i < lines; ++i)
m_lines[i] = &m_data[i*cols];
memset(m_data, ' ', lines*cols);
}
@@ -23,17 +27,15 @@ scrollback::add_line(const char *line, size_t len)
if (len > m_cols)
len = m_cols;
char *start = m_data + (i * m_cols);
memcpy(start, line, len);
memcpy(m_lines[i], line, len);
if (len < m_cols)
memset(start + len, ' ', m_cols - len);
memset(m_lines[i]+len, ' ', m_cols - len);
}
char *
scrollback::get_line(unsigned i)
{
unsigned line = (i + m_count) % m_rows;
return &m_data[line*m_cols];
return m_lines[(i+m_count)%m_rows];
}
void
@@ -45,12 +47,8 @@ scrollback::render(screen &scr, font &fnt)
const unsigned xstride = (m_margin + fnt.width());
const unsigned ystride = (m_margin + fnt.height());
unsigned start = m_count <= m_rows ? 0 :
m_count % m_rows;
for (unsigned y = 0; y < m_rows; ++y) {
unsigned i = (start + y) % m_rows;
char *line = &m_data[i*m_cols];
char *line = get_line(y);
for (unsigned x = 0; x < m_cols; ++x) {
fnt.draw_glyph(scr, line[x], fg, bg, m_margin+x*xstride, m_margin+y*ystride);
}

1
src/drivers/fb/scrollback.h
View File

@@ -17,6 +17,7 @@ public:
private:
char *m_data;
char **m_lines;
unsigned m_rows, m_cols;
unsigned m_start;
unsigned m_count;

69
src/drivers/nulldrv/main.cpp
View File

@@ -4,7 +4,8 @@
#include "j6/types.h"
#include "j6/errors.h"
#include "j6/signals.h"
#include "j6/syscalls.h"
#include <j6libc/syscalls.h>
#include "io.h"
#include "serial.h"
@@ -14,39 +15,40 @@ extern j6_handle_t __handle_sys;
j6_handle_t endp = j6_handle_invalid;
extern "C" {
void _init_libc(j6_process_init *);
int main(int, const char **);
}
void
thread_proc()
{
j6_system_log("sub thread starting");
_syscall_system_log("sub thread starting");
char buffer[512];
size_t len = sizeof(buffer);
j6_tag_t tag = 0;
j6_status_t result = j6_endpoint_receive(endp, &tag, &len, (void*)buffer);
j6_status_t result = _syscall_endpoint_receive(endp, &tag, &len, (void*)buffer);
if (result != j6_status_ok)
j6_thread_exit(result);
_syscall_thread_exit(result);
j6_system_log("sub thread received message");
_syscall_system_log("sub thread received message");
for (int i = 0; i < len; ++i)
if (buffer[i] >= 'A' && buffer[i] <= 'Z')
buffer[i] += 0x20;
tag++;
result = j6_endpoint_send(endp, tag, len, (void*)buffer);
result = _syscall_endpoint_send(endp, tag, len, (void*)buffer);
if (result != j6_status_ok)
j6_thread_exit(result);
_syscall_thread_exit(result);
j6_system_log("sub thread sent message");
_syscall_system_log("sub thread sent message");
for (int i = 1; i < 5; ++i)
j6_thread_sleep(i*10);
_syscall_thread_sleep(i*10);
j6_system_log("sub thread exiting");
j6_thread_exit(0);
_syscall_system_log("sub thread exiting");
_syscall_thread_exit(0);
}
int
@@ -55,10 +57,10 @@ main(int argc, const char **argv)
j6_handle_t child = j6_handle_invalid;
j6_signal_t out = 0;
j6_system_log("main thread starting");
_syscall_system_log("main thread starting");
for (int i = 0; i < argc; ++i)
j6_system_log(argv[i]);
_syscall_system_log(argv[i]);
void *base = malloc(0x1000);
if (!base)
@@ -68,48 +70,43 @@ main(int argc, const char **argv)
for (int i = 0; i < 3; ++i)
vma_ptr[i*100] = uint64_t(i);
j6_system_log("main thread wrote to memory area");
_syscall_system_log("main thread wrote to memory area");
j6_status_t result = j6_endpoint_create(&endp);
j6_status_t result = _syscall_endpoint_create(&endp);
if (result != j6_status_ok)
return result;
j6_system_log("main thread created endpoint");
_syscall_system_log("main thread created endpoint");
result = j6_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
result = _syscall_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
if (result != j6_status_ok)
return result;
j6_system_log("main thread created sub thread");
_syscall_system_log("main thread created sub thread");
char message[] = "MAIN THREAD SUCCESSFULLY CALLED SENDRECV IF THIS IS LOWERCASE";
size_t size = sizeof(message);
j6_tag_t tag = 16;
result = j6_endpoint_sendrecv(endp, &tag, &size, (void*)message);
result = _syscall_endpoint_sendrecv(endp, &tag, &size, (void*)message);
if (result != j6_status_ok)
return result;
if (tag != 17)
j6_system_log("GOT WRONG TAG FROM SENDRECV");
_syscall_system_log("GOT WRONG TAG FROM SENDRECV");
result = j6_system_bind_irq(__handle_sys, endp, 3);
result = _syscall_system_bind_irq(__handle_sys, endp, 3);
if (result != j6_status_ok)
return result;
j6_system_log(message);
_syscall_system_log(message);
j6_system_log("main thread waiting on child");
result = j6_object_wait(child, -1ull, &out);
_syscall_system_log("main thread waiting on child");
result = _syscall_object_wait(child, -1ull, &out);
if (result != j6_status_ok)
return result;
j6_system_log("main thread creating a new process");
j6_handle_t child_proc = j6_handle_invalid;
result = j6_process_create(&child_proc);
if (result != j6_status_ok)
return result;
j6_system_log("main testing irqs");
_syscall_system_log("main testing irqs");
serial_port com2(COM2);
@@ -123,21 +120,21 @@ main(int argc, const char **argv)
size_t len = 0;
while (true) {
result = j6_endpoint_receive(endp, &tag, &len, nullptr);
result = _syscall_endpoint_receive(endp, &tag, &len, nullptr);
if (result != j6_status_ok)
return result;
if (j6_tag_is_irq(tag))
j6_system_log("main thread got irq!");
_syscall_system_log("main thread got irq!");
}
j6_system_log("main thread closing endpoint");
_syscall_system_log("main thread closing endpoint");
result = j6_object_close(endp);
result = _syscall_object_close(endp);
if (result != j6_status_ok)
return result;
j6_system_log("main thread done, exiting");
_syscall_system_log("main thread done, exiting");
return 0;
}

10
src/include/j6/flags.h
View File

@@ -1,10 +0,0 @@
#pragma once
/// \file flags.h
/// Enums used as flags for syscalls
enum j6_vm_flags {
#define VM_FLAG(name, v) j6_vm_flag_ ## name = v,
#include "j6/tables/vm_flags.inc"
#undef VM_FLAG
j6_vm_flags_MAX
};

19
src/include/j6/init.h
View File

@@ -3,32 +3,25 @@
/// Types used in process and thread initialization
#include <stdint.h>
#include "j6/types.h"
enum j6_init_type { // `value` is a:
j6_init_handle_self, // Handle to the system
j6_init_handle_other, // Handle to this process
j6_init_handle_system, // Handle to the system
j6_init_handle_process, // Handle to this process
j6_init_handle_thread, // Handle to this thread
j6_init_handle_space, // Handle to this process' address space
j6_init_desc_framebuffer // Pointer to a j6_init_framebuffer descriptor
};
struct j6_typed_handle {
enum j6_object_type type;
j6_handle_t handle;
};
struct j6_init_value {
enum j6_init_type type;
union {
struct j6_typed_handle handle;
void *data;
};
uint64_t value;
};
/// Structure defining a framebuffer.
/// `flags` has the following bits:
/// 0-3: Pixel layout. 0000: rgb8, 0001: bgr8
struct j6_init_framebuffer {
uintptr_t addr;
void* addr;
size_t size;
uint32_t vertical;
uint32_t horizontal;

37
src/include/j6/signals.h
View File

@@ -8,42 +8,9 @@
// Signals 16-47 are defined per-object-type
// Event signals
#define j7_signal_event00 (1ull << 16)
#define j7_signal_event01 (1ull << 17)
#define j7_signal_event02 (1ull << 18)
#define j7_signal_event03 (1ull << 19)
#define j7_signal_event04 (1ull << 20)
#define j7_signal_event05 (1ull << 21)
#define j7_signal_event06 (1ull << 22)
#define j7_signal_event07 (1ull << 23)
#define j7_signal_event08 (1ull << 24)
#define j7_signal_event09 (1ull << 25)
#define j7_signal_event10 (1ull << 26)
#define j7_signal_event11 (1ull << 27)
#define j7_signal_event12 (1ull << 28)
#define j7_signal_event13 (1ull << 29)
#define j7_signal_event14 (1ull << 30)
#define j7_signal_event15 (1ull << 31)
#define j7_signal_event16 (1ull << 32)
#define j7_signal_event17 (1ull << 33)
#define j7_signal_event18 (1ull << 34)
#define j7_signal_event19 (1ull << 35)
#define j7_signal_event20 (1ull << 36)
#define j7_signal_event21 (1ull << 37)
#define j7_signal_event22 (1ull << 38)
#define j7_signal_event23 (1ull << 39)
#define j7_signal_event24 (1ull << 40)
#define j7_signal_event25 (1ull << 41)
#define j7_signal_event26 (1ull << 42)
#define j7_signal_event27 (1ull << 43)
#define j7_signal_event28 (1ull << 44)
#define j7_signal_event29 (1ull << 45)
#define j7_signal_event30 (1ull << 46)
#define j7_signal_event31 (1ull << 47)
// Process signals
// System signals
#define j6_signal_system_has_log (1ull << 16)
// Thread signals
// Channel signals
#define j6_signal_channel_can_send (1ull << 16)

12
src/include/j6/tables/object_types.inc
View File

@@ -1,12 +0,0 @@
OBJECT_TYPE( none, 0x00 )
OBJECT_TYPE( system, 0x01 )
OBJECT_TYPE( event, 0x02 )
OBJECT_TYPE( channel, 0x03 )
OBJECT_TYPE( endpoint, 0x04 )
OBJECT_TYPE( vma, 0x05 )
OBJECT_TYPE( process, 0x06 )
OBJECT_TYPE( thread, 0x07 )

9
src/include/j6/tables/vm_flags.inc
View File

@@ -1,9 +0,0 @@
VM_FLAG( none, 0x00000000)
VM_FLAG( write, 0x00000001)
VM_FLAG( exec, 0x00000002)
VM_FLAG( zero, 0x00000010)
VM_FLAG( contiguous, 0x00000020)
VM_FLAG( large_pages, 0x00000100)
VM_FLAG( huge_pages, 0x00000200)
VM_FLAG( write_combine, 0x00001000)
VM_FLAG( mmio, 0x00010000)

17
src/include/j6/types.h
View File

@@ -36,10 +36,15 @@ typedef uint64_t j6_handle_t;
#define j6_handle_id_mask 0xffffffffull
#define j6_handle_invalid 0xffffffffull
enum j6_object_type {
#define OBJECT_TYPE( name, val ) j6_object_type_ ## name = val,
#include "j6/tables/object_types.inc"
#undef OBJECT_TYPE
j6_object_type_max
/// A process' initial data structure for communicating with the system
struct j6_process_init
{
j6_handle_t process;
j6_handle_t handles[3];
};
/// A thread's initial data structure
struct j6_thread_init
{
j6_handle_t thread;
};

36
src/include/kernel_args.h
View File

@@ -35,26 +35,11 @@ struct framebuffer {
fb_type type;
};
enum class section_flags : uint32_t {
none = 0,
execute = 1,
write = 2,
read = 4,
};
struct program_section {
struct program {
uintptr_t phys_addr;
uintptr_t virt_addr;
uint32_t size;
section_flags type;
};
struct program {
uintptr_t entrypoint;
uintptr_t base;
size_t total_size;
size_t num_sections;
program_section sections[3];
size_t size;
};
enum class mem_type : uint32_t {
@@ -75,18 +60,6 @@ struct mem_entry
uint32_t attr;
};
constexpr size_t frames_per_block = 64 * 64 * 64;
struct frame_block
{
uintptr_t base;
uint32_t count;
uint32_t attrs;
uint64_t map1;
uint64_t map2[64];
uint64_t *bitmap;
};
enum class boot_flags : uint16_t {
none = 0x0000,
debug = 0x0001
@@ -100,7 +73,6 @@ struct header {
void *pml4;
void *page_tables;
size_t table_count;
size_t table_pages;
program *programs;
size_t num_programs;
@@ -111,10 +83,6 @@ struct header {
mem_entry *mem_map;
size_t map_count;
frame_block *frame_blocks;
size_t frame_block_count;
size_t frame_block_pages;
void *runtime_services;
void *acpi_table;

15
src/include/kernel_memory.h
View File

@@ -17,7 +17,7 @@ namespace memory {
constexpr uintptr_t page_offset = 0xffffc00000000000ull;
/// Max number of pages for a kernel stack
constexpr unsigned kernel_stack_pages = 2;
constexpr unsigned kernel_stack_pages = 4;
/// Max number of pages for a kernel buffer
constexpr unsigned kernel_buffer_pages = 16;
@@ -35,17 +35,11 @@ namespace memory {
constexpr uintptr_t stacks_start = heap_start - kernel_max_stacks;
/// Max size of kernel buffers area
constexpr size_t kernel_max_buffers = 0x8000000000ull; // 512GiB
constexpr size_t kernel_max_buffers = 0x10000000000ull; // 1TiB
/// Start of kernel buffers
constexpr uintptr_t buffers_start = stacks_start - kernel_max_buffers;
/// Max size of kernel bitmap area
constexpr size_t kernel_max_bitmap = 0x8000000000ull; // 512GiB
/// Start of kernel bitmap
constexpr uintptr_t bitmap_start = buffers_start - kernel_max_bitmap;
/// First kernel space PML4 entry
constexpr unsigned pml4e_kernel = 256;
@@ -64,11 +58,6 @@ namespace memory {
return reinterpret_cast<T*>(a|page_offset);
}
/// Convert a physical address to a virtual one (in the offset-mapped area)
template <typename T> T * to_virtual(T *p) {
return to_virtual<T>(reinterpret_cast<uintptr_t>(p));
}
/// Get the number of pages needed for a given number of bytes.
/// \arg bytes The number of bytes desired
/// \returns The number of pages needed to contain the desired bytes

11
src/include/j6/tables/log_areas.inc → src/include/log_areas.inc
View File

@@ -1,14 +1,13 @@
LOG(apic, info);
LOG(device, debug);
LOG(paging, info);
LOG(paging, warn);
LOG(driver, info);
LOG(memory, debug);
LOG(memory, info);
LOG(fs, info);
LOG(task, info);
LOG(sched, info);
LOG(loader, debug);
LOG(task, debug);
LOG(loader, info);
LOG(boot, debug);
LOG(syscall,info);
LOG(syscall,debug);
LOG(vmem, debug);
LOG(objs, debug);
LOG(timer, debug);

14
src/include/j6/tables/syscalls.inc → src/include/syscalls.inc
View File

@@ -1,21 +1,17 @@
SYSCALL(0x00, system_log, const char *)
SYSCALL(0x01, system_noop, void)
SYSCALL(0x02, system_get_log, j6_handle_t, void *, size_t *)
SYSCALL(0x02, system_get_log, j6_handle_t, char *, size_t *)
SYSCALL(0x03, system_bind_irq, j6_handle_t, j6_handle_t, unsigned)
SYSCALL(0x04, system_map_mmio, j6_handle_t, j6_handle_t *, uintptr_t, size_t, uint32_t)
SYSCALL(0x08, object_koid, j6_handle_t, j6_koid_t *)
SYSCALL(0x09, object_wait, j6_handle_t, j6_signal_t, j6_signal_t *)
SYSCALL(0x0a, object_signal, j6_handle_t, j6_signal_t)
SYSCALL(0x0b, object_close, j6_handle_t)
SYSCALL(0x10, process_create, j6_handle_t *)
SYSCALL(0x11, process_start, j6_handle_t, uintptr_t, j6_handle_t *, size_t)
SYSCALL(0x11, process_kill, j6_handle_t)
SYSCALL(0x17, process_exit, int32_t)
SYSCALL(0x10, process_exit, int64_t)
SYSCALL(0x18, thread_create, void *, j6_handle_t *)
SYSCALL(0x19, thread_exit, int32_t)
SYSCALL(0x19, thread_exit, int64_t)
SYSCALL(0x1a, thread_pause, void)
SYSCALL(0x1b, thread_sleep, uint64_t)
@@ -30,6 +26,6 @@ SYSCALL(0x2b, endpoint_sendrecv, j6_handle_t, j6_tag_t *, size_t *, void *)
SYSCALL(0x30, vma_create, j6_handle_t *, size_t, uint32_t)
SYSCALL(0x31, vma_create_map, j6_handle_t *, size_t, uintptr_t, uint32_t)
SYSCALL(0x32, vma_map, j6_handle_t, j6_handle_t, uintptr_t)
SYSCALL(0x33, vma_unmap, j6_handle_t, j6_handle_t)
SYSCALL(0x32, vma_map, j6_handle_t, uintptr_t)
SYSCALL(0x33, vma_unmap, j6_handle_t)
SYSCALL(0x34, vma_resize, j6_handle_t, size_t *)

13
src/kernel/acpi_tables.h
View File

@@ -23,7 +23,7 @@ struct acpi_table_header
} __attribute__ ((packed));
#define TABLE_HEADER(signature) \
static constexpr uint32_t type_id = kutil::byteswap(signature); \
static const uint32_t type_id = kutil::byteswap(signature); \
acpi_table_header header;
@@ -198,14 +198,3 @@ struct acpi_hpet
uint8_t attributes;
} __attribute__ ((packed));
struct acpi_bgrt
{
TABLE_HEADER('BGRT');
uint16_t version;
uint8_t status;
uint8_t type;
uintptr_t address;
uint32_t offset_x;
uint32_t offset_y;
} __attribute__ ((packed));

149
src/kernel/ap_startup.s
View File

@@ -1,149 +0,0 @@
%include "tasking.inc"
section .ap_startup
BASE equ 0x8000 ; Where the kernel will map this at runtime
CR0_PE equ (1 << 0)
CR0_MP equ (1 << 1)
CR0_ET equ (1 << 4)
CR0_NE equ (1 << 5)
CR0_WP equ (1 << 16)
CR0_PG equ (1 << 31)
CR0_VAL equ CR0_PE|CR0_MP|CR0_ET|CR0_NE|CR0_WP|CR0_PG
CR4_DE equ (1 << 3)
CR4_PAE equ (1 << 5)
CR4_MCE equ (1 << 6)
CR4_PGE equ (1 << 7)
CR4_OSFXSR equ (1 << 9)
CR4_OSCMMEXCPT equ (1 << 10)
CR4_FSGSBASE equ (1 << 16)
CR4_PCIDE equ (1 << 17)
CR4_INIT equ CR4_PAE|CR4_PGE
CR4_VAL equ CR4_DE|CR4_PAE|CR4_MCE|CR4_PGE|CR4_OSFXSR|CR4_OSCMMEXCPT|CR4_FSGSBASE|CR4_PCIDE
EFER_MSR equ 0xC0000080
EFER_SCE equ (1 << 0)
EFER_LME equ (1 << 8)
EFER_NXE equ (1 << 11)
EFER_VAL equ EFER_SCE|EFER_LME|EFER_NXE
bits 16
default rel
align 8
global ap_startup
ap_startup:
jmp .start_real
align 8
.pml4: dq 0
.cpu: dq 0
.ret: dq 0
align 16
.gdt:
dq 0x0 ; Null GDT entry
dq 0x00209A0000000000 ; Code
dq 0x0000920000000000 ; Data
align 4
.gdtd:
dw ($ - .gdt)
dd BASE + (.gdt - ap_startup)
align 4
.idtd:
dw 0 ; zero-length IDT descriptor
dd 0
.start_real:
cli
cld
xor ax, ax
mov ds, ax
; set the temporary null IDT
lidt [BASE + (.idtd - ap_startup)]
; Enter long mode
mov eax, cr4
or eax, CR4_INIT
mov cr4, eax
mov eax, [BASE + (.pml4 - ap_startup)]
mov cr3, eax
mov ecx, EFER_MSR
rdmsr
or eax, EFER_VAL
wrmsr
mov eax, CR0_VAL
mov cr0, eax
; Set the temporary minimal GDT
lgdt [BASE + (.gdtd - ap_startup)]
jmp (1 << 3):(BASE + (.start_long - ap_startup))
bits 64
default abs
align 8
.start_long:
; set data segments
mov ax, (2 << 3)
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
mov eax, CR4_VAL
mov rdi, [BASE + (.cpu - ap_startup)]
mov rax, [rdi + CPU_DATA.rsp0]
mov rsp, rax
mov rax, [BASE + (.ret - ap_startup)]
jmp rax
global ap_startup_code_size
ap_startup_code_size:
dq ($ - ap_startup)
section .text
global init_ap_trampoline
init_ap_trampoline:
push rbp
mov rbp, rsp
; rdi is the kernel pml4
mov [BASE + (ap_startup.pml4 - ap_startup)], rdi
; rsi is the cpu data for this AP
mov [BASE + (ap_startup.cpu - ap_startup)], rsi
; rdx is the address to jump to
mov [BASE + (ap_startup.ret - ap_startup)], rdx
; rcx is the processor id
mov rdi, rdx
pop rbp
ret
extern long_ap_startup
global ap_idle
ap_idle:
call long_ap_startup
sti
.hang:
hlt
jmp .hang

66
src/kernel/apic.cpp
View File

@@ -6,18 +6,11 @@
#include "kernel_memory.h"
#include "log.h"
uint64_t lapic::s_ticks_per_us = 0;
static constexpr uint16_t lapic_id = 0x0020;
static constexpr uint16_t lapic_spurious = 0x00f0;
static constexpr uint16_t lapic_icr_low = 0x0300;
static constexpr uint16_t lapic_icr_high = 0x0310;
static constexpr uint16_t lapic_lvt_timer = 0x0320;
static constexpr uint16_t lapic_lvt_lint0 = 0x0350;
static constexpr uint16_t lapic_lvt_lint1 = 0x0360;
static constexpr uint16_t lapic_lvt_error = 0x0370;
static constexpr uint16_t lapic_timer_init = 0x0380;
static constexpr uint16_t lapic_timer_cur = 0x0390;
@@ -32,7 +25,6 @@ apic_read(uint32_t volatile *apic, uint16_t offset)
static void
apic_write(uint32_t volatile *apic, uint16_t offset, uint32_t value)
{
log::debug(logs::apic, "LAPIC write: %x = %08lx", offset, value);
*(apic + offset/sizeof(uint32_t)) = value;
}
@@ -56,58 +48,14 @@ apic::apic(uintptr_t base) :
}
lapic::lapic(uintptr_t base) :
lapic::lapic(uintptr_t base, isr spurious) :
apic(base),
m_divisor(0)
{
apic_write(m_base, lapic_lvt_error, static_cast<uint32_t>(isr::isrAPICError));
apic_write(m_base, lapic_spurious, static_cast<uint32_t>(isr::isrSpurious));
apic_write(m_base, lapic_spurious, static_cast<uint32_t>(spurious));
log::info(logs::apic, "LAPIC created, base %lx", m_base);
}
uint8_t
lapic::get_id()
{
return static_cast<uint8_t>(apic_read(m_base, lapic_id) >> 24);
}
void
lapic::send_ipi(ipi mode, uint8_t vector, uint8_t dest)
{
// Wait until the APIC is ready to send
ipi_wait();
uint32_t command =
static_cast<uint32_t>(vector) |
static_cast<uint32_t>(mode);
apic_write(m_base, lapic_icr_high, static_cast<uint32_t>(dest) << 24);
apic_write(m_base, lapic_icr_low, command);
}
void
lapic::send_ipi_broadcast(ipi mode, bool self, uint8_t vector)
{
// Wait until the APIC is ready to send
ipi_wait();
uint32_t command =
static_cast<uint32_t>(vector) |
static_cast<uint32_t>(mode) |
(self ? 0 : (1 << 18)) |
(1 << 19);
apic_write(m_base, lapic_icr_high, 0);
apic_write(m_base, lapic_icr_low, command);
}
void
lapic::ipi_wait()
{
while (apic_read(m_base, lapic_icr_low) & (1<<12))
asm volatile ("pause" : : : "memory");
}
void
lapic::calibrate_timer()
{
@@ -120,14 +68,14 @@ lapic::calibrate_timer()
set_divisor(1);
apic_write(m_base, lapic_timer_init, initial);
uint64_t us = 20000;
uint64_t us = 200000;
clock::get().spinwait(us);
uint32_t remaining = apic_read(m_base, lapic_timer_cur);
uint64_t ticks_total = initial - remaining;
s_ticks_per_us = ticks_total / us;
uint32_t ticks_total = initial - remaining;
m_ticks_per_us = ticks_total / us;
log::info(logs::apic, "APIC timer ticks %d times per microsecond.", s_ticks_per_us);
log::info(logs::apic, "APIC timer ticks %d times per microsecond.", m_ticks_per_us);
interrupts_enable();
}
@@ -147,7 +95,7 @@ lapic::set_divisor(uint8_t divisor)
case 64: divbits = 0x9; break;
case 128: divbits = 0xa; break;
default:
kassert(0, "Invalid divisor passed to lapic::set_divisor");
kassert(0, "Invalid divisor passed to lapic::enable_timer");
}
apic_write(m_base, lapic_timer_div, divbits);

50
src/kernel/apic.h
View File

@@ -3,7 +3,6 @@
/// Classes to control both local and I/O APICs.
#include <stdint.h>
#include "kutil/enum_bitfields.h"
enum class isr : uint8_t;
@@ -19,22 +18,6 @@ protected:
uint32_t *m_base;
};
enum class ipi : uint32_t
{
// Delivery modes
fixed = 0x0000,
smi = 0x0200,
nmi = 0x0400,
init = 0x0500,
startup = 0x0600,
// Flags
deassert = 0x0000,
assert = 0x4000,
edge = 0x0000, ///< edge-triggered
level = 0x8000, ///< level-triggered
};
IS_BITFIELD(ipi);
/// Controller for processor-local APICs
class lapic :
@@ -43,26 +26,8 @@ class lapic :
public:
/// Constructor
/// \arg base Physicl base address of the APIC's MMIO registers
lapic(uintptr_t base);
/// Get the local APIC's ID
uint8_t get_id();
/// Send an inter-processor interrupt.
/// \arg mode The sending mode
/// \arg vector The interrupt vector
/// \arg dest The APIC ID of the destination
void send_ipi(ipi mode, uint8_t vector, uint8_t dest);
/// Send an inter-processor broadcast interrupt to all other CPUs
/// \arg mode The sending mode
/// \arg self If true, include this CPU in the broadcast
/// \arg vector The interrupt vector
void send_ipi_broadcast(ipi mode, bool self, uint8_t vector);
/// Wait for an IPI to finish sending. This is done automatically
/// before sending another IPI with send_ipi().
void ipi_wait();
/// \arg spurious Vector of the spurious interrupt handler
lapic(uintptr_t base, isr spurious);
/// Enable interrupts for the LAPIC timer.
/// \arg vector Interrupt vector the timer should use
@@ -92,14 +57,19 @@ public:
void calibrate_timer();
private:
inline static uint64_t ticks_to_us(uint64_t ticks) { return ticks / s_ticks_per_us; }
inline static uint64_t us_to_ticks(uint64_t interval) { return interval * s_ticks_per_us; }
inline uint64_t ticks_to_us(uint32_t ticks) const {
return static_cast<uint64_t>(ticks) / m_ticks_per_us;
}
inline uint64_t us_to_ticks(uint64_t interval) const {
return interval * m_ticks_per_us;
}
void set_divisor(uint8_t divisor);
void set_repeat(bool repeat);
uint32_t m_divisor;
static uint64_t s_ticks_per_us;
uint32_t m_ticks_per_us;
};

4
src/kernel/clock.cpp
View File

@@ -16,7 +16,7 @@ clock::clock(uint64_t rate, clock::source source_func, void *data) :
void
clock::spinwait(uint64_t us) const
{
uint64_t when = value() + us;
while (value() < when) asm ("pause");
uint64_t when = m_source(m_data) + us;
while (value() < when);
}

39
src/kernel/cpu.cpp
View File

@@ -2,18 +2,10 @@
#include "kutil/assert.h"
#include "kutil/memory.h"
#include "cpu.h"
#include "cpu/cpu_id.h"
#include "device_manager.h"
#include "gdt.h"
#include "idt.h"
#include "kernel_memory.h"
#include "cpu/cpu.h"
#include "log.h"
#include "msr.h"
#include "objects/vm_area.h"
#include "syscall.h"
#include "tss.h"
cpu_data g_bsp_cpu_data;
cpu_data bsp_cpu_data;
void
cpu_validate()
@@ -37,30 +29,3 @@ cpu_validate()
#undef CPU_FEATURE_OPT
#undef CPU_FEATURE_REQ
}
void
cpu_early_init(cpu_data *cpu)
{
IDT::get().install();
cpu->gdt->install();
// Install the GS base pointint to the cpu_data
wrmsr(msr::ia32_gs_base, reinterpret_cast<uintptr_t>(cpu));
}
void
cpu_init(cpu_data *cpu, bool bsp)
{
if (!bsp) {
// The BSP already called cpu_early_init
cpu_early_init(cpu);
}
// Set up the syscall MSRs
syscall_enable();
// Set up the page attributes table
uint64_t pat = rdmsr(msr::ia32_pat);
pat = (pat & 0x00ffffffffffffffull) | (0x01ull << 56); // set PAT 7 to WC
wrmsr(msr::ia32_pat, pat);
}

39
src/kernel/cpu.h
View File

@@ -2,12 +2,9 @@
#include <stdint.h>
class GDT;
class lapic;
class process;
struct TCB;
class thread;
class TSS;
class process;
struct cpu_state
{
@@ -21,39 +18,15 @@ struct cpu_state
/// version in 'tasking.inc'
struct cpu_data
{
cpu_data *self;
uint16_t id;
uint16_t index;
uint32_t reserved;
uintptr_t rsp0;
uintptr_t rsp3;
TCB *tcb;
thread *thread;
process *process;
TSS *tss;
GDT *gdt;
// Members beyond this point do not appear in
// the assembly version
lapic *apic;
thread *t;
process *p;
};
extern "C" cpu_data * _current_gsbase();
extern cpu_data bsp_cpu_data;
/// Set up the running CPU. This sets GDT, IDT, and necessary MSRs as well as creating
/// the cpu_data structure for this processor.
/// \arg cpu The cpu_data structure for this CPU
/// \arg bsp True if this CPU is the BSP
void cpu_init(cpu_data *cpu, bool bsp);
/// Do early (before cpu_init) initialization work. Only needs to be called manually for
/// the BSP, otherwise cpu_init will call it.
/// \arg cpu The cpu_data structure for this CPU
void cpu_early_init(cpu_data *cpu);
/// Get the cpu_data struct for the current executing CPU
inline cpu_data & current_cpu() { return *_current_gsbase(); }
/// Validate the required CPU features are present. Really, the bootloader already
/// validated the required features, but still iterate the options and log about them.
// We already validated the required options in the bootloader,
// but iterate the options and log about them.
void cpu_validate();

7
src/kernel/debug.cpp
View File

@@ -13,7 +13,6 @@ void
print_regs(const cpu_state &regs)
{
console *cons = console::get();
cpu_data &cpu = current_cpu();
uint64_t cr2 = 0;
__asm__ __volatile__ ("mov %%cr2, %0" : "=r"(cr2));
@@ -21,8 +20,8 @@ print_regs(const cpu_state &regs)
uintptr_t cr3 = 0;
__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (cr3) );
cons->printf(" process: %llx", cpu.process->koid());
cons->printf(" thread: %llx\n", cpu.thread->koid());
cons->printf(" process: %llx", bsp_cpu_data.p->koid());
cons->printf(" thread: %llx\n", bsp_cpu_data.t->koid());
print_regL("rax", regs.rax);
print_regM("rbx", regs.rbx);
@@ -44,7 +43,7 @@ print_regs(const cpu_state &regs)
cons->puts("\n\n");
print_regL("rbp", regs.rbp);
print_regM("rsp", regs.user_rsp);
print_regR("sp0", cpu.rsp0);
print_regR("sp0", bsp_cpu_data.rsp0);
print_regL("rip", regs.rip);
print_regM("cr3", cr3);

2
src/kernel/debug.h
View File

@@ -4,8 +4,6 @@
#include <stdint.h>
struct cpu_state;
extern "C" {
uintptr_t get_rsp();
uintptr_t get_rip();

168
src/kernel/device_manager.cpp
View File

@@ -38,7 +38,7 @@ struct acpi2_rsdp
uint32_t rsdt_address;
uint32_t length;
acpi_table_header *xsdt_address;
uint64_t xsdt_address;
uint8_t checksum20;
uint8_t reserved[3];
} __attribute__ ((packed));
@@ -63,7 +63,7 @@ void irq4_callback(void *)
device_manager::device_manager() :
m_lapic_base(0)
m_lapic(0)
{
m_irqs.ensure_capacity(32);
m_irqs.set_size(16);
@@ -73,20 +73,13 @@ device_manager::device_manager() :
m_irqs[2] = ignore_endpoint;
}
template <typename T> static const T *
check_get_table(const acpi_table_header *header)
{
kassert(header && header->validate(T::type_id), "Invalid ACPI table.");
return reinterpret_cast<const T *>(header);
}
void
device_manager::parse_acpi(const void *root_table)
{
kassert(root_table != 0, "ACPI root table pointer is null.");
const acpi1_rsdp *acpi1 = memory::to_virtual(
reinterpret_cast<const acpi1_rsdp *>(root_table));
const acpi1_rsdp *acpi1 =
reinterpret_cast<const acpi1_rsdp *>(root_table);
for (int i = 0; i < sizeof(acpi1->signature); ++i)
kassert(acpi1->signature[i] == expected_signature[i],
@@ -103,27 +96,7 @@ device_manager::parse_acpi(const void *root_table)
sum = kutil::checksum(acpi2, sizeof(acpi2_rsdp), sizeof(acpi1_rsdp));
kassert(sum == 0, "ACPI 2.0 RSDP checksum mismatch.");
load_xsdt(memory::to_virtual(acpi2->xsdt_address));
}
const device_manager::apic_nmi *
device_manager::get_lapic_nmi(uint8_t id) const
{
for (const auto &nmi : m_nmis) {
if (nmi.cpu == 0xff || nmi.cpu == id)
return &nmi;
}
return nullptr;
}
const device_manager::irq_override *
device_manager::get_irq_override(uint8_t irq) const
{
for (const auto &o : m_overrides)
if (o.source == irq) return &o;
return nullptr;
load_xsdt(reinterpret_cast<const acpi_xsdt *>(acpi2->xsdt_address));
}
ioapic *
@@ -139,9 +112,9 @@ put_sig(char *into, uint32_t type)
}
void
device_manager::load_xsdt(const acpi_table_header *header)
device_manager::load_xsdt(const acpi_xsdt *xsdt)
{
const auto *xsdt = check_get_table<acpi_xsdt>(header);
kassert(xsdt && acpi_validate(xsdt), "Invalid ACPI XSDT.");
char sig[5] = {0,0,0,0,0};
log::info(logs::device, "ACPI 2.0+ tables loading");
@@ -151,8 +124,7 @@ device_manager::load_xsdt(const acpi_table_header *header)
size_t num_tables = acpi_table_entries(xsdt, sizeof(void*));
for (size_t i = 0; i < num_tables; ++i) {
const acpi_table_header *header =
memory::to_virtual(xsdt->headers[i]);
const acpi_table_header *header = xsdt->headers[i];
put_sig(sig, header->type);
log::debug(logs::device, " Found table %s", sig);
@@ -161,15 +133,15 @@ device_manager::load_xsdt(const acpi_table_header *header)
switch (header->type) {
case acpi_apic::type_id:
load_apic(header);
load_apic(reinterpret_cast<const acpi_apic *>(header));
break;
case acpi_mcfg::type_id:
load_mcfg(header);
load_mcfg(reinterpret_cast<const acpi_mcfg *>(header));
break;
case acpi_hpet::type_id:
load_hpet(header);
load_hpet(reinterpret_cast<const acpi_hpet *>(header));
break;
default:
@@ -179,42 +151,40 @@ device_manager::load_xsdt(const acpi_table_header *header)
}
void
device_manager::load_apic(const acpi_table_header *header)
device_manager::load_apic(const acpi_apic *apic)
{
const auto *apic = check_get_table<acpi_apic>(header);
m_lapic_base = apic->local_address;
uintptr_t local = apic->local_address;
m_lapic = new lapic(local, isr::isrSpurious);
size_t count = acpi_table_entries(apic, 1);
uint8_t const *p = apic->controller_data;
uint8_t const *end = p + count;
// Pass one: count objcts
unsigned num_lapics = 0;
unsigned num_ioapics = 0;
unsigned num_overrides = 0;
unsigned num_nmis = 0;
// Pass one: count IOAPIC objcts
int num_ioapics = 0;
while (p < end) {
const uint8_t type = p[0];
const uint8_t length = p[1];
switch (type) {
case 0: ++num_lapics; break;
case 1: ++num_ioapics; break;
case 2: ++num_overrides; break;
case 4: ++num_nmis; break;
default: break;
}
if (type == 1) num_ioapics++;
p += length;
}
m_apic_ids.set_capacity(num_lapics);
m_ioapics.set_capacity(num_ioapics);
m_overrides.set_capacity(num_overrides);
m_nmis.set_capacity(num_nmis);
// Pass two: configure objects
// Pass two: set up IOAPIC objcts
p = apic->controller_data;
while (p < end) {
const uint8_t type = p[0];
const uint8_t length = p[1];
if (type == 1) {
uintptr_t base = kutil::read_from<uint32_t>(p+4);
uint32_t base_gsr = kutil::read_from<uint32_t>(p+8);
m_ioapics.emplace(base, base_gsr);
}
p += length;
}
// Pass three: configure APIC objects
p = apic->controller_data;
while (p < end) {
const uint8_t type = p[0];
@@ -224,42 +194,38 @@ device_manager::load_apic(const acpi_table_header *header)
case 0: { // Local APIC
uint8_t uid = kutil::read_from<uint8_t>(p+2);
uint8_t id = kutil::read_from<uint8_t>(p+3);
m_apic_ids.append(id);
log::debug(logs::device, " Local APIC uid %x id %x", uid, id);
log::debug(logs::device, " Local APIC uid %x id %x", id);
}
break;
case 1: { // I/O APIC
uintptr_t base = kutil::read_from<uint32_t>(p+4);
uint32_t base_gsi = kutil::read_from<uint32_t>(p+8);
m_ioapics.emplace(base, base_gsi);
log::debug(logs::device, " IO APIC gsi %x base %x", base_gsi, base);
}
case 1: // I/O APIC
break;
case 2: { // Interrupt source override
irq_override o;
o.source = kutil::read_from<uint8_t>(p+3);
o.gsi = kutil::read_from<uint32_t>(p+4);
o.flags = kutil::read_from<uint16_t>(p+8);
m_overrides.append(o);
uint8_t source = kutil::read_from<uint8_t>(p+3);
isr gsi = isr::irq00 + kutil::read_from<uint32_t>(p+4);
uint16_t flags = kutil::read_from<uint16_t>(p+8);
log::debug(logs::device, " Intr source override IRQ %d -> %d Pol %d Tri %d",
o.source, o.gsi, (o.flags & 0x3), ((o.flags >> 2) & 0x3));
source, gsi, (flags & 0x3), ((flags >> 2) & 0x3));
// TODO: in a multiple-IOAPIC system this might be elsewhere
m_ioapics[0].redirect(source, static_cast<isr>(gsi), flags, true);
}
break;
case 4: {// LAPIC NMI
apic_nmi nmi;
nmi.cpu = kutil::read_from<uint8_t>(p + 2);
nmi.lint = kutil::read_from<uint8_t>(p + 5);
nmi.flags = kutil::read_from<uint16_t>(p + 3);
m_nmis.append(nmi);
uint8_t cpu = kutil::read_from<uint8_t>(p + 2);
uint8_t num = kutil::read_from<uint8_t>(p + 5);
uint16_t flags = kutil::read_from<uint16_t>(p + 3);
log::debug(logs::device, " LAPIC NMI Proc %02x LINT%d Pol %d Tri %d",
nmi.cpu, nmi.lint, nmi.flags & 0x3, (nmi.flags >> 2) & 0x3);
log::debug(logs::device, " LAPIC NMI Proc %d LINT%d Pol %d Tri %d",
kutil::read_from<uint8_t>(p+2),
kutil::read_from<uint8_t>(p+5),
kutil::read_from<uint16_t>(p+3) & 0x3,
(kutil::read_from<uint16_t>(p+3) >> 2) & 0x3);
m_lapic->enable_lint(num, num == 0 ? isr::isrLINT0 : isr::isrLINT1, true, flags);
}
break;
@@ -269,13 +235,20 @@ device_manager::load_apic(const acpi_table_header *header)
p += length;
}
for (uint8_t i = 0; i < m_ioapics[0].get_num_gsi(); ++i) {
switch (i) {
case 2: break;
default: m_ioapics[0].mask(i, false);
}
}
m_lapic->enable();
}
void
device_manager::load_mcfg(const acpi_table_header *header)
device_manager::load_mcfg(const acpi_mcfg *mcfg)
{
const auto *mcfg = check_get_table<acpi_mcfg>(header);
size_t count = acpi_table_entries(mcfg, sizeof(acpi_mcfg_entry));
m_pci.set_size(count);
m_devices.set_capacity(16);
@@ -296,10 +269,8 @@ device_manager::load_mcfg(const acpi_table_header *header)
}
void
device_manager::load_hpet(const acpi_table_header *header)
device_manager::load_hpet(const acpi_hpet *hpet)
{
const auto *hpet = check_get_table<acpi_hpet>(header);
log::debug(logs::device, " Found HPET device #%3d: base %016lx pmin %d attr %02x",
hpet->index, hpet->base_address.address, hpet->periodic_min, hpet->attributes);
@@ -340,13 +311,6 @@ device_manager::probe_pci()
}
}
static uint64_t
fake_clock_source(void*)
{
static uint64_t value = 0;
return value++;
}
void
device_manager::init_drivers()
{
@@ -365,20 +329,18 @@ device_manager::init_drivers()
ahcid.register_device(&device);
}
*/
clock *master_clock = nullptr;
if (m_hpets.count() > 0) {
hpet &h = m_hpets[0];
h.enable();
// becomes the singleton
master_clock = new clock(h.rate(), hpet_clock_source, &h);
clock *master_clock = new clock(h.rate(), hpet_clock_source, &h);
kassert(master_clock, "Failed to allocate master clock");
log::info(logs::clock, "Created master clock using HPET 0: Rate %d", h.rate());
} else {
//TODO: Other clocks, APIC clock?
master_clock = new clock(5000, fake_clock_source, nullptr);
//TODO: APIC clock?
kassert(0, "No HPET master clock");
}
kassert(master_clock, "Failed to allocate master clock");
}
bool

56
src/kernel/device_manager.h
View File

@@ -6,7 +6,10 @@
#include "hpet.h"
#include "pci.h"
struct acpi_table_header;
struct acpi_xsdt;
struct acpi_apic;
struct acpi_mcfg;
struct acpi_hpet;
class block_device;
class endpoint;
@@ -24,6 +27,10 @@ public:
/// \returns A reference to the system device manager
static device_manager & get() { return s_instance; }
/// Get the LAPIC
/// \returns An object representing the local APIC
lapic * get_lapic() { return m_lapic; }
/// Get an IOAPIC
/// \arg i Index of the requested IOAPIC
/// \returns An object representing the given IOAPIC if it exists,
@@ -64,39 +71,6 @@ public:
/// \returns True if the interrupt was handled
bool dispatch_irq(unsigned irq);
struct apic_nmi
{
uint8_t cpu;
uint8_t lint;
uint16_t flags;
};
struct irq_override
{
uint8_t source;
uint16_t flags;
uint32_t gsi;
};
/// Get the list of APIC ids for other CPUs
inline const kutil::vector<uint8_t> & get_apic_ids() const { return m_apic_ids; }
/// Get the LAPIC base address
/// \returns The physical base address of the local apic registers
uintptr_t get_lapic_base() const { return m_lapic_base; }
/// Get the NMI mapping for the given local APIC
/// \arg id ID of the local APIC
/// \returns apic_nmi structure describing the NMI configuration,
/// or null if no configuration was provided
const apic_nmi * get_lapic_nmi(uint8_t id) const;
/// Get the IRQ source override for the given IRQ
/// \arg irq IRQ number (not isr vector)
/// \returns irq_override structure describing that IRQ's
/// configuration, or null if no configuration was provided
const irq_override * get_irq_override(uint8_t irq) const;
/// Register the existance of a block device.
/// \arg blockdev Pointer to the block device
void register_block_device(block_device *blockdev);
@@ -126,19 +100,19 @@ public:
private:
/// Parse the ACPI XSDT and load relevant sub-tables.
/// \arg xsdt Pointer to the XSDT from the firmware
void load_xsdt(const acpi_table_header *xsdt);
void load_xsdt(const acpi_xsdt *xsdt);
/// Parse the ACPI MADT and initialize APICs from it.
/// \arg apic Pointer to the MADT from the XSDT
void load_apic(const acpi_table_header *apic);
void load_apic(const acpi_apic *apic);
/// Parse the ACPI MCFG and initialize PCIe from it.
/// \arg mcfg Pointer to the MCFG from the XSDT
void load_mcfg(const acpi_table_header *mcfg);
void load_mcfg(const acpi_mcfg *mcfg);
/// Parse the ACPI HPET and initialize an HPET from it.
/// \arg hpet Pointer to the HPET from the XSDT
void load_hpet(const acpi_table_header *hpet);
void load_hpet(const acpi_hpet *hpet);
/// Probe the PCIe busses and add found devices to our
/// device list. The device list is destroyed and rebuilt.
@@ -148,13 +122,9 @@ private:
/// that has no callback.
void bad_irq(uint8_t irq);
uintptr_t m_lapic_base;
lapic *m_lapic;
kutil::vector<ioapic> m_ioapics;
kutil::vector<hpet> m_hpets;
kutil::vector<uint8_t> m_apic_ids;
kutil::vector<apic_nmi> m_nmis;
kutil::vector<irq_override> m_overrides;
kutil::vector<pci_group> m_pci;
kutil::vector<pci_device> m_devices;

171
src/kernel/frame_allocator.cpp
View File

@@ -1,12 +1,21 @@
#include "kernel_memory.h"
#include "kutil/assert.h"
#include "kutil/memory.h"
#include "frame_allocator.h"
#include "kernel_args.h"
#include "kernel_memory.h"
#include "log.h"
using memory::frame_size;
using memory::page_offset;
using frame_block_node = kutil::list_node<frame_block>;
int
frame_block::compare(const frame_block &rhs) const
{
if (address < rhs.address)
return -1;
else if (address > rhs.address)
return 1;
return 0;
}
frame_allocator &
@@ -16,148 +25,54 @@ frame_allocator::get()
return g_frame_allocator;
}
frame_allocator::frame_allocator(kernel::args::frame_block *frames, size_t count) :
m_blocks {frames},
m_count {count}
{
}
inline unsigned
bsf(uint64_t v)
{
asm ("tzcntq %q0, %q1" : "=r"(v) : "0"(v) : "cc");
return v;
}
frame_allocator::frame_allocator() {}
size_t
frame_allocator::allocate(size_t count, uintptr_t *address)
{
kutil::scoped_lock lock {m_lock};
kassert(!m_free.empty(), "frame_allocator::pop_frames ran out of free frames!");
if (m_free.empty())
return 0;
for (long i = m_count - 1; i >= 0; --i) {
frame_block &block = m_blocks[i];
auto *first = m_free.front();
if (!block.map1)
continue;
// Tree walk to find the first available page
unsigned o1 = bsf(block.map1);
uint64_t m2 = block.map2[o1];
unsigned o2 = bsf(m2);
uint64_t m3 = block.bitmap[(o1 << 6) + o2];
unsigned o3 = bsf(m3);
unsigned frame = (o1 << 12) + (o2 << 6) + o3;
// See how many contiguous pages are here
unsigned n = bsf(~m3 >> o3);
if (n > count)
n = count;
*address = block.base + frame * frame_size;
// Clear the bits to mark these pages allocated
m3 &= ~(((1 << n) - 1) << o3);
block.bitmap[(o1 << 6) + o2] = m3;
if (!m3) {
// if that was it for this group, clear the next level bit
m2 &= ~(1 << o2);
block.map2[o1] = m2;
if (!m2) {
// if that was cleared too, update the top level
block.map1 &= ~(1 << o1);
}
}
return n;
if (count >= first->count) {
*address = first->address;
m_free.remove(first);
return first->count;
} else {
first->count -= count;
*address = first->address + (first->count * frame_size);
return count;
}
kassert(false, "frame_allocator ran out of free frames!");
return 0;
}
inline uintptr_t end(frame_block *node) { return node->address + node->count * frame_size; }
void
frame_allocator::free(uintptr_t address, size_t count)
{
kutil::scoped_lock lock {m_lock};
kassert(address % frame_size == 0, "Trying to free a non page-aligned frame!");
if (!count)
return;
frame_block_node *node =
reinterpret_cast<frame_block_node*>(address + page_offset);
for (long i = 0; i < m_count; ++i) {
frame_block &block = m_blocks[i];
uintptr_t end = block.base + block.count * frame_size;
kutil::memset(node, 0, sizeof(frame_block_node));
node->address = address;
node->count = count;
if (address < block.base || address >= end)
continue;
m_free.sorted_insert(node);
uint64_t frame = (address - block.base) >> 12;
unsigned o1 = (frame >> 12) & 0x3f;
unsigned o2 = (frame >> 6) & 0x3f;
unsigned o3 = frame & 0x3f;
frame_block_node *next = node->next();
if (next && end(node) == next->address) {
node->count += next->count;
m_free.remove(next);
}
while (count--) {
block.map1 |= (1 << o1);
block.map2[o1] |= (1 << o2);
block.bitmap[o2] |= (1 << o3);
if (++o3 == 64) {
o3 = 0;
if (++o2 == 64) {
o2 = 0;
++o1;
kassert(o1 < 64, "Tried to free pages past the end of a block");
}
}
}
}
}
void
frame_allocator::used(uintptr_t address, size_t count)
{
kutil::scoped_lock lock {m_lock};
kassert(address % frame_size == 0, "Trying to mark a non page-aligned frame!");
if (!count)
return;
for (long i = 0; i < m_count; ++i) {
frame_block &block = m_blocks[i];
uintptr_t end = block.base + block.count * frame_size;
if (address < block.base || address >= end)
continue;
uint64_t frame = (address - block.base) >> 12;
unsigned o1 = (frame >> 12) & 0x3f;
unsigned o2 = (frame >> 6) & 0x3f;
unsigned o3 = frame & 0x3f;
while (count--) {
block.bitmap[o2] &= ~(1 << o3);
if (!block.bitmap[o2]) {
block.map2[o1] &= ~(1 << o2);
if (!block.map2[o1]) {
block.map1 &= ~(1 << o1);
}
}
if (++o3 == 64) {
o3 = 0;
if (++o2 == 64) {
o2 = 0;
++o1;
kassert(o1 < 64, "Tried to mark pages past the end of a block");
}
}
}
frame_block_node *prev = node->prev();
if (prev && end(prev) == address) {
prev->count += node->count;
m_free.remove(node);
}
}

43
src/kernel/frame_allocator.h
View File

@@ -3,23 +3,18 @@
/// Allocator for physical memory frames
#include <stdint.h>
#include "kutil/spinlock.h"
namespace kernel {
namespace args {
struct frame_block;
}}
#include "kutil/linked_list.h"
struct frame_block;
using frame_block_list = kutil::linked_list<frame_block>;
/// Allocator for physical memory frames
class frame_allocator
{
public:
using frame_block = kernel::args::frame_block;
/// Constructor
/// \arg blocks The bootloader-supplied frame bitmap block list
/// \arg count Number of entries in the block list
frame_allocator(frame_block *frames, size_t count);
/// Default constructor
frame_allocator();
/// Get free frames from the free list. Only frames from the first free block
/// are returned, so the number may be less than requested, but they will
@@ -34,20 +29,26 @@ public:
/// \arg count The number of frames to be freed
void free(uintptr_t address, size_t count);
/// Mark frames as used
/// \arg address The physical address of the first frame to free
/// \arg count The number of frames to be freed
void used(uintptr_t address, size_t count);
/// Get the global frame allocator
static frame_allocator & get();
private:
frame_block *m_blocks;
size_t m_count;
frame_block_list m_free; ///< Free frames list
kutil::spinlock m_lock;
frame_allocator() = delete;
frame_allocator(const frame_allocator &) = delete;
};
/// A block of contiguous frames. Each `frame_block` represents contiguous
/// physical frames with the same attributes.
struct frame_block
{
uintptr_t address;
uint32_t count;
/// Compare two blocks by address.
/// \arg rhs The right-hand comparator
/// \returns <0 if this is sorts earlier, >0 if this sorts later, 0 for equal
int compare(const frame_block &rhs) const;
};

276
src/kernel/gdt.cpp
View File

@@ -1,80 +1,35 @@
#include <stdint.h>
#include "kutil/assert.h"
#include "kutil/enum_bitfields.h"
#include "kutil/memory.h"
#include "kutil/no_construct.h"
#include "console.h"
#include "cpu.h"
#include "gdt.h"
#include "log.h"
#include "tss.h"
extern "C" void gdt_write(const void *gdt_ptr, uint16_t cs, uint16_t ds, uint16_t tr);
static constexpr uint8_t kern_cs_index = 1;
static constexpr uint8_t kern_ss_index = 2;
static constexpr uint8_t user_cs32_index = 3;
static constexpr uint8_t user_ss_index = 4;
static constexpr uint8_t user_cs64_index = 5;
static constexpr uint8_t tss_index = 6; // Note that this takes TWO GDT entries
// The BSP's GDT is initialized _before_ global constructors are called,
// so we don't want it to have a global constructor, lest it overwrite
// the previous initialization.
static kutil::no_construct<GDT> __g_bsp_gdt_storage;
GDT &g_bsp_gdt = __g_bsp_gdt_storage.value;
GDT::GDT(TSS *tss) :
m_tss(tss)
enum class gdt_type : uint8_t
{
kutil::memset(this, 0, sizeof(GDT));
accessed = 0x01,
read_write = 0x02,
conforming = 0x04,
execute = 0x08,
system = 0x10,
ring1 = 0x20,
ring2 = 0x40,
ring3 = 0x60,
present = 0x80
};
IS_BITFIELD(gdt_type);
m_ptr.limit = sizeof(m_entries) - 1;
m_ptr.base = &m_entries[0];
// Kernel CS/SS - always 64bit
set(kern_cs_index, 0, 0xfffff, true, gdt_type::read_write | gdt_type::execute);
set(kern_ss_index, 0, 0xfffff, true, gdt_type::read_write);
// User CS32/SS/CS64 - layout expected by SYSRET
set(user_cs32_index, 0, 0xfffff, false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
set(user_ss_index, 0, 0xfffff, true, gdt_type::ring3 | gdt_type::read_write);
set(user_cs64_index, 0, 0xfffff, true, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
set_tss(tss);
}
GDT &
GDT::current()
struct gdt_descriptor
{
cpu_data &cpu = current_cpu();
return *cpu.gdt;
}
void
GDT::install() const
{
gdt_write(
static_cast<const void*>(&m_ptr),
kern_cs_index << 3,
kern_ss_index << 3,
tss_index << 3);
}
void
GDT::set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
{
m_entries[i].limit_low = limit & 0xffff;
m_entries[i].size = (limit >> 16) & 0xf;
m_entries[i].size |= (is64 ? 0xa0 : 0xc0);
m_entries[i].base_low = base & 0xffff;
m_entries[i].base_mid = (base >> 16) & 0xff;
m_entries[i].base_high = (base >> 24) & 0xff;
m_entries[i].type = type | gdt_type::system | gdt_type::present;
}
uint16_t limit_low;
uint16_t base_low;
uint8_t base_mid;
gdt_type type;
uint8_t size;
uint8_t base_high;
} __attribute__ ((packed));
struct tss_descriptor
{
@@ -88,16 +43,72 @@ struct tss_descriptor
uint32_t reserved;
} __attribute__ ((packed));
struct tss_entry
{
uint32_t reserved0;
uint64_t rsp[3]; // stack pointers for CPL 0-2
uint64_t ist[8]; // ist[0] is reserved
uint64_t reserved1;
uint16_t reserved2;
uint16_t iomap_offset;
} __attribute__ ((packed));
struct idt_descriptor
{
uint16_t base_low;
uint16_t selector;
uint8_t ist;
uint8_t flags;
uint16_t base_mid;
uint32_t base_high;
uint32_t reserved; // must be zero
} __attribute__ ((packed));
struct table_ptr
{
uint16_t limit;
uint64_t base;
} __attribute__ ((packed));
gdt_descriptor g_gdt_table[10];
idt_descriptor g_idt_table[256];
table_ptr g_gdtr;
table_ptr g_idtr;
tss_entry g_tss;
extern "C" {
void idt_write();
void idt_load();
void gdt_write(uint16_t cs, uint16_t ds, uint16_t tr);
void gdt_load();
}
void
GDT::set_tss(TSS *tss)
gdt_set_entry(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
{
g_gdt_table[i].limit_low = limit & 0xffff;
g_gdt_table[i].size = (limit >> 16) & 0xf;
g_gdt_table[i].size |= (is64 ? 0xa0 : 0xc0);
g_gdt_table[i].base_low = base & 0xffff;
g_gdt_table[i].base_mid = (base >> 16) & 0xff;
g_gdt_table[i].base_high = (base >> 24) & 0xff;
g_gdt_table[i].type = type | gdt_type::system | gdt_type::present;
}
void
tss_set_entry(uint8_t i, uint64_t base, uint64_t limit)
{
tss_descriptor tssd;
size_t limit = sizeof(TSS);
tssd.limit_low = limit & 0xffff;
tssd.size = (limit >> 16) & 0xf;
uintptr_t base = reinterpret_cast<uintptr_t>(tss);
tssd.base_00 = base & 0xffff;
tssd.base_16 = (base >> 16) & 0xff;
tssd.base_24 = (base >> 24) & 0xff;
@@ -109,26 +120,87 @@ GDT::set_tss(TSS *tss)
gdt_type::execute |
gdt_type::ring3 |
gdt_type::present;
kutil::memcpy(&m_entries[tss_index], &tssd, sizeof(tss_descriptor));
kutil::memcpy(&g_gdt_table[i], &tssd, sizeof(tss_descriptor));
}
void
GDT::dump(unsigned index) const
idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags)
{
g_idt_table[i].base_low = addr & 0xffff;
g_idt_table[i].base_mid = (addr >> 16) & 0xffff;
g_idt_table[i].base_high = (addr >> 32) & 0xffffffff;
g_idt_table[i].selector = selector;
g_idt_table[i].flags = flags;
g_idt_table[i].ist = 0;
g_idt_table[i].reserved = 0;
}
void
tss_set_stack(int ring, uintptr_t rsp)
{
kassert(ring < 3, "Bad ring passed to tss_set_stack.");
g_tss.rsp[ring] = rsp;
}
uintptr_t
tss_get_stack(int ring)
{
kassert(ring < 3, "Bad ring passed to tss_get_stack.");
return g_tss.rsp[ring];
}
void
gdt_init()
{
kutil::memset(&g_gdt_table, 0, sizeof(g_gdt_table));
kutil::memset(&g_idt_table, 0, sizeof(g_idt_table));
g_gdtr.limit = sizeof(g_gdt_table) - 1;
g_gdtr.base = reinterpret_cast<uint64_t>(&g_gdt_table);
// Kernel CS/SS - always 64bit
gdt_set_entry(1, 0, 0xfffff, true, gdt_type::read_write | gdt_type::execute);
gdt_set_entry(2, 0, 0xfffff, true, gdt_type::read_write);
// User CS32/SS/CS64 - layout expected by SYSRET
gdt_set_entry(3, 0, 0xfffff, false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
gdt_set_entry(4, 0, 0xfffff, true, gdt_type::ring3 | gdt_type::read_write);
gdt_set_entry(5, 0, 0xfffff, true, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
kutil::memset(&g_tss, 0, sizeof(tss_entry));
g_tss.iomap_offset = sizeof(tss_entry);
uintptr_t tss_base = reinterpret_cast<uintptr_t>(&g_tss);
// Note that this takes TWO GDT entries
tss_set_entry(6, tss_base, sizeof(tss_entry));
gdt_write(1 << 3, 2 << 3, 6 << 3);
g_idtr.limit = sizeof(g_idt_table) - 1;
g_idtr.base = reinterpret_cast<uint64_t>(&g_idt_table);
idt_write();
}
void
gdt_dump(int index)
{
const table_ptr &table = g_gdtr;
console *cons = console::get();
unsigned start = 0;
unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
int start = 0;
int count = (table.limit + 1) / sizeof(gdt_descriptor);
if (index != -1) {
start = index;
count = 1;
} else {
cons->printf(" GDT: loc:%lx size:%d\n", m_ptr.base, m_ptr.limit+1);
cons->printf(" GDT: loc:%lx size:%d\n", table.base, table.limit+1);
}
const descriptor *gdt =
reinterpret_cast<const descriptor *>(m_ptr.base);
const gdt_descriptor *gdt =
reinterpret_cast<const gdt_descriptor *>(table.base);
for (int i = start; i < start+count; ++i) {
uint32_t base =
@@ -166,3 +238,51 @@ GDT::dump(unsigned index) const
(gdt[i].size & 0x60) == 0x40 ? "32" : "16");
}
}
void
idt_dump(int index)
{
const table_ptr &table = g_idtr;
int start = 0;
int count = (table.limit + 1) / sizeof(idt_descriptor);
if (index != -1) {
start = index;
count = 1;
log::info(logs::boot, "IDT FOR INDEX %02x", index);
} else {
log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", table.base, table.limit+1);
}
const idt_descriptor *idt =
reinterpret_cast<const idt_descriptor *>(table.base);
for (int i = start; i < start+count; ++i) {
uint64_t base =
(static_cast<uint64_t>(idt[i].base_high) << 32) |
(static_cast<uint64_t>(idt[i].base_mid) << 16) |
idt[i].base_low;
char const *type;
switch (idt[i].flags & 0xf) {
case 0x5: type = " 32tsk "; break;
case 0x6: type = " 16int "; break;
case 0x7: type = " 16trp "; break;
case 0xe: type = " 32int "; break;
case 0xf: type = " 32trp "; break;
default: type = " ????? "; break;
}
if (idt[i].flags & 0x80) {
log::debug(logs::boot,
" Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
(idt[i].selector & 0x3),
((idt[i].selector & 0x4) >> 2),
(idt[i].selector >> 3),
idt[i].ist,
type,
((idt[i].flags >> 5) & 0x3));
}
}
}

81
src/kernel/gdt.h
View File

@@ -1,66 +1,33 @@
#pragma once
/// \file gdt.h
/// Definitions relating to a CPU's GDT table
/// Definitions relating to system descriptor tables: GDT, IDT, TSS
#include <stdint.h>
#include "kutil/enum_bitfields.h"
/// Set up the GDT and TSS, and switch segment registers to point
/// to them.
void gdt_init();
class TSS;
/// Set an entry in the IDT
/// \arg i Index in the IDT (vector of the interrupt this handles)
/// \arg addr Address of the handler
/// \arg selector GDT selector to set when invoking this handler
/// \arg flags Descriptor flags to set
void idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags);
enum class gdt_type : uint8_t
{
accessed = 0x01,
read_write = 0x02,
conforming = 0x04,
execute = 0x08,
system = 0x10,
ring1 = 0x20,
ring2 = 0x40,
ring3 = 0x60,
present = 0x80
};
IS_BITFIELD(gdt_type);
/// Set the stack pointer for a given ring in the TSS
/// \arg ring Ring to set for (0-2)
/// \arg rsp Stack pointer to set
void tss_set_stack(int ring, uintptr_t rsp);
class GDT
{
public:
GDT(TSS *tss);
/// Get the stack pointer for a given ring in the TSS
/// \arg ring Ring to get (0-2)
/// \returns Stack pointers for that ring
uintptr_t tss_get_stack(int ring);
/// Get the currently running CPU's GDT
static GDT & current();
/// Dump information about the current GDT to the screen
/// \arg index Which entry to print, or -1 for all entries
void gdt_dump(int index = -1);
/// Install this GDT to the current CPU
void install() const;
/// Get the addrss of the pointer
inline const void * pointer() const { return static_cast<const void*>(&m_ptr); }
/// Dump debug information about the GDT to the console.
/// \arg index Which entry to print, or -1 for all entries
void dump(unsigned index = -1) const;
private:
void set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type);
void set_tss(TSS *tss);
struct descriptor
{
uint16_t limit_low;
uint16_t base_low;
uint8_t base_mid;
gdt_type type;
uint8_t size;
uint8_t base_high;
} __attribute__ ((packed, align(8)));
struct ptr
{
uint16_t limit;
descriptor *base;
} __attribute__ ((packed, align(4)));
descriptor m_entries[8];
TSS *m_tss;
ptr m_ptr;
};
/// Dump information about the current IDT to the screen
/// \arg index Which entry to print, or -1 for all entries
void idt_dump(int index = -1);

35
src/kernel/gdt.s Normal file
View File

@@ -0,0 +1,35 @@
extern g_idtr
extern g_gdtr
global idt_write
idt_write:
lidt [rel g_idtr]
ret
global idt_load
idt_load:
sidt [rel g_idtr]
ret
global gdt_write
gdt_write:
lgdt [rel g_gdtr]
mov ax, si ; second arg is data segment
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
push qword rdi ; first arg is code segment
lea rax, [rel .next]
push rax
o64 retf
.next:
ltr dx ; third arg is the TSS
ret
global gdt_load
gdt_load:
sgdt [rel g_gdtr]
ret

35
src/kernel/gdtidt.s
View File

@@ -1,35 +0,0 @@
global idt_write
idt_write:
lidt [rdi] ; first arg is the IDT pointer location
ret
global idt_load
idt_load:
sidt [rdi] ; first arg is where to write the idtr value
ret
global gdt_write
gdt_write:
lgdt [rdi] ; first arg is the GDT pointer location
mov ax, dx ; third arg is data segment
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
push qword rsi ; second arg is code segment
lea rax, [rel .next]
push rax
o64 retf
.next:
ltr cx ; fourth arg is the TSS
ret
global gdt_load
gdt_load:
sgdt [rdi] ; first arg is where to write the gdtr value
ret

137
src/kernel/idt.cpp
View File

@@ -1,137 +0,0 @@
#include "kutil/memory.h"
#include "kutil/no_construct.h"
#include "idt.h"
#include "log.h"
extern "C" {
void idt_write(const void *idt_ptr);
#define ISR(i, s, name) extern void name ();
#define EISR(i, s, name) extern void name ();
#define IRQ(i, q, name) extern void name ();
#include "interrupt_isrs.inc"
#undef IRQ
#undef EISR
#undef ISR
}
// The IDT is initialized _before_ global constructors are called,
// so we don't want it to have a global constructor, lest it overwrite
// the previous initialization.
static kutil::no_construct<IDT> __g_idt_storage;
IDT &g_idt = __g_idt_storage.value;
IDT::IDT()
{
kutil::memset(this, 0, sizeof(IDT));
m_ptr.limit = sizeof(m_entries) - 1;
m_ptr.base = &m_entries[0];
#define ISR(i, s, name) set(i, & name, 0x08, 0x8e);
#define EISR(i, s, name) set(i, & name, 0x08, 0x8e);
#define IRQ(i, q, name) set(i, & name, 0x08, 0x8e);
#include "interrupt_isrs.inc"
#undef IRQ
#undef EISR
#undef ISR
}
IDT &
IDT::get()
{
return g_idt;
}
void
IDT::install() const
{
idt_write(static_cast<const void*>(&m_ptr));
}
void
IDT::add_ist_entries()
{
#define ISR(i, s, name) if (s) { set_ist(i, s); }
#define EISR(i, s, name) if (s) { set_ist(i, s); }
#define IRQ(i, q, name)
#include "interrupt_isrs.inc"
#undef IRQ
#undef EISR
#undef ISR
}
uint8_t
IDT::used_ist_entries() const
{
uint8_t entries = 0;
#define ISR(i, s, name) if (s) { entries |= (1 << s); }
#define EISR(i, s, name) if (s) { entries |= (1 << s); }
#define IRQ(i, q, name)
#include "interrupt_isrs.inc"
#undef IRQ
#undef EISR
#undef ISR
return entries;
}
void
IDT::set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags)
{
uintptr_t addr = reinterpret_cast<uintptr_t>(handler);
m_entries[i].base_low = addr & 0xffff;
m_entries[i].base_mid = (addr >> 16) & 0xffff;
m_entries[i].base_high = (addr >> 32) & 0xffffffff;
m_entries[i].selector = selector;
m_entries[i].flags = flags;
m_entries[i].ist = 0;
m_entries[i].reserved = 0;
}
void
IDT::dump(unsigned index) const
{
unsigned start = 0;
unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
if (index != -1) {
start = index;
count = 1;
log::info(logs::boot, "IDT FOR INDEX %02x", index);
} else {
log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", m_ptr.base, m_ptr.limit+1);
}
const descriptor *idt =
reinterpret_cast<const descriptor *>(m_ptr.base);
for (int i = start; i < start+count; ++i) {
uint64_t base =
(static_cast<uint64_t>(idt[i].base_high) << 32) |
(static_cast<uint64_t>(idt[i].base_mid) << 16) |
idt[i].base_low;
char const *type;
switch (idt[i].flags & 0xf) {
case 0x5: type = " 32tsk "; break;
case 0x6: type = " 16int "; break;
case 0x7: type = " 16trp "; break;
case 0xe: type = " 32int "; break;
case 0xf: type = " 32trp "; break;
default: type = " ????? "; break;
}
if (idt[i].flags & 0x80) {
log::debug(logs::boot,
" Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
(idt[i].selector & 0x3),
((idt[i].selector & 0x4) >> 2),
(idt[i].selector >> 3),
idt[i].ist,
type,
((idt[i].flags >> 5) & 0x3));
}
}
}

63
src/kernel/idt.h
View File

@@ -1,63 +0,0 @@
#pragma once
/// \file idt.h
/// Definitions relating to a CPU's IDT table
#include <stdint.h>
class IDT
{
public:
IDT();
/// Install this IDT to the current CPU
void install() const;
/// Add the IST entries listed in the ISR table into the IDT.
/// This can't be done until after memory is set up so the
/// stacks can be created.
void add_ist_entries();
/// Get the IST entry used by an entry.
/// \arg i Which IDT entry to look in
/// \returns The IST index used by entry i, or 0 for none
inline uint8_t get_ist(uint8_t i) const {
return m_entries[i].ist;
}
/// Set the IST entry used by an entry.
/// \arg i Which IDT entry to set
/// \arg ist The IST index for entry i, or 0 for none
void set_ist(uint8_t i, uint8_t ist) { m_entries[i].ist = ist; }
/// Get the IST entries that are used by this table, as a bitmap
uint8_t used_ist_entries() const;
/// Dump debug information about the IDT to the console.
/// \arg index Which entry to print, or -1 for all entries
void dump(unsigned index = -1) const;
/// Get the global IDT
static IDT & get();
private:
void set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags);
struct descriptor
{
uint16_t base_low;
uint16_t selector;
uint8_t ist;
uint8_t flags;
uint16_t base_mid;
uint32_t base_high;
uint32_t reserved; // must be zero
} __attribute__ ((packed, aligned(16)));
struct ptr
{
uint16_t limit;
descriptor *base;
} __attribute__ ((packed, aligned(4)));
descriptor m_entries[256];
ptr m_ptr;
};

108
src/kernel/interrupt_isrs.inc
View File

@@ -1,36 +1,36 @@
ISR (0x00, 0, isrDivideByZero)
ISR (0x01, 0, isrDebug)
ISR (0x02, 1, isrNMI)
ISR (0x03, 0, isrBreakpoint)
ISR (0x04, 0, isrOverflow)
ISR (0x05, 0, isrBRE)
ISR (0x06, 0, isrInvalidOp)
ISR (0x07, 0, isrDNA)
EISR(0x08, 2, isrDoubleFault)
ISR (0x09, 0, isrCoprocessor)
EISR(0x0a, 0, isrInvalidTSS)
EISR(0x0b, 0, isrSegmentNP)
EISR(0x0c, 0, isrSSFault)
EISR(0x0d, 0, isrGPFault)
EISR(0x0e, 3, isrPageFault)
ISR (0x0f, 0, isr15)
ISR (0x00, isrDivideByZero)
ISR (0x01, isrDebug)
ISR (0x02, isrNMI)
ISR (0x03, isrBreakpoint)
ISR (0x04, isrOverflow)
ISR (0x05, isrBRE)
ISR (0x06, isrInvalidOp)
ISR (0x07, isrDNA)
EISR(0x08, isrDoubleFault)
ISR (0x09, isrCoprocessor)
EISR(0x0a, isrInvalidTSS)
EISR(0x0b, isrSegmentNP)
EISR(0x0c, isrSSFault)
EISR(0x0d, isrGPFault)
EISR(0x0e, isrPageFault)
ISR (0x0f, isr15)
ISR (0x10, 0, isrX87FPE)
ISR (0x11, 0, isrAlignmentChk)
ISR (0x12, 0, isrMachineChk)
ISR (0x13, 0, isrSIMDFPE)
ISR (0x14, 0, isrVirt)
ISR (0x15, 0, isr21)
ISR (0x16, 0, isr22)
ISR (0x17, 0, isr23)
ISR (0x18, 0, isr24)
ISR (0x19, 0, isr25)
ISR (0x1a, 0, isr26)
ISR (0x1b, 0, isr27)
ISR (0x1c, 0, isr28)
ISR (0x1d, 0, isr29)
ISR (0x1e, 0, isrSecurity)
ISR (0x1f, 0, isr31)
ISR (0x10, isrX87FPE)
ISR (0x11, isrAlignmentChk)
ISR (0x12, isrMachineChk)
ISR (0x13, isrSIMDFPE)
ISR (0x14, isrVirt)
ISR (0x15, isr21)
ISR (0x16, isr22)
ISR (0x17, isr23)
ISR (0x18, isr24)
ISR (0x19, isr25)
ISR (0x1a, isr26)
ISR (0x1b, isr27)
ISR (0x1c, isr28)
ISR (0x1d, isr29)
ISR (0x1e, isrSecurity)
ISR (0x1f, isr31)
IRQ (0x20, 0x00, irq00)
IRQ (0x21, 0x01, irq01)
@@ -237,28 +237,28 @@ IRQ (0xde, 0xbe, irqBE)
IRQ (0xdf, 0xbf, irqBF)
ISR (0xe0, 0, isrTimer)
ISR (0xe1, 0, isrLINT0)
ISR (0xe2, 0, isrLINT1)
ISR (0xe3, 0, isrAPICError)
ISR (0xe4, 0, isrAssert)
ISR (0xe0, isrTimer)
ISR (0xe1, isrLINT0)
ISR (0xe2, isrLINT1)
ISR (0xe4, isrAssert)
ISR (0xef, 0, isrSpurious)
UISR(0xee, isrSyscall)
ISR (0xef, isrSpurious)
ISR (0xf0, 0, isrIgnore0)
ISR (0xf1, 0, isrIgnore1)
ISR (0xf2, 0, isrIgnore2)
ISR (0xf3, 0, isrIgnore3)
ISR (0xf4, 0, isrIgnore4)
ISR (0xf5, 0, isrIgnore5)
ISR (0xf6, 0, isrIgnore6)
ISR (0xf7, 0, isrIgnore7)
ISR (0xf0, isrIgnore0)
ISR (0xf1, isrIgnore1)
ISR (0xf2, isrIgnore2)
ISR (0xf3, isrIgnore3)
ISR (0xf4, isrIgnore4)
ISR (0xf5, isrIgnore5)
ISR (0xf6, isrIgnore6)
ISR (0xf7, isrIgnore7)
ISR (0xf8, 0, isrIgnore8)
ISR (0xf9, 0, isrIgnore9)
ISR (0xfa, 0, isrIgnoreA)
ISR (0xfb, 0, isrIgnoreB)
ISR (0xfc, 0, isrIgnoreC)
ISR (0xfd, 0, isrIgnoreD)
ISR (0xfe, 0, isrIgnoreE)
ISR (0xff, 0, isrIgnoreF)
ISR (0xf8, isrIgnore8)
ISR (0xf9, isrIgnore9)
ISR (0xfa, isrIgnoreA)
ISR (0xfb, isrIgnoreB)
ISR (0xfc, isrIgnoreC)
ISR (0xfd, isrIgnoreD)
ISR (0xfe, isrIgnoreE)
ISR (0xff, isrIgnoreF)

68
src/kernel/interrupts.cpp
View File

@@ -8,7 +8,6 @@
#include "debug.h"
#include "device_manager.h"
#include "gdt.h"
#include "idt.h"
#include "interrupts.h"
#include "io.h"
#include "kernel_memory.h"
@@ -16,7 +15,6 @@
#include "objects/process.h"
#include "scheduler.h"
#include "syscall.h"
#include "tss.h"
#include "vm_space.h"
static const uint16_t PIC1 = 0x20;
@@ -24,14 +22,21 @@ static const uint16_t PIC2 = 0xa0;
constexpr uintptr_t apic_eoi_addr = 0xfee000b0 + ::memory::page_offset;
constexpr size_t increment_offset = 0x1000;
extern "C" {
void _halt();
void isr_handler(cpu_state*);
void irq_handler(cpu_state*);
#define ISR(i, name) extern void name ();
#define EISR(i, name) extern void name ();
#define UISR(i, name) extern void name ();
#define IRQ(i, q, name) extern void name ();
#include "interrupt_isrs.inc"
#undef IRQ
#undef UISR
#undef EISR
#undef ISR
}
isr
@@ -45,11 +50,13 @@ uint8_t
get_irq(unsigned vector)
{
switch (vector) {
#define ISR(i, s, name)
#define EISR(i, s, name)
#define ISR(i, name)
#define EISR(i, name)
#define UISR(i, name)
#define IRQ(i, q, name) case i : return q;
#include "interrupt_isrs.inc"
#undef IRQ
#undef UISR
#undef EISR
#undef ISR
@@ -57,7 +64,7 @@ get_irq(unsigned vector)
}
}
void
static void
disable_legacy_pic()
{
// Mask all interrupts
@@ -77,19 +84,30 @@ disable_legacy_pic()
outb(PIC2+1, 0x02); io_wait();
}
void
interrupts_init()
{
#define ISR(i, name) idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
#define EISR(i, name) idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
#define UISR(i, name) idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0xee);
#define IRQ(i, q, name) idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
#include "interrupt_isrs.inc"
#undef IRQ
#undef UISR
#undef EISR
#undef ISR
disable_legacy_pic();
log::info(logs::boot, "Interrupts enabled.");
}
void
isr_handler(cpu_state *regs)
{
console *cons = console::get();
uint8_t vector = regs->interrupt & 0xff;
// Clear out the IST for this vector so we just keep using
// this stack
uint8_t old_ist = IDT::get().get_ist(vector);
if (old_ist)
IDT::get().set_ist(vector, 0);
switch (static_cast<isr>(vector)) {
switch (static_cast<isr>(regs->interrupt & 0xff)) {
case isr::isrDebug: {
cons->set_color(11);
@@ -123,16 +141,6 @@ isr_handler(cpu_state *regs)
}
break;
case isr::isrDoubleFault:
cons->set_color(9);
cons->printf("\nDouble Fault:\n");
cons->set_color();
print_regs(*regs);
print_stacktrace(2);
_halt();
break;
case isr::isrGPFault: {
cons->set_color(9);
cons->puts("\nGeneral Protection Fault:\n");
@@ -146,13 +154,13 @@ isr_handler(cpu_state *regs)
switch ((regs->errorcode & 0x07) >> 1) {
case 0:
cons->printf(" GDT[%x]\n", index);
GDT::current().dump(index);
gdt_dump(index);
break;
case 1:
case 3:
cons->printf(" IDT[%x]\n", index);
IDT::get().dump(index);
idt_dump(index);
break;
default:
@@ -271,10 +279,6 @@ isr_handler(cpu_state *regs)
print_stacktrace(2);
_halt();
}
// Return the IST for this vector to what it was
if (old_ist)
IDT::get().set_ist(vector, old_ist);
*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;
}
@@ -288,8 +292,8 @@ irq_handler(cpu_state *regs)
cons->printf("\nReceived unknown IRQ: %d (vec %d)\n",
irq, regs->interrupt);
cons->set_color();
print_regs(*regs);
_halt();
}
*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;

11
src/kernel/interrupts.h
View File

@@ -7,11 +7,13 @@
/// Enum of all defined ISR/IRQ vectors
enum class isr : uint8_t
{
#define ISR(i, s, name) name = i,
#define EISR(i, s, name) name = i,
#define ISR(i, name) name = i,
#define EISR(i, name) name = i,
#define UISR(i, name) name = i,
#define IRQ(i, q, name) name = i,
#include "interrupt_isrs.inc"
#undef IRQ
#undef UISR
#undef EISR
#undef ISR
@@ -29,5 +31,6 @@ extern "C" {
void interrupts_disable();
}
/// Disable the legacy PIC
void disable_legacy_pic();
/// Fill the IDT with our ISRs, and disable the legacy
/// PIC interrupts.
void interrupts_init();

39
src/kernel/interrupts.s
View File

@@ -1,14 +1,8 @@
%include "push_all.inc"
section .text
extern isr_handler
global isr_handler_prelude:function (isr_handler_prelude.end - isr_handler_prelude)
global isr_handler_prelude
isr_handler_prelude:
push rbp ; Never executed, fake function prelude
mov rbp, rsp ; to calm down gdb
.real:
push_all
check_swap_gs
@@ -16,15 +10,10 @@ isr_handler_prelude:
mov rsi, rsp
call isr_handler
jmp isr_handler_return
.end:
extern irq_handler
global irq_handler_prelude:function (irq_handler_prelude.end - irq_handler_prelude)
global irq_handler_prelude
irq_handler_prelude:
push rbp ; Never executed, fake function prelude
mov rbp, rsp ; to calm down gdb
.real:
push_all
check_swap_gs
@@ -32,45 +21,41 @@ irq_handler_prelude:
mov rsi, rsp
call irq_handler
; fall through to isr_handler_return
.end:
global isr_handler_return:function (isr_handler_return.end - isr_handler_return)
global isr_handler_return
isr_handler_return:
check_swap_gs
pop_all
add rsp, 16 ; because the ISRs added err/num
iretq
.end:
%macro EMIT_ISR 2
global %1:function (%1.end - %1)
global %1
%1:
push 0
push %2
jmp isr_handler_prelude.real
.end:
jmp isr_handler_prelude
%endmacro
%macro EMIT_EISR 2
global %1:function (%1.end - %1)
global %1
%1:
push %2
jmp isr_handler_prelude.real
.end:
jmp isr_handler_prelude
%endmacro
%macro EMIT_IRQ 2
global %1:function (%1.end - %1)
global %1
%1:
push 0
push %2
jmp irq_handler_prelude.real
.end:
jmp irq_handler_prelude
%endmacro
%define EISR(i, s, name) EMIT_EISR name, i ; ISR with error code
%define ISR(i, s, name) EMIT_ISR name, i
%define EISR(i, name) EMIT_EISR name, i ; ISR with error code
%define UISR(i, name) EMIT_ISR name, i ; ISR callable from user space
%define ISR(i, name) EMIT_ISR name, i
%define IRQ(i, q, name) EMIT_IRQ name, i
section .isrs

24
src/kernel/loader.s Normal file
View File

@@ -0,0 +1,24 @@
%include "push_all.inc"
extern load_process_image
global preloaded_process_init
preloaded_process_init:
; create_process already pushed the arguments for load_process_image and
; the following iretq onto the stack for us
pop rdi ; the physical address of the program image
pop rsi ; the virtual address of the program image
pop rdx ; the size in bytes of the program image
pop rcx ; the address of this thread's TCB
call load_process_image
; user rsp is now in rax, put it in the right place for iret
mov [rsp + 0x18], rax
; the entrypoint should already be on the stack
swapgs
iretq

41
src/kernel/log.cpp
View File

@@ -1,10 +1,8 @@
#include "j6/signals.h"
#include "kutil/memory.h"
#include "kutil/no_construct.h"
#include "console.h"
#include "log.h"
#include "objects/system.h"
#include "objects/thread.h"
#include "scheduler.h"
static uint8_t log_buffer[0x10000];
@@ -28,54 +26,29 @@ output_log(log::area_t area, log::level severity, const char *message)
cons->set_color();
}
static void
log_flush()
{
system &sys = system::get();
sys.assert_signal(j6_signal_system_has_log);
}
void
logger_task()
{
uint8_t buffer[257];
auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
auto *cons = console::get();
log::info(logs::task, "Starting kernel logger task");
g_logger.set_immediate(nullptr);
g_logger.set_flush(log_flush);
thread &self = thread::current();
system &sys = system::get();
size_t buffer_size = 1;
uint8_t *buffer = nullptr;
scheduler &s = scheduler::get();
while (true) {
size_t size = g_logger.get_entry(buffer, buffer_size);
if (size > buffer_size) {
while (size > buffer_size) buffer_size *= 2;
kutil::kfree(buffer);
buffer = reinterpret_cast<uint8_t*>(kutil::kalloc(buffer_size));
kassert(buffer, "Could not allocate logger task buffer");
continue;
}
if(size) {
auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
if(g_logger.get_entry(buffer, sizeof(buffer))) {
buffer[ent->bytes] = 0;
cons->set_color(level_colors[static_cast<int>(ent->severity)]);
cons->printf("%7s %5s: %s\n",
g_logger.area_name(ent->area),
g_logger.level_name(ent->severity),
ent->message);
cons->set_color();
}
if (!g_logger.has_log()) {
sys.deassert_signal(j6_signal_system_has_log);
sys.add_blocked_thread(&self);
self.wait_on_signals(&sys, j6_signal_system_has_log);
} else {
s.schedule();
}
}
}

285
src/kernel/main.cpp
View File

@@ -6,28 +6,22 @@
#include "kutil/assert.h"
#include "apic.h"
#include "block_device.h"
#include "clock.h"
#include "console.h"
#include "cpu.h"
#include "device_manager.h"
#include "gdt.h"
#include "idt.h"
#include "interrupts.h"
#include "io.h"
#include "kernel_args.h"
#include "kernel_memory.h"
#include "log.h"
#include "msr.h"
#include "objects/channel.h"
#include "objects/event.h"
#include "objects/thread.h"
#include "objects/vm_area.h"
#include "scheduler.h"
#include "serial.h"
#include "symbol_table.h"
#include "syscall.h"
#include "tss.h"
#include "vm_space.h"
#ifndef GIT_VERSION
#define GIT_VERSION
@@ -37,25 +31,16 @@ extern "C" {
void kernel_main(kernel::args::header *header);
void (*__ctors)(void);
void (*__ctors_end)(void);
void long_ap_startup(cpu_data *cpu);
void ap_startup();
void ap_idle();
void init_ap_trampoline(void*, cpu_data *, void (*)());
}
extern void __kernel_assert(const char *, unsigned, const char *);
using namespace kernel;
volatile size_t ap_startup_count;
static bool scheduler_ready = false;
/// Bootstrap the memory managers.
void memory_initialize_pre_ctors(args::header &kargs);
void memory_initialize_post_ctors(args::header &kargs);
process * load_simple_process(args::program &program);
void setup_pat();
void memory_initialize_pre_ctors(kernel::args::header *kargs);
void memory_initialize_post_ctors(kernel::args::header *kargs);
unsigned start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4);
using namespace kernel;
/// TODO: not this. this is awful.
args::framebuffer *fb = nullptr;
@@ -82,6 +67,46 @@ run_constructors()
}
}
channel *std_out = nullptr;
void
stdout_task()
{
uint8_t buffer[257];
auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
auto *cons = console::get();
log::info(logs::task, "Starting kernel stdout task");
scheduler &s = scheduler::get();
thread *th = thread::from_tcb(s.current());
while (true) {
j6_signal_t current = std_out->signals();
if (!(current & j6_signal_channel_can_recv)) {
th->wait_on_signals(std_out, j6_signal_channel_can_recv);
s.schedule();
}
size_t n = 256;
j6_status_t status = std_out->dequeue(&n, buffer);
if (status != j6_status_ok) {
log::warn(logs::task, "Kernel stdout error: %x", status);
return;
}
buffer[n] = 0;
const char *s = reinterpret_cast<const char *>(buffer);
while (n) {
size_t r = cons->puts(s);
n -= r + 1;
s += r + 1;
}
}
}
void
kernel_main(args::header *header)
{
@@ -90,64 +115,30 @@ kernel_main(args::header *header)
init_console();
logger_init();
cpu_validate();
log::debug(logs::boot, " jsix header is at: %016lx", header);
log::debug(logs::boot, " Memory map is at: %016lx", header->mem_map);
log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
log::debug(logs::boot, " Kernel PML4 is at: %016lx", header->pml4);
uint64_t cr0, cr4;
asm ("mov %%cr0, %0" : "=r"(cr0));
asm ("mov %%cr4, %0" : "=r"(cr4));
uint64_t efer = rdmsr(msr::ia32_efer);
log::debug(logs::boot, "Control regs: cr0:%lx cr4:%lx efer:%lx", cr0, cr4, efer);
setup_pat();
bool has_video = false;
if (header->video.size > 0) {
has_video = true;
fb = &header->video;
fb = memory::to_virtual<args::framebuffer>(reinterpret_cast<uintptr_t>(&header->video));
const args::framebuffer &video = header->video;
log::debug(logs::boot, "Framebuffer: %dx%d[%d] type %d @ %llx size %llx",
video.horizontal,
video.vertical,
video.scanline,
video.type,
video.phys_addr,
video.size);
log::debug(logs::boot, "Framebuffer: %dx%d[%d] type %s @ %016llx",
video.horizontal, video.vertical, video.scanline, video.type, video.phys_addr);
logger_clear_immediate();
}
extern IDT &g_idt;
extern TSS &g_bsp_tss;
extern GDT &g_bsp_gdt;
extern cpu_data g_bsp_cpu_data;
extern uintptr_t idle_stack_end;
gdt_init();
interrupts_init();
IDT *idt = new (&g_idt) IDT;
cpu_data *cpu = &g_bsp_cpu_data;
kutil::memset(cpu, 0, sizeof(cpu_data));
cpu->self = cpu;
cpu->tss = new (&g_bsp_tss) TSS;
cpu->gdt = new (&g_bsp_gdt) GDT {cpu->tss};
cpu->rsp0 = idle_stack_end;
cpu_early_init(cpu);
disable_legacy_pic();
memory_initialize_pre_ctors(*header);
memory_initialize_pre_ctors(header);
run_constructors();
memory_initialize_post_ctors(*header);
memory_initialize_post_ctors(header);
cpu->tss->create_ist_stacks(idt->used_ist_entries());
cpu_validate();
for (size_t i = 0; i < header->num_modules; ++i) {
args::module &mod = header->modules[i];
switch (mod.type) {
case args::mod_type::symbol_table:
new symbol_table {mod.location, mod.size};
@@ -158,30 +149,17 @@ kernel_main(args::header *header)
}
}
syscall_initialize();
log::debug(logs::boot, " jsix header is at: %016lx", header);
log::debug(logs::boot, " Memory map is at: %016lx", header->mem_map);
log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
device_manager &devices = device_manager::get();
devices.parse_acpi(header->acpi_table);
// Need the local APIC to get the BSP's id
uintptr_t apic_base = devices.get_lapic_base();
lapic *apic = new lapic(apic_base);
apic->enable();
cpu->id = apic->get_id();
cpu->apic = apic;
cpu_init(cpu, true);
devices.init_drivers();
apic->calibrate_timer();
const auto &apic_ids = devices.get_apic_ids();
unsigned num_cpus = start_aps(*apic, apic_ids, header->pml4);
idt->add_ist_entries();
interrupts_enable();
devices.init_drivers();
devices.get_lapic()->calibrate_timer();
/*
block_device *disk = devices->get_block_device(0);
@@ -207,138 +185,27 @@ kernel_main(args::header *header)
}
*/
scheduler *sched = new scheduler {num_cpus};
scheduler_ready = true;
syscall_enable();
scheduler *sched = new scheduler(devices.get_lapic());
std_out = new channel;
// Skip program 0, which is the kernel itself
for (unsigned i = 1; i < header->num_programs; ++i)
load_simple_process(header->programs[i]);
for (size_t i = 1; i < header->num_programs; ++i) {
args::program &prog = header->programs[i];
thread *th = sched->load_process(prog.phys_addr, prog.virt_addr, prog.size, prog.entrypoint);
if (i == 2) {
//th->set_state(thread::state::constant);
}
}
if (!has_video)
sched->create_kernel_task(logger_task, scheduler::max_priority/2, true);
sched->create_kernel_task(logger_task, scheduler::max_priority-1, true);
sched->create_kernel_task(stdout_task, scheduler::max_priority-1, true);
const char stdout_message[] = "Hello on the fake stdout channel\n";
size_t message_size = sizeof(stdout_message);
std_out->enqueue(&message_size, reinterpret_cast<const void*>(stdout_message));
sched->start();
}
unsigned
start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4)
{
using memory::frame_size;
using memory::kernel_stack_pages;
extern size_t ap_startup_code_size;
extern process &g_kernel_process;
extern vm_area_guarded &g_kernel_stacks;
clock &clk = clock::get();
ap_startup_count = 1; // BSP processor
log::info(logs::boot, "Starting %d other CPUs", ids.count() - 1);
// Since we're using address space outside kernel space, make sure
// the kernel's vm_space is used
cpu_data &bsp = current_cpu();
bsp.process = &g_kernel_process;
uint16_t index = bsp.index;
// Copy the startup code somwhere the real mode trampoline can run
uintptr_t addr = 0x8000; // TODO: find a valid address, rewrite addresses
uint8_t vector = addr >> 12;
vm_area *vma = new vm_area_fixed(addr, 0x1000, vm_flags::write);
vm_space::kernel_space().add(addr, vma);
kutil::memcpy(
reinterpret_cast<void*>(addr),
reinterpret_cast<void*>(&ap_startup),
ap_startup_code_size);
// AP idle stacks need less room than normal stacks, so pack multiple
// into a normal stack area
static constexpr size_t idle_stack_bytes = 2048; // 2KiB is generous
static constexpr size_t full_stack_bytes = kernel_stack_pages * frame_size;
static constexpr size_t idle_stacks_per = full_stack_bytes / idle_stack_bytes;
uint8_t ist_entries = IDT::get().used_ist_entries();
size_t free_stack_count = 0;
uintptr_t stack_area_start = 0;
ipi mode = ipi::init | ipi::level | ipi::assert;
apic.send_ipi_broadcast(mode, false, 0);
for (uint8_t id : ids) {
if (id == bsp.id) continue;
// Set up the CPU data structures
TSS *tss = new TSS;
GDT *gdt = new GDT {tss};
cpu_data *cpu = new cpu_data;
kutil::memset(cpu, 0, sizeof(cpu_data));
cpu->self = cpu;
cpu->id = id;
cpu->index = ++index;
cpu->gdt = gdt;
cpu->tss = tss;
tss->create_ist_stacks(ist_entries);
// Set up the CPU's idle task stack
if (free_stack_count == 0) {
stack_area_start = g_kernel_stacks.get_section();
free_stack_count = idle_stacks_per;
}
uintptr_t stack_end = stack_area_start + free_stack_count-- * idle_stack_bytes;
stack_end -= 2 * sizeof(void*); // Null frame
*reinterpret_cast<uint64_t*>(stack_end) = 0; // pre-fault the page
cpu->rsp0 = stack_end;
// Set up the trampoline with this CPU's data
init_ap_trampoline(kpml4, cpu, ap_idle);
// Kick it off!
size_t current_count = ap_startup_count;
log::debug(logs::boot, "Starting AP %d: stack %llx", cpu->index, stack_end);
ipi startup = ipi::startup | ipi::assert;
apic.send_ipi(startup, vector, id);
for (unsigned i = 0; i < 20; ++i) {
if (ap_startup_count > current_count) break;
clk.spinwait(20);
}
// If the CPU already incremented ap_startup_count, it's done
if (ap_startup_count > current_count)
continue;
// Send the second SIPI (intel recommends this)
apic.send_ipi(startup, vector, id);
for (unsigned i = 0; i < 100; ++i) {
if (ap_startup_count > current_count) break;
clk.spinwait(100);
}
log::warn(logs::boot, "No response from AP %d within timeout", id);
}
log::info(logs::boot, "%d CPUs running", ap_startup_count);
vm_space::kernel_space().remove(vma);
return ap_startup_count;
}
void
long_ap_startup(cpu_data *cpu)
{
cpu_init(cpu, false);
++ap_startup_count;
while (!scheduler_ready) asm ("pause");
uintptr_t apic_base =
device_manager::get().get_lapic_base();
cpu->apic = new lapic(apic_base);
cpu->apic->enable();
scheduler::get().start();
}

274
src/kernel/memory_bootstrap.cpp
View File

@@ -1,31 +1,31 @@
#include <utility>
#include "kernel_args.h"
#include "j6/init.h"
#include "kutil/assert.h"
#include "kutil/heap_allocator.h"
#include "kutil/no_construct.h"
#include "device_manager.h"
#include "frame_allocator.h"
#include "gdt.h"
#include "io.h"
#include "log.h"
#include "msr.h"
#include "objects/process.h"
#include "objects/thread.h"
#include "objects/system.h"
#include "objects/vm_area.h"
#include "vm_space.h"
using memory::frame_size;
using memory::heap_start;
using memory::kernel_max_heap;
using memory::kernel_offset;
using memory::heap_start;
using memory::page_offset;
using memory::pml4e_kernel;
using memory::pml4e_offset;
using memory::table_entries;
using namespace kernel;
extern "C" void initialize_main_thread();
extern "C" uintptr_t initialize_main_user_stack();
// These objects are initialized _before_ global constructors are called,
// so we don't want them to have global constructors at all, lest they
@@ -36,17 +36,20 @@ kutil::heap_allocator &g_kernel_heap = __g_kernel_heap_storage.value;
static kutil::no_construct<frame_allocator> __g_frame_allocator_storage;
frame_allocator &g_frame_allocator = __g_frame_allocator_storage.value;
static kutil::no_construct<vm_area_untracked> __g_kernel_heap_area_storage;
vm_area_untracked &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
static kutil::no_construct<vm_area_open> __g_kernel_heap_area_storage;
vm_area_open &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
static kutil::no_construct<vm_area_guarded> __g_kernel_stacks_storage;
vm_area_guarded &g_kernel_stacks = __g_kernel_stacks_storage.value;
vm_area_buffers g_kernel_stacks {
memory::kernel_max_stacks,
vm_space::kernel_space(),
vm_flags::write,
memory::kernel_stack_pages};
vm_area_guarded g_kernel_buffers {
memory::buffers_start,
memory::kernel_buffer_pages,
vm_area_buffers g_kernel_buffers {
memory::kernel_max_buffers,
vm_flags::write};
vm_space::kernel_space(),
vm_flags::write,
memory::kernel_buffer_pages};
void * operator new(size_t size) { return g_kernel_heap.allocate(size); }
void * operator new [] (size_t size) { return g_kernel_heap.allocate(size); }
@@ -54,88 +57,49 @@ void operator delete (void *p) noexcept { return g_kernel_heap.free(p); }
void operator delete [] (void *p) noexcept { return g_kernel_heap.free(p); }
namespace kutil {
void * kalloc(size_t size) { return g_kernel_heap.allocate(size); }
void kfree(void *p) { return g_kernel_heap.free(p); }
void * kalloc(size_t size) { return g_kernel_heap.allocate(size); }
void kfree(void *p) { return g_kernel_heap.free(p); }
}
template <typename T>
uintptr_t
get_physical_page(T *p) {
return memory::page_align_down(reinterpret_cast<uintptr_t>(p));
}
void
memory_initialize_pre_ctors(args::header &kargs)
/*
void walk_page_table(
page_table *table,
page_table::level level,
uintptr_t &current_start,
size_t &current_bytes,
vm_area &karea)
{
using kernel::args::frame_block;
constexpr size_t huge_page_size = (1ull<<30);
constexpr size_t large_page_size = (1ull<<21);
page_table *kpml4 = static_cast<page_table*>(kargs.pml4);
for (unsigned i = 0; i < table_entries; ++i) {
page_table *next = table->get(i);
if (!next) {
if (current_bytes)
karea.commit(current_start, current_bytes);
current_start = 0;
current_bytes = 0;
continue;
} else if (table->is_page(level, i)) {
if (!current_bytes)
current_start = reinterpret_cast<uintptr_t>(next);
current_bytes +=
(level == page_table::level::pt
? frame_size
: level == page_table::level::pd
? large_page_size
: huge_page_size);
} else {
page_table::level deeper =
static_cast<page_table::level>(
static_cast<unsigned>(level) + 1);
new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
frame_block *blocks = reinterpret_cast<frame_block*>(memory::bitmap_start);
new (&g_frame_allocator) frame_allocator {blocks, kargs.frame_block_count};
// Mark all the things the bootloader allocated for us as used
g_frame_allocator.used(
get_physical_page(&kargs),
memory::page_count(sizeof(kargs)));
g_frame_allocator.used(
get_physical_page(kargs.frame_blocks),
kargs.frame_block_pages);
g_frame_allocator.used(
get_physical_page(kargs.pml4),
kargs.table_pages);
for (unsigned i = 0; i < kargs.num_modules; ++i) {
const kernel::args::module &mod = kargs.modules[i];
g_frame_allocator.used(
get_physical_page(mod.location),
memory::page_count(mod.size));
}
for (unsigned i = 0; i < kargs.num_programs; ++i) {
const kernel::args::program &prog = kargs.programs[i];
for (auto &sect : prog.sections) {
if (!sect.size) continue;
g_frame_allocator.used(
sect.phys_addr,
memory::page_count(sect.size));
walk_page_table(
next, deeper, current_start, current_bytes, kspace);
}
}
process *kp = process::create_kernel_process(kpml4);
vm_space &vm = kp->space();
vm_area *heap = new (&g_kernel_heap_area)
vm_area_untracked(kernel_max_heap, vm_flags::write);
vm.add(heap_start, heap);
vm_area *stacks = new (&g_kernel_stacks) vm_area_guarded {
memory::stacks_start,
memory::kernel_stack_pages,
memory::kernel_max_stacks,
vm_flags::write};
vm.add(memory::stacks_start, &g_kernel_stacks);
// Clean out any remaning bootloader page table entries
for (unsigned i = 0; i < memory::pml4e_kernel; ++i)
kpml4->entries[i] = 0;
}
void
memory_initialize_post_ctors(args::header &kargs)
{
vm_space &vm = vm_space::kernel_space();
vm.add(memory::buffers_start, &g_kernel_buffers);
g_frame_allocator.free(
get_physical_page(kargs.page_tables),
kargs.table_count);
}
*/
static void
log_mtrrs()
@@ -192,104 +156,70 @@ log_mtrrs()
pat_names[(pat >> (6*8)) & 7], pat_names[(pat >> (7*8)) & 7]);
}
process *
load_simple_process(args::program &program)
void
setup_pat()
{
using kernel::args::section_flags;
uint64_t pat = rdmsr(msr::ia32_pat);
pat = (pat & 0x00ffffffffffffffull) | (0x01ull << 56); // set PAT 7 to WC
wrmsr(msr::ia32_pat, pat);
log_mtrrs();
}
process *p = new process;
vm_space &space = p->space();
for (const auto &sect : program.sections) {
vm_flags flags =
(bitfield_has(sect.type, section_flags::execute) ? vm_flags::exec : vm_flags::none) |
(bitfield_has(sect.type, section_flags::write) ? vm_flags::write : vm_flags::none);
void
memory_initialize_pre_ctors(args::header *kargs)
{
new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
new (&g_frame_allocator) frame_allocator;
vm_area *vma = new vm_area_fixed(sect.phys_addr, sect.size, flags);
space.add(sect.virt_addr, vma);
args::mem_entry *entries = kargs->mem_map;
const size_t count = kargs->map_count;
for (unsigned i = 0; i < count; ++i) {
// TODO: use entry attributes
// TODO: copy anything we need from "pending" memory and free it
args::mem_entry &e = entries[i];
if (e.type == args::mem_type::free)
g_frame_allocator.free(e.start, e.pages);
}
uint64_t iopl = (3ull << 12);
uintptr_t trampoline = reinterpret_cast<uintptr_t>(initialize_main_thread);
page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
process *kp = process::create_kernel_process(kpml4);
vm_space &vm = kp->space();
thread *main = p->create_thread();
main->add_thunk_user(program.entrypoint, trampoline, iopl);
main->set_state(thread::state::ready);
vm_area *heap = new (&g_kernel_heap_area)
vm_area_open(memory::kernel_max_heap, vm, vm_flags::write);
return p;
vm.add(memory::heap_start, heap);
}
template <typename T>
inline T * push(uintptr_t &rsp, size_t size = sizeof(T)) {
rsp -= size;
T *p = reinterpret_cast<T*>(rsp);
rsp &= ~(sizeof(uint64_t)-1); // Align the stack
return p;
}
uintptr_t
initialize_main_user_stack()
void
memory_initialize_post_ctors(args::header *kargs)
{
process &proc = process::current();
thread &th = thread::current();
TCB *tcb = th.tcb();
/*
uintptr_t current_start = 0;
size_t current_bytes = 0;
const char message[] = "Hello from the kernel!";
char *message_arg = push<char>(tcb->rsp3, sizeof(message));
kutil::memcpy(message_arg, message, sizeof(message));
// TODO: Should we exclude the top of this area? (eg, buffers, stacks, etc)
page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
for (unsigned i = pml4e_kernel; i < pml4e_offset; ++i) {
page_table *pdp = kpml4->get(i);
kassert(pdp, "Bootloader did not create all kernelspace PDs");
j6_init_value *initv = nullptr;
unsigned n = 0;
extern args::framebuffer *fb;
if (fb) {
j6_init_framebuffer *fb_desc = push<j6_init_framebuffer>(tcb->rsp3);
kutil::memset(fb_desc, 0, sizeof(j6_init_framebuffer));
fb_desc->addr = fb->phys_addr;
fb_desc->size = fb->size;
fb_desc->vertical = fb->vertical;
fb_desc->horizontal = fb->horizontal;
fb_desc->scanline = fb->scanline;
if (fb->type == kernel::args::fb_type::bgr8)
fb_desc->flags |= 1;
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_desc_framebuffer;
initv->data = fb_desc;
++n;
walk_page_table(
pdp, page_table::level::pdp,
current_start, current_bytes,
g_kernel_space);
}
if (current_bytes)
g_kernel_space.commit(current_start, current_bytes);
*/
vm_space &vm = vm_space::kernel_space();
vm.add(memory::stacks_start, &g_kernel_stacks);
vm.add(memory::buffers_start, &g_kernel_buffers);
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_other;
initv->handle.type = j6_object_type_system;
initv->handle.handle = proc.add_handle(&system::get());
++n;
g_frame_allocator.free(
reinterpret_cast<uintptr_t>(kargs->page_tables),
kargs->table_count);
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_self;
initv->handle.type = j6_object_type_process;
initv->handle.handle = proc.self_handle();
++n;
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_self;
initv->handle.type = j6_object_type_thread;
initv->handle.handle = th.self_handle();
++n;
uint64_t *initc = push<uint64_t>(tcb->rsp3);
*initc = n;
char **argv0 = push<char*>(tcb->rsp3);
*argv0 = message_arg;
uint64_t *argc = push<uint64_t>(tcb->rsp3);
*argc = 1;
th.clear_state(thread::state::loading);
return tcb->rsp3;
}

6
src/kernel/objects/channel.cpp
View File

@@ -4,13 +4,13 @@
#include "objects/channel.h"
#include "objects/vm_area.h"
extern vm_area_guarded g_kernel_buffers;
extern vm_area_buffers g_kernel_buffers;
constexpr size_t buffer_bytes = memory::kernel_buffer_pages * memory::frame_size;
channel::channel() :
m_len(0),
m_data(g_kernel_buffers.get_section()),
m_data(g_kernel_buffers.get_buffer()),
m_buffer(reinterpret_cast<uint8_t*>(m_data), buffer_bytes),
kobject(kobject::type::channel, j6_signal_channel_can_send)
{
@@ -79,7 +79,7 @@ void
channel::close()
{
kobject::close();
g_kernel_buffers.return_section(m_data);
g_kernel_buffers.return_buffer(m_data);
}
void

14
src/kernel/objects/kobject.h
View File

@@ -16,9 +16,17 @@ public:
/// Types of kernel objects.
enum class type : uint16_t
{
#define OBJECT_TYPE( name, val ) name = val,
#include "j6/tables/object_types.inc"
#undef OBJECT_TYPE
none,
system,
event,
channel,
endpoint,
vma,
process,
thread,
max
};

27
src/kernel/objects/process.cpp
View File

@@ -13,13 +13,17 @@ static kutil::no_construct<process> __g_kernel_process_storage;
process &g_kernel_process = __g_kernel_process_storage.value;
kutil::vector<process*> process::s_processes;
process::process() :
kobject {kobject::type::process},
m_next_handle {1},
m_next_handle {0},
m_state {state::running}
{
s_processes.append(this);
j6_handle_t self = add_handle(this);
kassert(self == self_handle(), "Process self-handle is not 1");
kassert(self == self_handle(), "Process self-handle is not 0");
}
// The "kernel process"-only constructor
@@ -35,9 +39,10 @@ process::~process()
{
for (auto &it : m_handles)
if (it.val) it.val->handle_release();
s_processes.remove_swap(this);
}
process & process::current() { return *current_cpu().process; }
process & process::current() { return *bsp_cpu_data.p; }
process & process::kernel_process() { return g_kernel_process; }
process *
@@ -47,7 +52,7 @@ process::create_kernel_process(page_table *pml4)
}
void
process::exit(int32_t code)
process::exit(unsigned code)
{
// TODO: make this thread-safe
m_state = state::exited;
@@ -58,7 +63,7 @@ process::exit(int32_t code)
thread->exit(code);
}
if (this == current_cpu().process)
if (this == bsp_cpu_data.p)
scheduler::get().schedule();
}
@@ -88,7 +93,7 @@ process::update()
thread *
process::create_thread(uint8_t priority, bool user)
{
if (priority == default_priority)
if (priority == default_pri)
priority = scheduler::default_priority;
thread *th = new thread(*this, priority);
@@ -97,13 +102,11 @@ process::create_thread(uint8_t priority, bool user)
if (user) {
uintptr_t stack_top = stacks_top - (m_threads.count() * stack_size);
vm_flags flags = vm_flags::zero|vm_flags::write;
vm_area *vma = new vm_area_open(stack_size, flags);
vm_area *vma = new vm_area_open(stack_size, m_space,
vm_flags::zero|vm_flags::write);
m_space.add(stack_top - stack_size, vma);
// Space for null frame - because the page gets zeroed on
// allocation, just pointing rsp here does the trick
th->tcb()->rsp3 = stack_top - 2 * sizeof(uint64_t);
th->tcb()->rsp3 = stack_top;
}
m_threads.append(th);
@@ -120,8 +123,6 @@ process::thread_exited(thread *th)
remove_handle(th->self_handle());
delete th;
// TODO: delete the thread's stack VMA
if (m_threads.count() == 0) {
exit(status);
return true;

14
src/kernel/objects/process.h
View File

@@ -16,10 +16,10 @@ public:
constexpr static uintptr_t stacks_top = 0x0000800000000000;
/// Size of userspace thread stacks
constexpr static size_t stack_size = 0x4000000; // 64MiB
constexpr static size_t stack_size = 0x4000;
/// Value that represents default priority
constexpr static uint8_t default_priority = 0xff;
constexpr static uint8_t default_pri = 0xff;
/// Constructor.
process();
@@ -34,7 +34,7 @@ public:
/// Terminate this process.
/// \arg code The return code to exit with.
void exit(int32_t code);
void exit(unsigned code);
/// Update internal bookkeeping about threads.
void update();
@@ -46,7 +46,7 @@ public:
/// \args priority The new thread's scheduling priority
/// \args user If true, create a userspace stack for this thread
/// \returns The newly created thread object
thread * create_thread(uint8_t priorty = default_priority, bool user = true);
thread * create_thread(uint8_t priorty = default_pri, bool user = true);
/// Start tracking an object with a handle.
/// \args obj The object this handle refers to
@@ -69,7 +69,7 @@ public:
bool thread_exited(thread *th);
/// Get the handle for this process to refer to itself
inline j6_handle_t self_handle() const { return 1; }
inline j6_handle_t self_handle() const { return 0; }
/// Get the process object that owns kernel threads and the
/// kernel address space
@@ -84,7 +84,7 @@ private:
// This constructor is called by create_kernel_process
process(page_table *kpml4);
int32_t m_return_code;
uint32_t m_return_code;
vm_space m_space;
@@ -94,4 +94,6 @@ private:
enum class state : uint8_t { running, exited };
state m_state;
static kutil::vector<process*> s_processes;
};

2
src/kernel/objects/system.h
View File

@@ -10,7 +10,7 @@ class system :
public:
static constexpr kobject::type type = kobject::type::event;
inline static system & get() { return s_instance; }
inline static system * get() { return &s_instance; }
private:
static system s_instance;

39
src/kernel/objects/thread.cpp
View File

@@ -9,7 +9,7 @@
extern "C" void kernel_to_user_trampoline();
static constexpr j6_signal_t thread_default_signals = 0;
extern vm_area_guarded &g_kernel_stacks;
extern vm_area_buffers g_kernel_stacks;
thread::thread(process &parent, uint8_t pri, uintptr_t rsp0) :
kobject(kobject::type::thread, thread_default_signals),
@@ -32,7 +32,7 @@ thread::thread(process &parent, uint8_t pri, uintptr_t rsp0) :
thread::~thread()
{
g_kernel_stacks.return_section(m_tcb.kernel_stack);
g_kernel_stacks.return_buffer(m_tcb.kernel_stack);
}
thread *
@@ -43,9 +43,13 @@ thread::from_tcb(TCB *tcb)
return reinterpret_cast<thread*>(kutil::offset_pointer(tcb, offset));
}
thread & thread::current() { return *current_cpu().thread; }
thread &
thread::current()
{
return *bsp_cpu_data.t;
}
inline void schedule_if_current(thread *t) { if (t == current_cpu().thread) scheduler::get().schedule(); }
inline void schedule_if_current(thread *t) { if (t == bsp_cpu_data.t) scheduler::get().schedule(); }
void
thread::wait_on_signals(kobject *obj, j6_signal_t signals)
@@ -132,7 +136,7 @@ thread::wake_on_result(kobject *obj, j6_status_t result)
}
void
thread::exit(int32_t code)
thread::exit(uint32_t code)
{
m_return_code = code;
set_state(state::exited);
@@ -145,10 +149,6 @@ thread::exit(int32_t code)
void
thread::add_thunk_kernel(uintptr_t rip)
{
// This adds just enough values to the top of the
// kernel stack to come out of task_switch correctly
// and start executing at rip (still in kernel mode)
m_tcb.rsp -= sizeof(uintptr_t) * 7;
uintptr_t *stack = reinterpret_cast<uintptr_t*>(m_tcb.rsp);
@@ -162,24 +162,15 @@ thread::add_thunk_kernel(uintptr_t rip)
}
void
thread::add_thunk_user(uintptr_t rip3, uintptr_t rip0, uint64_t flags)
thread::add_thunk_user(uintptr_t rip)
{
// This sets up the stack to:
// a) come out of task_switch and return to rip0 (default is the
// kernel/user trampoline) (via add_thunk_kernel) - if this is
// changed, it needs to end up at the trampoline with the stack
// as it was
// b) come out of the kernel/user trampoline and start executing
// in user mode at rip
m_tcb.rsp -= sizeof(uintptr_t) * 8;
uintptr_t *stack = reinterpret_cast<uintptr_t*>(m_tcb.rsp);
flags |= 0x200;
stack[7] = rip3; // return rip in rcx
stack[7] = rip; // return rip in rcx
stack[6] = m_tcb.rsp3; // rbp
stack[5] = 0xbbbbbbbb; // rbx
stack[4] = flags; // r11 sets RFLAGS
stack[4] = 0x00000200; // r11 sets RFLAGS
stack[3] = 0x12121212; // r12
stack[2] = 0x13131313; // r13
stack[1] = 0x14141414; // r14
@@ -187,7 +178,7 @@ thread::add_thunk_user(uintptr_t rip3, uintptr_t rip0, uint64_t flags)
static const uintptr_t trampoline =
reinterpret_cast<uintptr_t>(kernel_to_user_trampoline);
add_thunk_kernel(rip0 ? rip0 : trampoline);
add_thunk_kernel(trampoline);
}
void
@@ -200,7 +191,7 @@ thread::setup_kernel_stack()
constexpr unsigned null_frame_entries = 2;
constexpr size_t null_frame_size = null_frame_entries * sizeof(uint64_t);
uintptr_t stack_addr = g_kernel_stacks.get_section();
uintptr_t stack_addr = g_kernel_stacks.get_buffer();
uintptr_t stack_end = stack_addr + stack_bytes;
uint64_t *null_frame = reinterpret_cast<uint64_t*>(stack_end - null_frame_size);
@@ -221,5 +212,7 @@ thread::create_idle_thread(process &kernel, uint8_t pri, uintptr_t rsp0)
thread *idle = new thread(kernel, pri, rsp0);
idle->set_state(state::constant);
idle->set_state(state::ready);
log::info(logs::task, "Created idle thread as koid %llx", idle->koid());
return idle;
}

13
src/kernel/objects/thread.h
View File

@@ -131,18 +131,15 @@ public:
/// Terminate this thread.
/// \arg code The return code to exit with.
void exit(int32_t code);
void exit(unsigned code);
/// Add a stack header that returns to the given address in kernel space.
/// \arg rip The address to return to, must be kernel space
void add_thunk_kernel(uintptr_t rip);
/// Add a stack header that returns to the given address in user space
/// via a function in kernel space.
/// \arg rip3 The user space address to return to
/// \arg rip0 The kernel function to pass through, optional
/// \arg flags Extra RFLAGS values to set, optional
void add_thunk_user(uintptr_t rip3, uintptr_t rip0 = 0, uint64_t flags = 0);
/// Add a stack header that returns to the given address in user space.
/// \arg rip The address to return to, must be user space
void add_thunk_user(uintptr_t rip);
/// Get the handle representing this thread to its process
j6_handle_t self_handle() const { return m_self_handle; }
@@ -176,7 +173,7 @@ private:
wait_type m_wait_type;
// There should be 1 byte of padding here
int32_t m_return_code;
uint32_t m_return_code;
uint64_t m_wait_data;
j6_status_t m_wait_result;

150
src/kernel/objects/vm_area.cpp
View File

@@ -1,7 +1,5 @@
#include "frame_allocator.h"
#include "kernel_memory.h"
#include "objects/vm_area.h"
#include "page_tree.h"
#include "vm_space.h"
using memory::frame_size;
@@ -9,31 +7,30 @@ using memory::frame_size;
vm_area::vm_area(size_t size, vm_flags flags) :
m_size {size},
m_flags {flags},
m_spaces {m_vector_static, 0, static_size},
kobject {kobject::type::vma}
{
}
vm_area::~vm_area() {}
bool
vm_area::add_to(vm_space *space)
size_t
vm_area::resize(size_t size)
{
for (auto *s : m_spaces) {
if (s == space)
return true;
}
m_spaces.append(space);
return true;
if (mapper().can_resize(size))
m_size = size;
return m_size;
}
bool
vm_area::remove_from(vm_space *space)
void
vm_area::commit(uintptr_t phys, uintptr_t offset, size_t count)
{
m_spaces.remove_swap(space);
return
!m_spaces.count() &&
!(m_flags && vm_flags::mmio);
mapper().map(offset, count, phys);
}
void
vm_area::uncommit(uintptr_t offset, size_t count)
{
mapper().unmap(offset, count);
}
void
@@ -43,99 +40,48 @@ vm_area::on_no_handles()
delete this;
}
size_t
vm_area::resize(size_t size)
{
if (can_resize(size))
m_size = size;
return m_size;
}
bool
vm_area::can_resize(size_t size)
{
for (auto *space : m_spaces)
if (!space->can_resize(*this, size))
return false;
return true;
}
vm_area_fixed::vm_area_fixed(uintptr_t start, size_t size, vm_flags flags) :
m_start {start},
vm_area_shared::vm_area_shared(size_t size, vm_flags flags) :
m_mapper {*this},
vm_area {size, flags}
{
}
vm_area_fixed::~vm_area_fixed() {}
size_t vm_area_fixed::resize(size_t size)
{
// Not resizable
return m_size;
}
bool vm_area_fixed::get_page(uintptr_t offset, uintptr_t &phys)
{
if (offset > m_size)
return false;
phys = m_start + offset;
return true;
}
vm_area_untracked::vm_area_untracked(size_t size, vm_flags flags) :
vm_area {size, flags}
vm_area_shared::~vm_area_shared()
{
}
vm_area_untracked::~vm_area_untracked() {}
bool
vm_area_untracked::get_page(uintptr_t offset, uintptr_t &phys)
{
if (offset > m_size)
return false;
return frame_allocator::get().allocate(1, &phys);
}
bool
vm_area_untracked::add_to(vm_space *space)
{
if (!m_spaces.count())
return vm_area::add_to(space);
return m_spaces[0] == space;
}
vm_area_open::vm_area_open(size_t size, vm_flags flags) :
m_mapped {nullptr},
vm_area {size, flags}
vm_area_open::vm_area_open(size_t size, vm_space &space, vm_flags flags) :
m_mapper(*this, space),
vm_area(size, flags)
{
}
vm_area_open::~vm_area_open() {}
bool
vm_area_open::get_page(uintptr_t offset, uintptr_t &phys)
void
vm_area_open::commit(uintptr_t phys, uintptr_t offset, size_t count)
{
return page_tree::find_or_add(m_mapped, offset, phys);
m_mapper.map(offset, count, phys);
}
void
vm_area_open::uncommit(uintptr_t offset, size_t count)
{
m_mapper.unmap(offset, count);
}
vm_area_guarded::vm_area_guarded(uintptr_t start, size_t buf_pages, size_t size, vm_flags flags) :
m_start {start},
vm_area_buffers::vm_area_buffers(size_t size, vm_space &space, vm_flags flags, size_t buf_pages) :
m_mapper {*this, space},
m_pages {buf_pages},
m_next {memory::frame_size},
vm_area_untracked {size, flags}
vm_area {size, flags}
{
}
vm_area_guarded::~vm_area_guarded() {}
uintptr_t
vm_area_guarded::get_section()
vm_area_buffers::get_buffer()
{
if (m_cache.count() > 0) {
return m_cache.pop();
@@ -143,27 +89,33 @@ vm_area_guarded::get_section()
uintptr_t addr = m_next;
m_next += (m_pages + 1) * memory::frame_size;
return m_start + addr;
return m_mapper.space().lookup(*this, addr);
}
void
vm_area_guarded::return_section(uintptr_t addr)
vm_area_buffers::return_buffer(uintptr_t addr)
{
m_cache.append(addr);
}
bool
vm_area_guarded::get_page(uintptr_t offset, uintptr_t &phys)
vm_area_buffers::allowed(uintptr_t offset) const
{
if (offset > m_next)
return false;
if (offset >= m_next) return false;
// make sure this isn't in a guard page. (sections are
// m_pages big plus 1 leading guard page, so page 0 is
// invalid)
if ((offset >> 12) % (m_pages+1) == 0)
return false;
return vm_area_untracked::get_page(offset, phys);
// Buffers are m_pages big plus 1 leading guard page
return memory::page_align_down(offset) % (m_pages+1);
}
void
vm_area_buffers::commit(uintptr_t phys, uintptr_t offset, size_t count)
{
m_mapper.map(offset, count, phys);
}
void
vm_area_buffers::uncommit(uintptr_t offset, size_t count)
{
m_mapper.unmap(offset, count);
}

177
src/kernel/objects/vm_area.h
View File

@@ -11,15 +11,26 @@
#include "kernel_memory.h"
#include "objects/kobject.h"
#include "vm_mapper.h"
class page_tree;
class vm_space;
enum class vm_flags : uint32_t
{
#define VM_FLAG(name, v) name = v,
#include "j6/tables/vm_flags.inc"
#undef VM_FLAG
none = 0x00000000,
write = 0x00000001,
exec = 0x00000002,
zero = 0x00000010,
contiguous = 0x00000020,
large_pages = 0x00000100,
huge_pages = 0x00000200,
mmio = 0x00010000,
write_combine = 0x00020000,
user_mask = 0x0000ffff ///< flags allowed via syscall
};
@@ -44,134 +55,136 @@ public:
/// Get the flags set for this area
inline vm_flags flags() const { return m_flags; }
/// Track that this area was added to a vm_space
/// \arg space The space to add this area to
/// \returns False if this area cannot be added
virtual bool add_to(vm_space *space);
/// Track that this area was removed frm a vm_space
/// \arg space The space that is removing this area
/// \returns True if the removing space should free the pages
/// mapped for this area
virtual bool remove_from(vm_space *space);
/// Change the virtual size of the memory area. This may cause
/// deallocation if the new size is smaller than the current size.
/// Note that if resizing is unsuccessful, the previous size will
/// be returned.
/// \arg size The desired new virtual size
/// \returns The new virtual size
virtual size_t resize(size_t size);
size_t resize(size_t size);
/// Get the physical page for the given offset
/// \arg offset The offset into the VMA
/// \arg phys [out] Receives the physical page address, if any
/// \returns True if there should be a page at the given offset
virtual bool get_page(uintptr_t offset, uintptr_t &phys) = 0;
/// Get the mapper object that maps this area to address spaces
virtual vm_mapper & mapper() = 0;
virtual const vm_mapper & mapper() const = 0;
/// Check whether allocation at the given offset is allowed
virtual bool allowed(uintptr_t offset) const { return true; }
/// Commit contiguous physical pages to this area
/// \arg phys The physical address of the first page
/// \arg offset The offset from the start of this area these pages represent
/// \arg count The number of pages
virtual void commit(uintptr_t phys, uintptr_t offset, size_t count);
/// Uncommit physical pages from this area
/// \arg offset The offset from the start of this area these pages represent
/// \arg count The number of pages
virtual void uncommit(uintptr_t offset, size_t count);
protected:
virtual void on_no_handles() override;
bool can_resize(size_t size);
size_t m_size;
vm_flags m_flags;
kutil::vector<vm_space*> m_spaces;
// Initial static space for m_spaces - most areas will never grow
// beyond this size, so avoid allocations
static constexpr size_t static_size = 2;
vm_space *m_vector_static[static_size];
};
/// A shareable but non-allocatable memory area of contiguous physical
/// addresses (like mmio)
class vm_area_fixed :
/// The standard, sharable, user-controllable VMA type
class vm_area_shared :
public vm_area
{
public:
/// Constructor.
/// \arg start Starting physical address of this area
/// \arg size Size of the physical memory area
/// \arg size Initial virtual size of the memory area
/// \arg flags Flags for this memory area
vm_area_fixed(uintptr_t start, size_t size, vm_flags flags = vm_flags::none);
virtual ~vm_area_fixed();
vm_area_shared(size_t size, vm_flags flags = vm_flags::none);
virtual ~vm_area_shared();
virtual size_t resize(size_t size) override;
virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
virtual vm_mapper & mapper() override { return m_mapper; }
virtual const vm_mapper & mapper() const override { return m_mapper; }
private:
uintptr_t m_start;
vm_mapper_multi m_mapper;
};
/// Area that allows open allocation
/// Area that allows open allocation (eg, kernel heap)
class vm_area_open :
public vm_area
{
public:
/// Constructor.
/// \arg size Initial virtual size of the memory area
/// \arg space The address space this area belongs to
/// \arg flags Flags for this memory area
vm_area_open(size_t size, vm_flags flags);
virtual ~vm_area_open();
vm_area_open(size_t size, vm_space &space, vm_flags flags);
virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
virtual vm_mapper & mapper() override { return m_mapper; }
virtual const vm_mapper & mapper() const override { return m_mapper; }
virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
virtual void uncommit(uintptr_t offset, size_t count) override;
private:
page_tree *m_mapped;
vm_mapper_single m_mapper;
};
/// Area that does not track its allocations and thus cannot be shared
class vm_area_untracked :
/// Area split into standard-sized segments
class vm_area_buffers :
public vm_area
{
public:
/// Constructor.
/// \arg size Initial virtual size of the memory area
/// \arg space The address space this area belongs to
/// \arg flags Flags for this memory area
/// \arg buf_pages Pages in an individual buffer
vm_area_buffers(
size_t size,
vm_space &space,
vm_flags flags,
size_t buf_pages);
/// Get an available stack address
uintptr_t get_buffer();
/// Return a buffer address to the available pool
void return_buffer(uintptr_t addr);
virtual vm_mapper & mapper() override { return m_mapper; }
virtual const vm_mapper & mapper() const override { return m_mapper; }
virtual bool allowed(uintptr_t offset) const override;
virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
virtual void uncommit(uintptr_t offset, size_t count) override;
private:
vm_mapper_single m_mapper;
kutil::vector<uintptr_t> m_cache;
size_t m_pages;
uintptr_t m_next;
};
/// Area backed by an external source (like a loaded program)
class vm_area_backed :
public vm_area
{
public:
/// Constructor.
/// \arg size Initial virtual size of the memory area
/// \arg flags Flags for this memory area
vm_area_untracked(size_t size, vm_flags flags);
virtual ~vm_area_untracked();
vm_area_backed(size_t size, vm_flags flags);
virtual bool add_to(vm_space *space) override;
virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
};
virtual vm_mapper & mapper() override { return m_mapper; }
virtual const vm_mapper & mapper() const override { return m_mapper; }
/// Area split into standard-sized segments, separated by guard pages.
/// Based on vm_area_untracked, can not be shared.
class vm_area_guarded :
public vm_area_untracked
{
public:
/// Constructor.
/// \arg start Initial address where this area is mapped
/// \arg sec_pages Pages in an individual section
/// \arg size Initial virtual size of the memory area
/// \arg flags Flags for this memory area
vm_area_guarded(
uintptr_t start,
size_t sec_pages,
size_t size,
vm_flags flags);
virtual ~vm_area_guarded();
/// Get an available section in this area
uintptr_t get_section();
/// Return a section address to the available pool
void return_section(uintptr_t addr);
virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
virtual void uncommit(uintptr_t offset, size_t count) override;
private:
kutil::vector<uintptr_t> m_cache;
uintptr_t m_start;
size_t m_pages;
uintptr_t m_next;
vm_mapper_multi m_mapper;
};
IS_BITFIELD(vm_flags);

152
src/kernel/page_tree.cpp
View File

@@ -1,152 +0,0 @@
#include "kutil/assert.h"
#include "kutil/memory.h"
#include "frame_allocator.h"
#include "page_tree.h"
// Page tree levels map the following parts of a pagewise offset:
// (Note that a level 0's entries are physical page addrs, the rest
// map other page_tree nodes)
//
// Level 0: 0000000003f 64 pages / 256 KiB
// Level 1: 00000000fc0 4K pages / 16 MiB
// Level 2: 0000003f000 256K pages / 1 GiB
// Level 3: 00000fc0000 16M pages / 64 GiB
// Level 4: 0003f000000 1G pages / 4 TiB
// Level 5: 00fc0000000 64G pages / 256 TiB
// Level 6: 3f000000000 4T pages / 16 PiB -- Not supported until 5-level paging
static constexpr unsigned max_level = 5;
static constexpr unsigned bits_per_level = 6;
inline uint64_t to_word(uint64_t base, uint64_t level, uint64_t flags = 0) {
// Clear out the non-appropriate bits for this level
base &= (~0x3full << (level*bits_per_level));
return
(base & 0x3ffffffffff) |
((level & 0x7) << 42) |
((flags & 0x7ffff) << 45);
}
inline uint64_t to_base(uint64_t word) {
return word & 0x3ffffffffff;
}
inline uint64_t to_level(uint64_t word) {
return (word >> 42) & 0x3f;
}
inline uint64_t to_flags(uint64_t word) {
return (word >> 45);
}
inline bool contains(uint64_t page_off, uint64_t word, uint8_t &index) {
uint64_t base = to_base(word);
uint64_t bits = to_level(word) * bits_per_level;
index = (page_off >> bits) & 0x3f;
return (page_off & (~0x3full << bits)) == base;
}
inline uint64_t index_for(uint64_t page_off, uint8_t level) {
return (page_off >> (level*bits_per_level)) & 0x3f;
}
page_tree::page_tree(uint64_t base, uint8_t level) :
m_base {to_word(base, level)}
{
kutil::memset(m_entries, 0, sizeof(m_entries));
}
bool
page_tree::find(const page_tree *root, uint64_t offset, uintptr_t &page)
{
uint64_t page_off = offset >> 12; // change to pagewise offset
page_tree const *node = root;
while (node) {
uint8_t level = to_level(node->m_base);
uint8_t index = 0;
if (!contains(page_off, node->m_base, index))
return false;
if (!level) {
uintptr_t entry = node->m_entries[index].entry;
page = entry & ~1ull; // bit 0 marks 'present'
return (entry & 1);
}
node = node->m_entries[index].child;
}
return false;
}
bool
page_tree::find_or_add(page_tree * &root, uint64_t offset, uintptr_t &page)
{
uint64_t page_off = offset >> 12; // change to pagewise offset
page_tree *level0 = nullptr;
if (!root) {
// There's no root yet, just make a level0 and make it
// the root.
level0 = new page_tree(page_off, 0);
root = level0;
} else {
// Find or insert an existing level0
page_tree **parent = &root;
page_tree *node = root;
uint8_t parent_level = max_level + 1;
while (node) {
uint8_t level = to_level(node->m_base);
uint8_t index = 0;
if (!contains(page_off, node->m_base, index)) {
// We found a valid parent but the slot where this node should
// go contains another node. Insert an intermediate parent of
// this node and a new level0 into the parent.
uint64_t other = to_base(node->m_base);
uint8_t lcl = parent_level;
while (index_for(page_off, lcl) == index_for(other, lcl))
--lcl;
page_tree *inter = new page_tree(page_off, lcl);
inter->m_entries[index_for(other, lcl)].child = node;
*parent = inter;
level0 = new page_tree(page_off, 0);
inter->m_entries[index_for(page_off, lcl)].child = level0;
break;
}
if (!level) {
level0 = node;
break;
}
parent = &node->m_entries[index].child;
node = *parent;
}
kassert( node || parent, "Both node and parent were null in find_or_add");
if (!node) {
// We found a parent with an empty spot where this node should
// be. Insert a new level0 there.
level0 = new page_tree(page_off, 0);
*parent = level0;
}
}
kassert(level0, "Got through find_or_add without a level0");
uint8_t index = index_for(page_off, 0);
uint64_t &ent = level0->m_entries[index].entry;
if (!(ent & 1)) {
// No entry for this page exists, so make one
if (!frame_allocator::get().allocate(1, &ent))
return false;
ent |= 1;
}
page = ent & ~0xfffull;
return true;
}

39
src/kernel/page_tree.h
View File

@@ -1,39 +0,0 @@
#pragma once
/// \file page_tree.h
/// Definition of mapped page tracking structure and related definitions
#include <stdint.h>
/// A radix tree node that tracks mapped pages
class page_tree
{
public:
/// Get the physical address of the page at the given offset.
/// \arg root The root node of the tree
/// \arg offset Offset into the VMA, in bytes
/// \arg page [out] Receives the page physical address, if found
/// \returns True if a page was found
static bool find(const page_tree *root, uint64_t offset, uintptr_t &page);
/// Get the physical address of the page at the given offset. If one does
/// not exist yet, allocate a page, insert it, and return that.
/// \arg root [inout] The root node of the tree. This pointer may be updated.
/// \arg offset Offset into the VMA, in bytes
/// \arg page [out] Receives the page physical address, if found
/// \returns True if a page was found
static bool find_or_add(page_tree * &root, uint64_t offset, uintptr_t &page);
private:
page_tree(uint64_t base, uint8_t level);
/// Stores the page offset of the start of this node's pages in bits 0:41
/// and the depth of tree this node represents in bits 42:44 (0-7)
uint64_t m_base;
/// For a level 0 node, the entries area all physical page addresses.
/// Other nodes contain pointers to child tree nodes.
union {
uintptr_t entry;
page_tree *child;
} m_entries[64];
};

385
src/kernel/scheduler.cpp
View File

@@ -7,7 +7,6 @@
#include "console.h"
#include "cpu.h"
#include "debug.h"
#include "device_manager.h"
#include "gdt.h"
#include "interrupts.h"
#include "io.h"
@@ -17,7 +16,6 @@
#include "objects/channel.h"
#include "objects/process.h"
#include "objects/system.h"
#include "objects/thread.h"
#include "objects/vm_area.h"
#include "scheduler.h"
@@ -26,37 +24,47 @@
#include "kutil/assert.h"
extern "C" void task_switch(TCB *tcb);
scheduler *scheduler::s_instance = nullptr;
struct run_queue
{
tcb_node *current = nullptr;
tcb_list ready[scheduler::num_priorities];
tcb_list blocked;
extern kernel::args::framebuffer *fb;
uint64_t last_promotion = 0;
uint64_t last_steal = 0;
kutil::spinlock lock;
const uint64_t rflags_noint = 0x002;
const uint64_t rflags_int = 0x202;
extern "C" {
void preloaded_process_init();
uintptr_t load_process_image(uintptr_t phys, uintptr_t virt, size_t bytes, TCB *tcb);
};
scheduler::scheduler(unsigned cpus) :
m_next_pid {1},
m_clock {0}
{
kassert(!s_instance, "Created multiple schedulers!");
if (!s_instance)
s_instance = this;
extern uint64_t idle_stack_end;
m_run_queues.set_size(cpus);
}
scheduler::~scheduler()
scheduler::scheduler(lapic *apic) :
m_apic(apic),
m_next_pid(1),
m_clock(0),
m_last_promotion(0)
{
// Not truly necessary - if the scheduler is going away, the whole
// system is probably going down. But let's be clean.
if (s_instance == this)
s_instance = nullptr;
kassert(!s_instance, "Multiple schedulers created!");
s_instance = this;
process *kp = &process::kernel_process();
log::debug(logs::task, "Kernel process koid %llx", kp->koid());
thread *idle = thread::create_idle_thread(*kp, max_priority,
reinterpret_cast<uintptr_t>(&idle_stack_end));
log::debug(logs::task, "Idle thread koid %llx", idle->koid());
auto *tcb = idle->tcb();
m_runlists[max_priority].push_back(tcb);
m_current = tcb;
bsp_cpu_data.rsp0 = tcb->rsp0;
bsp_cpu_data.tcb = tcb;
bsp_cpu_data.p = kp;
bsp_cpu_data.t = idle;
}
template <typename T>
@@ -67,6 +75,139 @@ inline T * push(uintptr_t &rsp, size_t size = sizeof(T)) {
return p;
}
uintptr_t
load_process_image(uintptr_t phys, uintptr_t virt, size_t bytes, TCB *tcb)
{
using memory::page_align_down;
using memory::page_align_up;
// We're now in the process space for this process, allocate memory for the
// process code and load it
process &proc = process::current();
thread &th = thread::current();
vm_space &space = proc.space();
vm_area *vma = new vm_area_open(bytes, space, vm_flags::zero|vm_flags::write);
space.add(virt, vma);
vma->commit(phys, 0, memory::page_count(bytes));
// double zero stack sentinel
*push<uint64_t>(tcb->rsp3) = 0;
*push<uint64_t>(tcb->rsp3) = 0;
const char message[] = "Hello from the kernel!";
char *message_arg = push<char>(tcb->rsp3, sizeof(message));
kutil::memcpy(message_arg, message, sizeof(message));
j6_init_framebuffer *fb_desc = push<j6_init_framebuffer>(tcb->rsp3);
fb_desc->addr = fb ? reinterpret_cast<void*>(0x100000000) : nullptr;
fb_desc->size = fb ? fb->size : 0;
fb_desc->vertical = fb ? fb->vertical : 0;
fb_desc->horizontal = fb ? fb->horizontal : 0;
fb_desc->scanline = fb ? fb->scanline : 0;
fb_desc->flags = 0;
if (fb && fb->type == kernel::args::fb_type::bgr8)
fb_desc->flags |= 1;
j6_init_value *initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_system;
initv->value = static_cast<uint64_t>(proc.add_handle(system::get()));
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_process;
initv->value = static_cast<uint64_t>(proc.self_handle());
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_thread;
initv->value = static_cast<uint64_t>(th.self_handle());
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_handle_space;
//initv->value = static_cast<uint64_t>(proc.add_handle(&space));
initv = push<j6_init_value>(tcb->rsp3);
initv->type = j6_init_desc_framebuffer;
initv->value = reinterpret_cast<uint64_t>(fb_desc);
uint64_t *initc = push<uint64_t>(tcb->rsp3);
*initc = 5;
char **argv0 = push<char*>(tcb->rsp3);
*argv0 = message_arg;
uint64_t *argc = push<uint64_t>(tcb->rsp3);
*argc = 1;
// Crazypants framebuffer part
if (fb) {
vma = new vm_area_open(fb->size, space, vm_flags::write|vm_flags::mmio|vm_flags::write_combine);
space.add(0x100000000, vma);
vma->commit(fb->phys_addr, 0, memory::page_count(fb->size));
}
th.clear_state(thread::state::loading);
return tcb->rsp3;
}
thread *
scheduler::create_process(bool user)
{
process *p = new process;
thread *th = p->create_thread(default_priority, user);
auto *tcb = th->tcb();
tcb->time_left = quantum(default_priority);
log::debug(logs::task, "Creating thread %llx, priority %d, time slice %d",
th->koid(), tcb->priority, tcb->time_left);
th->set_state(thread::state::ready);
return th;
}
thread *
scheduler::load_process(uintptr_t phys, uintptr_t virt, size_t size, uintptr_t entry)
{
uint16_t kcs = (1 << 3) | 0; // Kernel CS is GDT entry 1, ring 0
uint16_t cs = (5 << 3) | 3; // User CS is GDT entry 5, ring 3
uint16_t kss = (2 << 3) | 0; // Kernel SS is GDT entry 2, ring 0
uint16_t ss = (4 << 3) | 3; // User SS is GDT entry 4, ring 3
thread* th = create_process(true);
auto *tcb = th->tcb();
// Create an initial kernel stack space
uintptr_t *stack = reinterpret_cast<uintptr_t *>(tcb->rsp0) - 9;
// Pass args to preloaded_process_init on the stack
stack[0] = reinterpret_cast<uintptr_t>(phys);
stack[1] = reinterpret_cast<uintptr_t>(virt);
stack[2] = reinterpret_cast<uintptr_t>(size);
stack[3] = reinterpret_cast<uintptr_t>(tcb);
tcb->rsp = reinterpret_cast<uintptr_t>(stack);
th->add_thunk_kernel(reinterpret_cast<uintptr_t>(preloaded_process_init));
// Arguments for iret - rip will be pushed on before these
stack[4] = reinterpret_cast<uintptr_t>(entry);
stack[5] = cs;
stack[6] = rflags_int | (3 << 12);
stack[7] = process::stacks_top;
stack[8] = ss;
tcb->rsp3 = process::stacks_top;
log::debug(logs::task, "Loading thread %llx pri %d", th->koid(), tcb->priority);
log::debug(logs::task, " RSP %016lx", tcb->rsp);
log::debug(logs::task, " RSP0 %016lx", tcb->rsp0);
log::debug(logs::task, " PML4 %016lx", tcb->pml4);
return th;
}
void
scheduler::create_kernel_task(void (*task)(), uint8_t priority, bool constant)
{
@@ -96,42 +237,17 @@ scheduler::quantum(int priority)
void
scheduler::start()
{
cpu_data &cpu = current_cpu();
run_queue &queue = m_run_queues[cpu.index];
kutil::scoped_lock lock {queue.lock};
process *kp = &process::kernel_process();
thread *idle = thread::create_idle_thread(*kp, max_priority, cpu.rsp0);
log::debug(logs::task, "CPU%02x idle thread koid %llx", cpu.index, idle->koid());
auto *tcb = idle->tcb();
cpu.process = kp;
cpu.thread = idle;
cpu.tcb = tcb;
queue.current = tcb;
log::info(logs::sched, "CPU%02x starting scheduler", cpu.index);
cpu.apic->enable_timer(isr::isrTimer, false);
cpu.apic->reset_timer(10);
log::info(logs::task, "Starting scheduler.");
wrmsr(msr::ia32_gs_base, reinterpret_cast<uintptr_t>(&bsp_cpu_data));
m_apic->enable_timer(isr::isrTimer, false);
m_apic->reset_timer(10);
}
void
scheduler::add_thread(TCB *t)
{
cpu_data &cpu = current_cpu();
run_queue &queue = m_run_queues[cpu.index];
kutil::scoped_lock lock {queue.lock};
queue.blocked.push_back(static_cast<tcb_node*>(t));
t->time_left = quantum(t->priority);
}
void scheduler::prune(run_queue &queue, uint64_t now)
void scheduler::prune(uint64_t now)
{
// Find processes that are ready or have exited and
// move them to the appropriate lists.
auto *tcb = queue.blocked.front();
auto *tcb = m_blocked.front();
while (tcb) {
thread *th = thread::from_tcb(tcb);
uint8_t priority = tcb->priority;
@@ -139,7 +255,7 @@ void scheduler::prune(run_queue &queue, uint64_t now)
bool ready = th->has_state(thread::state::ready);
bool exited = th->has_state(thread::state::exited);
bool constant = th->has_state(thread::state::constant);
bool current = tcb == queue.current;
bool current = tcb == m_current;
ready |= th->wake_on_time(now);
@@ -154,7 +270,7 @@ void scheduler::prune(run_queue &queue, uint64_t now)
// page tables
if (current) continue;
queue.blocked.remove(remove);
m_blocked.remove(remove);
process &p = th->parent();
// thread_exited deletes the thread, and returns true if the process
@@ -162,19 +278,19 @@ void scheduler::prune(run_queue &queue, uint64_t now)
if(!current && p.thread_exited(th))
delete &p;
} else {
queue.blocked.remove(remove);
log::debug(logs::sched, "Prune: readying unblocked thread %llx", th->koid());
queue.ready[remove->priority].push_back(remove);
m_blocked.remove(remove);
log::debug(logs::task, "Prune: readying unblocked thread %llx", th->koid());
m_runlists[remove->priority].push_back(remove);
}
}
}
void
scheduler::check_promotions(run_queue &queue, uint64_t now)
scheduler::check_promotions(uint64_t now)
{
for (auto &pri_list : queue.ready) {
for (auto &pri_list : m_runlists) {
for (auto *tcb : pri_list) {
const thread *th = thread::from_tcb(queue.current);
const thread *th = thread::from_tcb(m_current);
const bool constant = th->has_state(thread::state::constant);
if (constant)
continue;
@@ -189,145 +305,80 @@ scheduler::check_promotions(run_queue &queue, uint64_t now)
if (stale) {
// If the thread is stale, promote it
queue.ready[priority].remove(tcb);
m_runlists[priority].remove(tcb);
tcb->priority -= 1;
tcb->time_left = quantum(tcb->priority);
queue.ready[tcb->priority].push_back(tcb);
log::info(logs::sched, "Scheduler promoting thread %llx, priority %d",
m_runlists[tcb->priority].push_back(tcb);
log::debug(logs::task, "Scheduler promoting thread %llx, priority %d",
th->koid(), tcb->priority);
}
}
}
queue.last_promotion = now;
}
static size_t
balance_lists(tcb_list &to, tcb_list &from)
{
size_t to_len = to.length();
size_t from_len = from.length();
// Only steal from the rich, don't be Dennis Moore
if (from_len <= to_len)
return 0;
size_t steal = (from_len - to_len) / 2;
for (size_t i = 0; i < steal; ++i)
to.push_front(from.pop_front());
return steal;
}
void
scheduler::steal_work(cpu_data &cpu)
{
// First grab a scheduler-wide lock to avoid deadlock
kutil::scoped_lock steal_lock {m_steal_lock};
// Lock this cpu's queue for the whole time while we modify it
run_queue &my_queue = m_run_queues[cpu.index];
kutil::scoped_lock my_queue_lock {my_queue.lock};
const unsigned count = m_run_queues.count();
for (unsigned i = 0; i < count; ++i) {
if (i == cpu.index) continue;
run_queue &other_queue = m_run_queues[i];
kutil::scoped_lock other_queue_lock {other_queue.lock};
size_t stolen = 0;
// Don't steal from max_priority, that's the idle thread
for (unsigned pri = 0; pri < max_priority; ++pri)
stolen += balance_lists(my_queue.ready[pri], other_queue.ready[pri]);
stolen += balance_lists(my_queue.blocked, other_queue.blocked);
if (stolen)
log::debug(logs::sched, "CPU%02x stole %2d tasks from CPU%02x",
cpu.index, stolen, i);
}
m_last_promotion = now;
}
void
scheduler::schedule()
{
cpu_data &cpu = current_cpu();
run_queue &queue = m_run_queues[cpu.index];
lapic &apic = *cpu.apic;
uint32_t remaining = apic.stop_timer();
if (m_clock - queue.last_steal > steal_frequency) {
steal_work(cpu);
queue.last_steal = m_clock;
}
// We need to explicitly lock/unlock here instead of
// using a scoped lock, because the scope doesn't "end"
// for the current thread until it gets scheduled again
kutil::spinlock::waiter waiter;
queue.lock.acquire(&waiter);
queue.current->time_left = remaining;
thread *th = thread::from_tcb(queue.current);
uint8_t priority = queue.current->priority;
uint8_t priority = m_current->priority;
uint32_t remaining = m_apic->stop_timer();
m_current->time_left = remaining;
thread *th = thread::from_tcb(m_current);
const bool constant = th->has_state(thread::state::constant);
if (remaining == 0) {
if (priority < max_priority && !constant) {
// Process used its whole timeslice, demote it
++queue.current->priority;
log::debug(logs::sched, "Scheduler demoting thread %llx, priority %d",
th->koid(), queue.current->priority);
++m_current->priority;
log::debug(logs::task, "Scheduler demoting thread %llx, priority %d",
th->koid(), m_current->priority);
}
queue.current->time_left = quantum(queue.current->priority);
m_current->time_left = quantum(m_current->priority);
} else if (remaining > 0) {
// Process gave up CPU, give it a small bonus to its
// remaining timeslice.
uint32_t bonus = quantum(priority) >> 4;
queue.current->time_left += bonus;
m_current->time_left += bonus;
}
m_runlists[priority].remove(m_current);
if (th->has_state(thread::state::ready)) {
queue.ready[queue.current->priority].push_back(queue.current);
m_runlists[m_current->priority].push_back(m_current);
} else {
queue.blocked.push_back(queue.current);
m_blocked.push_back(m_current);
}
clock::get().update();
prune(queue, ++m_clock);
if (m_clock - queue.last_promotion > promote_frequency)
check_promotions(queue, m_clock);
prune(++m_clock);
if (m_clock - m_last_promotion > promote_frequency)
check_promotions(m_clock);
priority = 0;
while (queue.ready[priority].empty()) {
while (m_runlists[priority].empty()) {
++priority;
kassert(priority < num_priorities, "All runlists are empty");
}
queue.current->last_ran = m_clock;
m_current->last_ran = m_clock;
auto *next = queue.ready[priority].pop_front();
auto *next = m_runlists[priority].pop_front();
next->last_ran = m_clock;
apic.reset_timer(next->time_left);
m_apic->reset_timer(next->time_left);
if (next == queue.current) {
queue.lock.release(&waiter);
return;
if (next != m_current) {
thread *next_thread = thread::from_tcb(next);
bsp_cpu_data.t = next_thread;
bsp_cpu_data.p = &next_thread->parent();
m_current = next;
log::debug(logs::task, "Scheduler switching threads %llx->%llx",
th->koid(), next_thread->koid());
log::debug(logs::task, " priority %d time left %d @ %lld.",
m_current->priority, m_current->time_left, m_clock);
log::debug(logs::task, " PML4 %llx", m_current->pml4);
task_switch(m_current);
}
thread *next_thread = thread::from_tcb(next);
cpu.thread = next_thread;
cpu.process = &next_thread->parent();
queue.current = next;
log::debug(logs::sched, "CPU%02x switching threads %llx->%llx",
cpu.index, th->koid(), next_thread->koid());
log::debug(logs::sched, " priority %d time left %d @ %lld.",
next->priority, next->time_left, m_clock);
log::debug(logs::sched, " PML4 %llx", next->pml4);
queue.lock.release(&waiter);
task_switch(queue.current);
}

62
src/kernel/scheduler.h
View File

@@ -3,19 +3,15 @@
/// The task scheduler and related definitions
#include <stdint.h>
#include "kutil/spinlock.h"
#include "kutil/vector.h"
#include "objects/thread.h"
namespace kernel {
namespace args {
struct program;
}}
struct cpu_data;
class lapic;
class process;
struct page_table;
struct run_queue;
struct cpu_state;
extern "C" void isr_handler(cpu_state*);
extern "C" void task_switch(TCB *next);
/// The task scheduler
@@ -41,14 +37,16 @@ public:
static const uint16_t process_quanta = 10;
/// Constructor.
/// \arg cpus The number of CPUs to schedule for
scheduler(unsigned cpus);
~scheduler();
/// \arg apic Pointer to the local APIC object
scheduler(lapic *apic);
/// Create a new process from a program image in memory.
/// \arg program The descriptor of the pogram in memory
/// \returns The main thread of the loaded process
thread * load_process(kernel::args::program &program);
/// \arg phys Physical address of the loaded program image
/// \arg virt Virtual address of the loaded program image
/// \arg size Size of the program image, in bytes
/// \arg entry Virtual address of the program entrypoint
/// \returns The main thread of the loaded process
thread * load_process(uintptr_t phys, uintptr_t virt, size_t size, uintptr_t entry);
/// Create a new kernel task
/// \arg proc Function to run as a kernel task
@@ -60,7 +58,7 @@ public:
bool constant = false);
/// Get the quantum for a given priority.
static uint32_t quantum(int priority);
uint32_t quantum(int priority);
/// Start the scheduler working. This may involve starting
/// timer interrupts or other preemption methods.
@@ -69,35 +67,45 @@ public:
/// Run the scheduler, possibly switching to a new task
void schedule();
/// Start scheduling a new thread.
/// \arg t The new thread's TCB
void add_thread(TCB *t);
/// Get the current TCB.
/// \returns A pointer to the current thread's TCB
inline TCB * current() { return m_current; }
/// Get a reference to the scheduler
inline void add_thread(TCB *t) { m_blocked.push_back(static_cast<tcb_node*>(t)); }
/// Get a reference to the system scheduler
/// \returns A reference to the global system scheduler
static scheduler & get() { return *s_instance; }
private:
friend uintptr_t syscall_dispatch(uintptr_t, cpu_state &);
friend class process;
static constexpr uint64_t promote_frequency = 10;
static constexpr uint64_t steal_frequency = 10;
void prune(run_queue &queue, uint64_t now);
void check_promotions(run_queue &queue, uint64_t now);
void steal_work(cpu_data &cpu);
/// Create a new process object. This process will have its pid
/// set but nothing else.
/// \arg user True if this thread will enter userspace
/// \returns The new process' main thread
thread * create_process(bool user);
void prune(uint64_t now);
void check_promotions(uint64_t now);
lapic *m_apic;
uint32_t m_next_pid;
uint32_t m_tick_count;
process *m_kernel_process;
kutil::vector<run_queue> m_run_queues;
tcb_node *m_current;
tcb_list m_runlists[num_priorities];
tcb_list m_blocked;
// TODO: lol a real clock
uint64_t m_clock = 0;
uint64_t m_last_promotion;
kutil::spinlock m_steal_lock;
static scheduler *s_instance;
};

120
src/kernel/syscall.cpp
View File

@@ -1,19 +1,24 @@
#include <stddef.h>
#include "kutil/memory.h"
#include "console.h"
#include "cpu.h"
#include "debug.h"
#include "log.h"
#include "msr.h"
#include "scheduler.h"
#include "syscall.h"
extern "C" {
void syscall_invalid(uint64_t call);
void syscall_handler_prelude();
}
uintptr_t syscall_registry[256] __attribute__((section(".syscall_registry")));
const char * syscall_names[256] __attribute__((section(".syscall_registry")));
static constexpr size_t num_syscalls = sizeof(syscall_registry) / sizeof(syscall_registry[0]);
namespace syscalls {
} // namespace syscalls
uintptr_t syscall_registry[static_cast<unsigned>(syscall::MAX)];
const char * syscall_names[static_cast<unsigned>(syscall::MAX)];
void
syscall_invalid(uint64_t call)
@@ -22,10 +27,13 @@ syscall_invalid(uint64_t call)
cons->set_color(9);
cons->printf("\nReceived unknown syscall: %02x\n", call);
const unsigned num_calls =
static_cast<unsigned>(syscall::MAX);
cons->printf(" Known syscalls:\n");
cons->printf(" invalid %016lx\n", syscall_invalid);
for (unsigned i = 0; i < num_syscalls; ++i) {
for (unsigned i = 0; i < num_calls; ++i) {
const char *name = syscall_names[i];
uintptr_t handler = syscall_registry[i];
if (name)
@@ -36,17 +44,113 @@ syscall_invalid(uint64_t call)
_halt();
}
/*
void
syscall_initialize()
syscall_dispatch(cpu_state *regs)
{
console *cons = console::get();
syscall call = static_cast<syscall>(regs->rax);
auto &s = scheduler::get();
auto *p = s.current();
switch (call) {
case syscall::noop:
break;
case syscall::debug:
cons->set_color(11);
cons->printf("\nProcess %d: Received DEBUG syscall\n", p->pid);
cons->set_color();
print_regs(*regs);
cons->printf("\n Syscall enters: %8d\n", __counter_syscall_enter);
cons->printf(" Syscall sysret: %8d\n", __counter_syscall_sysret);
break;
case syscall::send:
{
pid_t target = regs->rdi;
uintptr_t data = regs->rsi;
cons->set_color(11);
cons->printf("\nProcess %d: Received SEND syscall, target %d, data %016lx\n", p->pid, target, data);
cons->set_color();
if (p->wait_on_send(target))
s.schedule();
}
break;
case syscall::receive:
{
pid_t source = regs->rdi;
uintptr_t data = regs->rsi;
cons->set_color(11);
cons->printf("\nProcess %d: Received RECEIVE syscall, source %d, dat %016lx\n", p->pid, source, data);
cons->set_color();
if (p->wait_on_receive(source))
s.schedule();
}
break;
case syscall::fork:
{
cons->set_color(11);
cons->printf("\nProcess %d: Received FORK syscall\n", p->pid);
cons->set_color();
pid_t pid = p->fork(regs);
cons->printf("\n fork returning %d\n", pid);
regs->rax = pid;
}
break;
default:
cons->set_color(9);
cons->printf("\nReceived unknown syscall: %02x\n", call);
cons->set_color();
_halt();
break;
}
}
*/
void
syscall_enable()
{
// IA32_EFER - set bit 0, syscall enable
uint64_t efer = rdmsr(msr::ia32_efer);
wrmsr(msr::ia32_efer, efer|1);
// IA32_STAR - high 32 bits contain k+u CS
// Kernel CS: GDT[1] ring 0 bits[47:32]
// User CS: GDT[3] ring 3 bits[63:48]
uint64_t star =
(((1ull << 3) | 0) << 32) |
(((3ull << 3) | 3) << 48);
wrmsr(msr::ia32_star, star);
// IA32_LSTAR - RIP for syscall
wrmsr(msr::ia32_lstar,
reinterpret_cast<uintptr_t>(&syscall_handler_prelude));
// IA32_FMASK - FLAGS mask inside syscall
wrmsr(msr::ia32_fmask, 0x200);
static constexpr unsigned num_calls =
static_cast<unsigned>(syscall::MAX);
kutil::memset(&syscall_registry, 0, sizeof(syscall_registry));
kutil::memset(&syscall_names, 0, sizeof(syscall_names));
#define SYSCALL(id, name, result, ...) \
syscall_registry[id] = reinterpret_cast<uintptr_t>(syscalls::name); \
syscall_names[id] = #name; \
static_assert( id <= num_calls, "Syscall " #name " has id > syscall::MAX" ); \
log::debug(logs::syscall, "Enabling syscall 0x%02x as " #name , id);
#include "j6/tables/syscalls.inc"
#include "syscalls.inc"
#undef SYSCALL
}

13
src/kernel/syscall.h
View File

@@ -7,17 +7,20 @@ struct cpu_state;
enum class syscall : uint64_t
{
#define SYSCALL(id, name, ...) name = id,
#include "j6/tables/syscalls.inc"
#define SYSCALL(id, name, result, ...) name = id,
#include "syscalls.inc"
#undef SYSCALL
// Maximum syscall id. If you change this, also change
// MAX_SYSCALLS in syscall.s
MAX = 0x40
};
void syscall_initialize();
extern "C" void syscall_enable();
void syscall_enable();
namespace syscalls
{
#define SYSCALL(id, name, ...) j6_status_t name (__VA_ARGS__);
#include "j6/tables/syscalls.inc"
#include "syscalls.inc"
#undef SYSCALL
}

70
src/kernel/syscall.s
View File

@@ -1,32 +1,19 @@
%include "push_all.inc"
%include "tasking.inc"
; SYSCALL/SYSRET control MSRs
MSR_STAR equ 0xc0000081
MSR_LSTAR equ 0xc0000082
MSR_FMASK equ 0xc0000084
; IA32_STAR - high 32 bits contain k+u CS
; Kernel CS: GDT[1] ring 0 bits[47:32]
; User CS: GDT[3] ring 3 bits[63:48]
STAR_HIGH equ \
(((1 << 3) | 0)) | \
(((3 << 3) | 3) << 16)
; IA32_FMASK - Mask off interrupts in syscalls
FMASK_VAL equ 0x200
; Make sure to keep MAX_SYSCALLS in sync with
; syscall::MAX in syscall.h
MAX_SYSCALLS equ 0x40
extern __counter_syscall_enter
extern __counter_syscall_sysret
extern syscall_registry
extern syscall_invalid
global syscall_handler_prelude:function (syscall_handler_prelude.end - syscall_handler_prelude)
global syscall_handler_prelude
global syscall_handler_prelude.return
syscall_handler_prelude:
push rbp ; Never executed, fake function prelude
mov rbp, rsp ; to calm down gdb
.real:
swapgs
mov [gs:CPU_DATA.rsp3], rsp
mov rsp, [gs:CPU_DATA.rsp0]
@@ -51,7 +38,14 @@ syscall_handler_prelude:
inc qword [rel __counter_syscall_enter]
and rax, 0xff ; Only 256 possible syscall values
cmp rax, MAX_SYSCALLS
jle .ok_syscall
.bad_syscall:
mov rdi, rax
call syscall_invalid
.ok_syscall:
lea r11, [rel syscall_registry]
mov r11, [r11 + rax * 8]
cmp r11, 0
@@ -60,15 +54,8 @@ syscall_handler_prelude:
call r11
inc qword [rel __counter_syscall_sysret]
jmp kernel_to_user_trampoline
.bad_syscall:
mov rdi, rax
call syscall_invalid
.end:
global kernel_to_user_trampoline:function (kernel_to_user_trampoline.end - kernel_to_user_trampoline)
kernel_to_user_trampoline:
.return:
pop r15
pop r14
pop r13
@@ -84,28 +71,3 @@ kernel_to_user_trampoline:
swapgs
o64 sysret
.end:
global syscall_enable:function (syscall_enable.end - syscall_enable)
syscall_enable:
push rbp
mov rbp, rsp
mov rcx, MSR_STAR
mov rax, 0
mov rdx, STAR_HIGH
wrmsr
mov rcx, MSR_LSTAR
mov rax, syscall_handler_prelude.real
mov rdx, rax
shr rdx, 32
wrmsr
mov rcx, MSR_FMASK
mov rax, FMASK_VAL
wrmsr
pop rbp
ret
.end:

4
src/kernel/syscalls/endpoint.cpp
View File

@@ -36,7 +36,7 @@ endpoint_receive(j6_handle_t handle, j6_tag_t *tag, size_t *len, void *data)
if (!e) return j6_err_invalid_arg;
j6_tag_t out_tag = j6_tag_invalid;
size_t out_len = *len;
size_t out_len = 0;
j6_status_t s = e->receive(&out_tag, &out_len, data);
*tag = out_tag;
*len = out_len;
@@ -57,7 +57,7 @@ endpoint_sendrecv(j6_handle_t handle, j6_tag_t *tag, size_t *len, void *data)
return status;
j6_tag_t out_tag = j6_tag_invalid;
size_t out_len = *len;
size_t out_len = 0;
j6_status_t s = e->receive(&out_tag, &out_len, data);
*tag = out_tag;
*len = out_len;

44
src/kernel/syscalls/process.cpp
View File

@@ -3,56 +3,18 @@
#include "log.h"
#include "objects/process.h"
#include "syscalls/helpers.h"
namespace syscalls {
j6_status_t
process_create(j6_handle_t *handle)
{
process *child = construct_handle<process>(handle);
log::debug(logs::task, "Process %llx created", child->koid());
return j6_status_ok;
}
j6_status_t
process_start(j6_handle_t handle, uintptr_t entrypoint, j6_handle_t *handles, size_t handle_count)
process_exit(int64_t status)
{
process &p = process::current();
process *c = get_handle<process>(handle);
if (handle_count && !handles)
return j6_err_invalid_arg;
for (size_t i = 0; i < handle_count; ++i) {
kobject *o = p.lookup_handle(handles[i]);
if (o) c->add_handle(o);
}
return j6_err_nyi;
}
j6_status_t
process_kill(j6_handle_t handle)
{
process &p = process::current();
process *c = get_handle<process>(handle);
if (!c) return j6_err_invalid_arg;
log::debug(logs::task, "Process %llx killed by process %llx", c->koid(), p.koid());
c->exit(-1u);
return j6_status_ok;
}
j6_status_t
process_exit(int32_t status)
{
process &p = process::current();
log::debug(logs::task, "Process %llx exiting with code %d", p.koid(), status);
log::debug(logs::syscall, "Process %llx exiting with code %d", p.koid(), status);
p.exit(status);
log::error(logs::task, "returned to exit syscall");
log::error(logs::syscall, "returned to exit syscall");
return j6_err_unexpected;
}

25
src/kernel/syscalls/system.cpp
View File

@@ -5,8 +5,6 @@
#include "log.h"
#include "objects/endpoint.h"
#include "objects/thread.h"
#include "objects/system.h"
#include "objects/vm_area.h"
#include "syscalls/helpers.h"
extern log::logger &g_logger;
@@ -33,17 +31,10 @@ system_noop()
}
j6_status_t
system_get_log(j6_handle_t sys, void *buffer, size_t *size)
system_get_log(j6_handle_t sys, char *buffer, size_t *size)
{
if (!size || (*size && !buffer))
return j6_err_invalid_arg;
size_t orig_size = *size;
*size = g_logger.get_entry(buffer, *size);
if (!g_logger.has_log())
system::get().deassert_signal(j6_signal_system_has_log);
return (*size > orig_size) ? j6_err_insufficient : j6_status_ok;
return j6_status_ok;
}
j6_status_t
@@ -59,16 +50,4 @@ system_bind_irq(j6_handle_t sys, j6_handle_t endp, unsigned irq)
return j6_err_invalid_arg;
}
j6_status_t
system_map_mmio(j6_handle_t sys, j6_handle_t *vma_handle, uintptr_t phys_addr, size_t size, uint32_t flags)
{
// TODO: check capabilities on sys handle
if (!vma_handle) return j6_err_invalid_arg;
vm_flags vmf = vm_flags::mmio | (static_cast<vm_flags>(flags) & vm_flags::user_mask);
construct_handle<vm_area_fixed>(vma_handle, phys_addr, size, vmf);
return j6_status_ok;
}
} // namespace syscalls

12
src/kernel/syscalls/thread.cpp
View File

@@ -15,24 +15,24 @@ thread_create(void *rip, j6_handle_t *handle)
thread *child = p.create_thread();
child->add_thunk_user(reinterpret_cast<uintptr_t>(rip));
*handle = child->self_handle();
*handle = p.self_handle();
child->clear_state(thread::state::loading);
child->set_state(thread::state::ready);
log::debug(logs::task, "Thread %llx spawned new thread %llx, handle %d",
log::debug(logs::syscall, "Thread %llx spawned new thread %llx, handle %d",
parent.koid(), child->koid(), *handle);
return j6_status_ok;
}
j6_status_t
thread_exit(int32_t status)
thread_exit(int64_t status)
{
thread &th = thread::current();
log::debug(logs::task, "Thread %llx exiting with code %d", th.koid(), status);
log::debug(logs::syscall, "Thread %llx exiting with code %d", th.koid(), status);
th.exit(status);
log::error(logs::task, "returned to exit syscall");
log::error(logs::syscall, "returned to exit syscall");
return j6_err_unexpected;
}
@@ -48,7 +48,7 @@ j6_status_t
thread_sleep(uint64_t til)
{
thread &th = thread::current();
log::debug(logs::task, "Thread %llx sleeping until %llu", th.koid(), til);
log::debug(logs::syscall, "Thread %llx sleeping until %llu", th.koid(), til);
th.wait_on_time(til);
return j6_status_ok;

18
src/kernel/syscalls/vm_area.cpp
View File

@@ -14,7 +14,7 @@ j6_status_t
vma_create(j6_handle_t *handle, size_t size, uint32_t flags)
{
vm_flags f = vm_flags::user_mask & flags;
construct_handle<vm_area_open>(handle, size, f);
construct_handle<vm_area_shared>(handle, size, f);
return j6_status_ok;
}
@@ -22,34 +22,28 @@ j6_status_t
vma_create_map(j6_handle_t *handle, size_t size, uintptr_t base, uint32_t flags)
{
vm_flags f = vm_flags::user_mask & flags;
vm_area *a = construct_handle<vm_area_open>(handle, size, f);
vm_area *a = construct_handle<vm_area_shared>(handle, size, f);
process::current().space().add(base, a);
return j6_status_ok;
}
j6_status_t
vma_map(j6_handle_t handle, j6_handle_t proc, uintptr_t base)
vma_map(j6_handle_t handle, uintptr_t base)
{
vm_area *a = get_handle<vm_area>(handle);
if (!a) return j6_err_invalid_arg;
process *p = get_handle<process>(proc);
if (!p) return j6_err_invalid_arg;
p->space().add(base, a);
process::current().space().add(base, a);
return j6_status_ok;
}
j6_status_t
vma_unmap(j6_handle_t handle, j6_handle_t proc)
vma_unmap(j6_handle_t handle)
{
vm_area *a = get_handle<vm_area>(handle);
if (!a) return j6_err_invalid_arg;
process *p = get_handle<process>(proc);
if (!p) return j6_err_invalid_arg;
p->space().remove(a);
process::current().space().remove(a);
return j6_status_ok;
}

Some files were not shown because too many files have changed in this diff Show More