justin/jsix_import
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: justin:v0.6.0
Branches Tags
justin:main justin:feature/multi-arch justin:feature/dynamic_linking justin:framebuffer-issue justin:fb-driver justin:vdso
justin:v0.8.0 justin:v0.7.0 justin:v0.6.0 justin:v0.5.0 justin:v0.4.0 justin:v0.3.0 justin:v0.2.2 justin:v0.2.1 justin:v0.2.0 justin:v0.1.3 justin:v0.1.2 justin:v0.1.1 justin:v0.1.0
..
compare: justin:framebuffer-issue
Branches Tags
justin:feature/multi-arch justin:main justin:feature/dynamic_linking justin:framebuffer-issue justin:fb-driver justin:vdso
justin:v0.8.0 justin:v0.7.0 justin:v0.6.0 justin:v0.5.0 justin:v0.4.0 justin:v0.3.0 justin:v0.2.2 justin:v0.2.1 justin:v0.2.0 justin:v0.1.3 justin:v0.1.2 justin:v0.1.1 justin:v0.1.0

1 Commits
v0.6.0 ... framebuffe

Author SHA1 Message Date
Justin C. Miller
2652cc449c WIP framebuffer issue 2021-02-04 21:37:53 -08:00
688 changed files with 36435 additions and 29423 deletions
Expand all files Collapse all files

21
.clang-format Normal file
View File

@@ -0,0 +1,21 @@
---
Language: Cpp
BasedOnStyle: LLVM
ColumnLimit: 100
IndentWidth: 4
TabWidth: 4
UseTab: Always
AccessModifierOffset: -4
AlignEscapedNewlinesLeft: true
AllowShortIfStatementsOnASingleLine: true
AllowShortLoopsOnASingleLine: true
AlwaysBreakAfterReturnType: AllDefinitions
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
BreakBeforeBraces: Linux

5
.gitignore vendored
View File

@@ -1,15 +1,10 @@
.cache
.lock* .lock*
/build* /build*
*.bak *.bak
tags tags
jsix.log jsix.log
*.out
*.o *.o
*.a *.a
sysroot sysroot
.gdb_history .gdb_history
.peru .peru
__pycache__
/venv
compile_commands.json

38
NOTES.md Normal file
View File

@@ -0,0 +1,38 @@
# Design / WIP notes
## TODO
- Paging manager
- Copy-on-write pages
- Better page-allocation model?
- Allow for more than one IOAPIC in ACPI module
- The objects get created, but GSI lookup only uses the one at index 0
- mark kernel memory pages global
- Serial out based on circular/bip biffer and interrupts, not spinning on
`write_ready()`
- Split out more code into kutil for testing
- AHCI / MSI interrupts on Vbox break?
- FXSAVE to save XMM registers.
- optimization using #NM (0x7) to detect SSE usage
- Clean up of process memory maps
- Better stack tracer
- Bootloader rewrite
- C++ and sharing library code for ELF, initrd, etc
- Parse initrd and pre-load certain ELF images, eg the process loader process?
- Do initial memory bootstrap?
- Calling global ctors
- Device Tree
- Actual serial driver
- Disk driver
- File system
- Multiprocessing
- Fast syscalls using syscall/sysret
### Build
- Clean up build generator and its templates
- More robust objects to represent modules & targets to pass to templates
- Read project setup from a simple JSON/TOML/etc
- Better compartmentalizing when doing template inheritance
- Move to LLD as sysroot linker

173
README.md
View File

@@ -1,16 +1,15 @@
![jsix](assets/jsix.svg) ![jsix](assets/jsix.svg)
# The jsix operating system # jsix: A hobby operating system
**jsix** is a custom multi-core x64 operating system that I am building from **jsix** is the hobby operating system that I am currently building. It's far
scratch. It's far from finished, or even being usable (see the *Status and from finished, or even being usable. Instead, it's a sandbox for me to play
Roadmap* section, below) but all currently-planned major kernel features are with kernel-level code and explore architectures.
now implemented to at least a passable level.
The design goals of the project are: The design goals of the project are:
* Modernity - I'm not interested in designing for legacy systems, or running on * Modernity - I'm not interested in designing for legacy systems, or running on
all hardware out there. My target is only 64 bit architectures, and modern all hardware out there. My target is only 64 bit architecutres, and modern
commodity hardware. Currently that means x64 systems with Nehalem or newer commodity hardware. Currently that means x64 systems with Nehalem or newer
CPUs and UEFI firmware. (See [this list][cpu_features] for the currently CPUs and UEFI firmware. (See [this list][cpu_features] for the currently
required CPU features.) Eventually I'd like to work on an AArch64 port, required CPU features.) Eventually I'd like to work on an AArch64 port,
@@ -24,8 +23,9 @@ The design goals of the project are:
by the traditional microkernel problems. by the traditional microkernel problems.
* Exploration - I'm really mostly doing this to have fun learning and exploring * Exploration - I'm really mostly doing this to have fun learning and exploring
modern OS development. Initial feature implementations may temporarily throw modern OS development. Modular design may be tossed out (hopefully
away modular design to allow for exploration of the related hardware. temporarily) in some places to allow me to play around with the related
hardware.
A note on the name: This kernel was originally named Popcorn, but I have since A note on the name: This kernel was originally named Popcorn, but I have since
discovered that the Popcorn Linux project is also developing a kernel with that discovered that the Popcorn Linux project is also developing a kernel with that
@@ -35,147 +35,48 @@ and my wonderful wife.
[cpu_features]: https://github.com/justinian/jsix/blob/master/src/libraries/cpu/include/cpu/features.inc [cpu_features]: https://github.com/justinian/jsix/blob/master/src/libraries/cpu/include/cpu/features.inc
## Status and Roadmap
The following major feature areas are targets for jsix development:
#### UEFI boot loader
_Done._ The bootloader loads the kernel and initial userspace programs, and
sets up necessary kernel arguments about the memory map and EFI GOP
framebuffer. Possible future ideas:
- take over more init-time functions from the kernel
- rewrite it in Zig
#### Memory
_Virtual memory: Sufficient._ The kernel manages virtual memory with a number
of kinds of `vm_area` objects representing mapped areas, which can belong to
one or more `vm_space` objects which represent a whole virtual memory space.
(Each process has a `vm_space`, and so does the kernel itself.)
Remaining to do:
- TLB shootdowns
- Page swapping
- Large / huge page support
_Physical page allocation: Sufficient._ The current physical page allocator
implementation uses a group of blocks representing up-to-1GiB areas of usable
memory as defined by the bootloader. Each block has a three-level bitmap
denoting free/used pages.
Future work:
- Align blocks so that their first page is 1GiB-aligned, making finding free
large/huge pages easier.
#### Multitasking
_Sufficient._ The global scheduler object keeps separate ready/blocked lists
per core. Cores periodically attempt to balance load via work stealing.
User-space tasks are able to create threads as well as other processes.
#### API
_Syscalls: Sufficient._ User-space tasks are able to make syscalls to the
kernel via fast SYSCALL/SYSRET instructions. Syscalls made via `libj6` look to
both the callee and the caller like standard SysV ABI function calls. The
implementations are wrapped in generated wrapper functions which validate the
request, check capabilities, and find the appropriate kernel objects or handles
before calling the implementation functions.
_IPC: Working, needs optimization._ The current IPC primitives are:
- _Mailboxes_: endpoints for asynchronously-delivered small messages. Currently
these messages are double-copied - once from the caller into a kernel buffer,
and once from the kernel to the receiver. This works and is not a major cause
of slowdown, but will need to be optimized in the future.
- _Channels_: endpoints for asynchronous uni-directional streams of bytes.
Currently these also suffer from a double-copy problem, and should probably
be replaced eventually by userspace shared memory communication.
- _Events_: objects that can be signalled to send asynchronous notifications to
waiting threads.
#### Hardware Support
- Framebuffer driver: _In progress._ Currently on machines with a video
device accessible by UEFI, jsix starts a user-space framebuffer driver that
only prints out kernel logs.
- Serial driver: _In progress._ The current UART currently only exposes COM1
as an output channel, but no input or other serial ports are exposed.
- USB driver: _To do_
- AHCI (SATA) driver: _To do_
## Building ## Building
jsix uses the [Ninja][] build tool, and generates the build files for it with jsix uses the [Ninja][] build tool, and generates the build files for it with a
the `configure` script. The build also relies on a custom toolchain sysroot, which can be custom tool called [Bonnibel][]. Bonnibel can be installed with [Cargo][], or
downloaded or built using the scripts in [jsix-os/toolchain][]. downloaded as a prebuilt binary from its Github repository.
[Ninja]: https://ninja-build.org [Ninja]: https://ninja-build.org
[jsix-os/toolchain]: https://github.com/jsix-os/toolchain [Bonnibel]: https://github.com/justinian/bonnibel_rs
[Cargo]: https://crates.io/crates/bonnibel
Other build dependencies: Requrirements:
* [clang][]: the C/C++ compiler * bonnibel
* [nasm][]: the assembler * ninja
* [lld][]: the linker * clang
* [mtools][]: for creating the FAT image * nasm
* mtools
* curl for downloading the toolchain
[clang]: https://clang.llvm.org ### Setting up the cross toolchain
[nasm]: https://www.nasm.us
[lld]: https://lld.llvm.org
[mtools]: https://www.gnu.org/software/mtools/
The `configure` script has some Python dependencies - these can be installed via Running `pb sync` will download and unpack the toolchain into `sysroot`.
`pip`, though doing so in a python virtual environment is recommended.
Installing via `pip` will also install `ninja`.
A Debian 11 (Bullseye) system can be configured with the necessary build #### Compiling the toolchain yourself
dependencies by running the following commands from the jsix repository root:
```bash If you have `clang` and `curl` installed, runing the `scripts/build_sysroot.sh`
sudo apt install clang lld nasm mtools python3-pip python3-venv script will download and build a LLVM toolchain configured for building jsix
python3 -m venv ./venv host binaries.
source venv/bin/activate
pip install -r requirements.txt
peru sync
```
### Setting up the sysroot
Build or download the toolchain sysroot as mentioned above with
[jsix-os/toolchain][], and symlink the built toolchain directory as `sysroot`
at the root of this project.
```bash
# Example if both the toolchain and this project are cloned under ~/src
ln -s ~/src/toolchain/toolchains/llvm-13 ~/src/jsix/sysroot
```
### Building and running jsix ### Building and running jsix
Once the toolchain has been set up, running the `./configure` script (see Once the toolchain has been set up, running Bonnibel's `pb init` command will
`./configure --help` for available options) will set up the build configuration, set up the build configuration, and `pb build` will actually run the build. If
and `ninja -C build` (or wherever you put the build directory) will actually run you have `qemu-system-x86_64` installed, the `qemu.sh` script will to run jsix
the build. If you have `qemu-system-x86_64` installed, the `qemu.sh` script will in QEMU `-nographic` mode.
to run jsix in QEMU `-nographic` mode.
I personally run this either from a real debian amd64 bullseye machine or I personally run this either from a real debian amd64 testing/buster machine or
a windows WSL debian bullseye installation. Your mileage may vary with other a windows WSL debian testing/buster installation. The following should be
setups and distros. enough to set up such a system to build the kernel:
### Running the test suite sudo apt install qemu-system-x86 nasm clang-10 mtools curl ninja-build
sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-10 1000
sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-10 1000
curl -L -o pb https://github.com/justinian/bonnibel_rs/releases/download/v2.3.0/pb-linux-amd64 && chmod a+x pb
jsix now has the `test_runner` userspace program that runs various automated
tests. It is not included in the default build, but if you use the `test.yml`
manifest it will be built, and can be run with the `test.sh` script or the
`qemu.sh` script.
```bash
./configure --manifest=assets/manifests/test.yml
if ./test.sh; then echo "All tests passed!"; else echo "Failed."; fi
```

242
assets/debugging/jsix.elf-gdb.py
View File

@@ -10,7 +10,6 @@ class PrintStackCommand(gdb.Command):
base = "$rsp" base = "$rsp"
if len(args) > 0: if len(args) > 0:
base = args[0] base = args[0]
base = int(gdb.parse_and_eval(base))
depth = 22 depth = 22
if len(args) > 1: if len(args) > 1:
@@ -19,35 +18,14 @@ class PrintStackCommand(gdb.Command):
for i in range(depth-1, -1, -1): for i in range(depth-1, -1, -1):
try: try:
offset = i * 8 offset = i * 8
value = gdb.parse_and_eval(f"*(uint64_t*)({base:#x} + {offset:#x})") base_addr = gdb.parse_and_eval(base)
print("{:016x} (+{:04x}): {:016x}".format(base + offset, offset, int(value))) value = gdb.parse_and_eval(f"*(uint64_t*)({base} + {offset})")
print("{:016x} (+{:04x}): {:016x}".format(int(base_addr) + offset, offset, int(value)))
except Exception as e: except Exception as e:
print(e) print(e)
continue continue
def stack_walk(frame, depth):
for i in range(depth-1, -1, -1):
ret = gdb.parse_and_eval(f"*(uint64_t*)({frame:#x} + 0x8)")
name = ""
try:
block = gdb.block_for_pc(int(ret))
if block:
name = block.function or ""
except RuntimeError:
pass
try:
print("{:016x}: {:016x} {}".format(int(frame), int(ret), name))
frame = int(gdb.parse_and_eval(f"*(uint64_t*)({frame:#x})"))
except gdb.MemoryError:
return
if frame == 0 or ret == 0:
return
class PrintBacktraceCommand(gdb.Command): class PrintBacktraceCommand(gdb.Command):
def __init__(self): def __init__(self):
super().__init__("j6bt", gdb.COMMAND_DATA) super().__init__("j6bt", gdb.COMMAND_DATA)
@@ -55,215 +33,31 @@ class PrintBacktraceCommand(gdb.Command):
def invoke(self, arg, from_tty): def invoke(self, arg, from_tty):
args = gdb.string_to_argv(arg) args = gdb.string_to_argv(arg)
frame = "$rbp"
if len(args) > 0:
frame = args[0]
frame = int(gdb.parse_and_eval(f"{frame}"))
depth = 30 depth = 30
if len(args) > 0:
depth = int(args[0])
frame = "$rbp"
if len(args) > 1: if len(args) > 1:
depth = int(gdb.parse_and_eval(args[1])) frame = args[1]
stack_walk(frame, depth) for i in range(depth-1, -1, -1):
ret = gdb.parse_and_eval(f"*(uint64_t*)({frame} + 8)")
frame = gdb.parse_and_eval(f"*(uint64_t*)({frame})")
name = ""
block = gdb.block_for_pc(int(ret))
if block:
name = block.function or ""
class TableWalkCommand(gdb.Command): print("{:016x} {}".format(int(ret), name))
def __init__(self):
super().__init__("j6tw", gdb.COMMAND_DATA)
def invoke(self, arg, from_tty): if frame == 0 or ret == 0:
args = gdb.string_to_argv(arg)
if len(args) < 2:
raise Exception("Must be: j6tw <pml4> <addr>")
pml4 = int(gdb.parse_and_eval(args[0]))
addr = int(gdb.parse_and_eval(args[1]))
indices = [
(addr >> 39) & 0x1ff,
(addr >> 30) & 0x1ff,
(addr >> 21) & 0x1ff,
(addr >> 12) & 0x1ff,
]
names = ["PML4", "PDP", "PD", "PT"]
table_flags = [
(0x0001, "present"),
(0x0002, "write"),
(0x0004, "user"),
(0x0008, "pwt"),
(0x0010, "pcd"),
(0x0020, "accessed"),
(0x0040, "dirty"),
(0x0080, "largepage"),
(0x0100, "global"),
(0x1080, "pat"),
((1<<63), "xd"),
]
page_flags = [
(0x0001, "present"),
(0x0002, "write"),
(0x0004, "user"),
(0x0008, "pwt"),
(0x0010, "pcd"),
(0x0020, "accessed"),
(0x0040, "dirty"),
(0x0080, "pat"),
(0x0100, "global"),
((1<<63), "xd"),
]
flagsets = [table_flags, table_flags, table_flags, page_flags]
table = pml4
entry = 0
for i in range(len(indices)):
entry = int(gdb.parse_and_eval(f'((uint64_t*)0x{table:x})[0x{indices[i]:x}]'))
flagset = flagsets[i]
flag_names = " | ".join([f[1] for f in flagset if (entry & f[0]) == f[0]])
print(f"{names[i]:>4}: {table:016x}")
print(f" index: {indices[i]:3} {entry:016x}")
print(f" flags: {flag_names}")
if (entry & 1) == 0 or (i < 3 and (entry & 0x80)):
break
table = (entry & 0x7ffffffffffffe00) | 0xffffc00000000000
class GetThreadsCommand(gdb.Command):
FLAGS = {
"ready": 0x01,
"loading": 0x02,
"exited": 0x04,
"constant": 0x80,
}
def __init__(self):
super().__init__("j6threads", gdb.COMMAND_DATA)
def get_flags(self, bitset):
flags = []
for k, v in GetThreadsCommand.FLAGS.items():
if bitset & v:
flags.append(k)
return " ".join(flags)
def print_thread(self, addr):
if addr == 0:
print(" <no thread>\n")
return return
tcb = f"((TCB*){addr:#x})"
thread = f"({tcb}->thread)"
stack = int(gdb.parse_and_eval(f"{tcb}->kernel_stack"))
rsp = int(gdb.parse_and_eval(f"{tcb}->rsp"))
pri = int(gdb.parse_and_eval(f"{tcb}->priority"))
flags = int(gdb.parse_and_eval(f"{thread}->m_state"))
koid = int(gdb.parse_and_eval(f"{thread}->m_koid"))
proc = int(gdb.parse_and_eval(f"{thread}->m_parent.m_koid"))
creator = int(gdb.parse_and_eval(f"{thread}->m_creator"))
if creator != 0:
creator_koid = int(gdb.parse_and_eval(f"{thread}->m_creator->m_koid"))
creator = f"{creator_koid:x}"
else:
creator = "<no thread>"
print(f" Thread {proc:x}:{koid:x}")
print(f" creator: {creator}")
print(f" priority: {pri}")
print(f" flags: {self.get_flags(flags)}")
print(f" kstack: {stack:#x}")
print(f" rsp: {rsp:#x}")
print("------------------------------------")
if stack != 0:
rsp = int(gdb.parse_and_eval(f"{tcb}->rsp"))
stack_walk(rsp + 5*8, 5)
print("")
def print_thread_list(self, addr, name):
if addr == 0:
return
print(f"=== {name} ===")
while addr != 0:
self.print_thread(addr)
addr = int(gdb.parse_and_eval(f"((tcb_node*){addr:#x})->m_next"))
def invoke(self, arg, from_tty):
args = gdb.string_to_argv(arg)
if len(args) > 1:
raise RuntimeError("Usage: j6threads [cpu]")
ncpus = int(gdb.parse_and_eval("g_num_cpus"))
cpus = list(range(ncpus))
if len(args) == 1:
cpus = [int(args[0])]
for cpu in cpus:
runlist = f"scheduler::s_instance->m_run_queues.m_elements[{cpu:#x}]"
print(f"=== CPU {cpu:2}: CURRENT ===")
current = int(gdb.parse_and_eval(f"{runlist}.current"))
self.print_thread(current)
for pri in range(8):
ready = int(gdb.parse_and_eval(f"{runlist}.ready[{pri:#x}].m_head"))
self.print_thread_list(ready, f"CPU {cpu:2}: PRIORITY {pri}")
blocked = int(gdb.parse_and_eval(f"{runlist}.blocked.m_head"))
self.print_thread_list(blocked, f"CPU {cpu:2}: BLOCKED")
class PrintProfilesCommand(gdb.Command):
def __init__(self):
super().__init__("j6prof", gdb.COMMAND_DATA)
def invoke(self, arg, from_tty):
args = gdb.string_to_argv(arg)
if len(args) != 1:
raise RuntimeError("Usage: j6prof <profiler class>")
profclass = args[0]
root_type = f"profile_class<{profclass}>"
try:
baseclass = gdb.lookup_type(root_type)
except Exception as e:
print(e)
return
results = {}
max_len = 0
count = gdb.parse_and_eval(f"{profclass}::count")
for i in range(count):
name = gdb.parse_and_eval(f"{root_type}::function_names[{i:#x}]")
if name == 0: continue
call_counts = gdb.parse_and_eval(f"{root_type}::call_counts[{i:#x}]")
call_durations = gdb.parse_and_eval(f"{root_type}::call_durations[{i:#x}]")
results[name.string()] = float(call_durations) / float(call_counts)
max_len = max(max_len, len(name.string()))
for name, avg in results.items():
print(f"{name:>{max_len}}: {avg:15.3f}")
PrintStackCommand() PrintStackCommand()
PrintBacktraceCommand() PrintBacktraceCommand()
TableWalkCommand()
GetThreadsCommand()
PrintProfilesCommand()
gdb.execute("target remote :1234")
gdb.execute("display/i $rip") gdb.execute("display/i $rip")
if not gdb.selected_inferior().was_attached:
gdb.execute("add-symbol-file build/panic.serial.elf")
gdb.execute("target remote :1234")

BIN
assets/diskbase.img
View File

Binary file not shown.

47
assets/grammars/definitions.g
View File

@@ -1,47 +0,0 @@
start: import_statement* (object|interface)+
import_statement: "import" PATH
object: description? "object" name options? super? "{" uid cname? capabilities? method* "}"
interface: description? "interface" name options? "{" uid interface_param* "}"
?interface_param: expose | function
expose: "expose" type
uid: "uid" UID
cname: "cname" IDENTIFIER
capabilities: "capabilities" "[" IDENTIFIER+ "]"
super: ":" name
function: description? "function" name options? ("{" param* "}")?
method: description? "method" name options? ("{" param* "}")?
param: "param" name type options? description?
?type: PRIMITIVE | object_name
object_name: "ref" name
id: NUMBER
name: IDENTIFIER
options: "[" ( OPTION | IDENTIFIER )+ "]"
description: COMMENT+
PRIMITIVE: INT_TYPE | "size" | "string" | "buffer" | "address"
INT_TYPE: /u?int(8|16|32|64)?/
NUMBER: /(0x)?[0-9a-fA-F]+/
UID: /[0-9a-fA-F]{16}/
OPTION.2: IDENTIFIER ":" IDENTIFIER
COMMENT: /#.*/
PATH: /"[^"]*"/
%import common.LETTER
%import common.CNAME -> IDENTIFIER
%import common.WS
%ignore WS

28
assets/initrd.toml Normal file
View File

@@ -0,0 +1,28 @@
# This is the manifest for the initial ramdisk, read by the `makerd` tool.
# The contents should be a table array of files to add to the ramdistk:
#
# [[files]]
# dest = "foo.bar" # Name of the file in the ramdisk
# source = "build/foo/foo.bar" # Location of the file from the project root
# executable = true # Optional, default false. Whether this is an
# # initial application for the kernel to execute
# # on startup
[[files]]
dest = "screenfont.psf"
source = "../assets/fonts/tamsyn8x16r.psf"
[[files]]
dest = "symbol_table.dat"
source = "symbol_table.dat"
symbols = true
[[files]]
dest = "nulldrv1"
source = "user/nulldrv"
executable = true
[[files]]
dest = "nulldrv2"
source = "user/nulldrv"
executable = true

8
assets/manifests/default.yaml
View File

@@ -1,8 +0,0 @@
---
init: srv.init
programs:
- name: panic.serial
target: kernel
flags: panic
- name: srv.logger
- name: drv.uart

9
assets/manifests/test.yaml
View File

@@ -1,9 +0,0 @@
---
init: srv.init
flags: ["test"]
programs:
- name: panic.serial
target: kernel
flags: panic
- name: drv.uart
- name: test_runner

39
configs/base.yaml
View File

@@ -1,39 +0,0 @@
---
variables:
cc: "${source_root}/sysroot/bin/clang"
cxx: "${source_root}/sysroot/bin/clang++"
ld: "${source_root}/sysroot/bin/ld.lld"
ar: ar
nasm: nasm
objcopy: objcopy
ccflags: [
"-I${source_root}/src/include",
"-I${source_root}/src/include/x86_64",
"-fcolor-diagnostics",
"-U__STDCPP_THREADS__",
"-D_LIBCPP_HAS_NO_THREADS",
"-DVERSION_MAJOR=${version_major}",
"-DVERSION_MINOR=${version_minor}",
"-DVERSION_PATCH=${version_patch}",
"-DVERSION_GITSHA=0x${version_sha}",
'-DGIT_VERSION=\"${version_major}.${version_minor}.${version_patch}+${version_sha}\"',
'-DGIT_VERSION_WIDE=L\"${version_major}.${version_minor}.${version_patch}+${version_sha}\"',
"-Wformat=2", "-Winit-self", "-Wfloat-equal", "-Winline", "-Wmissing-format-attribute",
"-Wmissing-include-dirs", "-Wswitch", "-Wundef", "-Wdisabled-optimization",
"-Wpointer-arith", "-Wno-attributes", "-Wno-sign-compare", "-Wno-multichar",
"-Wno-div-by-zero", "-Wno-endif-labels", "-Wno-pragmas", "-Wno-format-extra-args",
"-Wno-unused-result", "-Wno-deprecated-declarations", "-Wno-unused-function",
"-Wno-address-of-packed-member", "-Wno-invalid-offsetof", "-Wno-format-nonliteral",
"-Werror" ]
asflags: [
"-DVERSION_MAJOR=${version_major}",
"-DVERSION_MINOR=${version_minor}",
"-DVERSION_PATCH=${version_patch}",
"-DVERSION_GITSHA=0x${version_sha}",
"-I${source_root}/src/include" ]
cflags: [ "-std=c11" ]
cxxflags: [ "-std=c++17" ]

30
configs/boot-debug.yaml
View File

@@ -1,30 +0,0 @@
---
extends: base
variables:
ld: clang++
ccflags: [
"-nostdlib",
"-nodefaultlibs",
"-fno-builtin",
"-I${source_root}/external",
"--target=x86_64-unknown-windows",
"-ffreestanding",
"-mno-red-zone",
"-fshort-wchar",
"-fno-omit-frame-pointer",
"-ggdb",
"-g3" ]
cxxflags: [ "-fno-exceptions", "-fno-rtti" ]
ldflags: [
"--target=x86_64-unknown-windows",
"-nostdlib",
"-Wl,-entry:efi_main",
"-Wl,-subsystem:efi_application",
"-fuse-ld=lld-link",
"-g" ]

55
configs/kernel-debug.yaml
View File

@@ -1,55 +0,0 @@
---
extends: base
variables:
asflags: [ "-I${source_root}/src/kernel/" ]
ccflags: [
"--target=x86_64-jsix-elf",
"-fno-stack-protector",
"-I${source_root}/external",
"-nostdinc",
"-nostdlib",
"-ffreestanding",
"-nodefaultlibs",
"-fno-builtin",
"-mno-sse",
"-fno-omit-frame-pointer",
"-mno-red-zone",
"-mcmodel=kernel",
"-g3",
"-ggdb",
"-D__ELF__",
"-D__jsix__",
"-D__j6kernel",
"-U__linux",
"-U__linux__",
"-DPRINTF_ALIAS_STANDARD_FUNCTION_NAMES=1",
"-DPRINTF_INCLUDE_CONFIG_H=1",
"-isystem${build_root}/include/libc",
"-isystem${source_root}/sysroot/include",
"--sysroot='${source_root}/sysroot'" ]
cflags: [ '-nostdinc' ]
cxxflags: [
"-fno-exceptions",
"-fno-rtti",
"-nostdinc",
"-isystem${source_root}/sysroot/include/c++/v1" ]
ldflags: [
"-g",
"-nostdlib",
"-Bstatic",
"--no-eh-frame-hdr",
"-z", "norelro",
"-z", "separate-code" ]

81
configs/rules.ninja
View File

@@ -1,81 +0,0 @@
# This file is automatically generated by bonnibel
rule compile.c
command = $cc -MMD -MF $out.d $cflags $ccflags -o $out -c $in
description = Compiling $name
depfile = $out.d
deps = gcc
rule dump_c_defs
command = echo | $cc $ccflags $cflags -dM -E - > $out
description = Dumping C defines for $target
rule dump_c_run
command = echo '#!/bin/bash' > $out; echo '$cc $ccflags $cflags $$*' >> $
$out; chmod a+x $out
description = Dumping C arguments for $target
rule compile.cpp
command = $cxx -MMD -MF $out.d $cxxflags $ccflags -o $out -c $in
description = Compiling $name
depfile = $out.d
deps = gcc
rule dump_cpp_defs
command = echo | $cxx -x c++ $ccflags $cxxflags -dM -E - > $out
description = Dumping C++ defines for $target
rule dump_cpp_run
command = echo '#!/bin/bash' > $out; echo '$cxx $ccflags $cxxflags $$*' $
>> $out; chmod a+x $out
description = Dumping C++ arguments for $target
rule compile.s
command = $nasm -o $out -felf64 -MD $out.d $asflags $in
description = Assembling $name
depfile = $out.d
deps = gcc
rule parse.cog
command = cog -o $out -d -D target=$target $cogflags $in
description = Parsing $name
rule exe
command = $ld $ldflags -o $out $in $libs
description = Linking $name
rule lib
command = $ar qcs $out $in
description = Archiving $name
rule cp
command = cp $in $out
description = Copying $name
rule dump
command = objdump -DSC -M intel $in > $out
description = Dumping decompiled $name
rule makest
description = Making symbol table
command = nm -n -S --demangle $in | ${source_root}/scripts/build_symbol_table.py $out
rule makefat
description = Creating $name
command = $
cp $in $out; $
mcopy -s -D o -i $out@@1M ${build_root}/fatroot/* ::/
rule strip
description = Stripping $name
command = $
cp $in $out; $
objcopy --only-keep-debug $out $debug; $
strip -g $out; $
objcopy --add-gnu-debuglink=$debug $out
rule touch
command = touch $out
rule compdb
command = ninja -t compdb > $out

39
configs/user-debug.yaml
View File

@@ -1,39 +0,0 @@
---
extends: base
variables:
asflags: [ "-I${source_root}/src/kernel/" ]
ccflags: [
"--target=x86_64-jsix-elf",
"-mno-sse",
"-fno-omit-frame-pointer",
"-fno-stack-protector",
"-g",
"-D__ELF__",
"-D__jsix__",
"-U__linux",
"-U__linux__",
"-isystem${source_root}/sysroot/include",
"-isystem${build_root}/include/libc",
"--sysroot='${source_root}/sysroot'" ]
cxxflags: [
"-fno-exceptions",
"-fno-rtti",
"-isystem${source_root}/sysroot/include/c++/v1" ]
ldflags: [
"-g",
"-Bstatic",
"--sysroot='${source_root}/sysroot'",
"-L", "${source_root}/sysroot/lib",
"-z", "separate-code",
"-lc++", "-lc++abi", "-lunwind",
"--no-dependent-libraries",
]

70
configure vendored
View File

@@ -1,70 +0,0 @@
#!/usr/bin/env python3
def generate(output, config, manifest):
from os import makedirs, walk
from pathlib import Path
from bonnibel.module import Module
from bonnibel.project import Project
root = Path(__file__).parent.resolve()
project = Project(root)
output = root / output
sources = root / "src"
manifest = root / manifest
modules = {}
for base, dirs, files in walk(sources):
path = Path(base)
for f in files:
if f.endswith(".module"):
fullpath = path / f
def module_init(name, **kwargs):
m = Module(name, fullpath, path, **kwargs)
modules[m.name] = m
return m
glo = {
"module": module_init,
"source_root": root,
"built_root": output,
"module_root": path,
}
code = compile(open(fullpath, 'r').read(), fullpath, "exec")
loc = {}
exec(code, glo, loc)
Module.update(modules)
makedirs(output.resolve(), exist_ok=True)
project.generate(root, output, modules, config, manifest)
for mod in modules.values():
mod.generate(output)
if __name__ == "__main__":
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent / "scripts"))
from argparse import ArgumentParser
from bonnibel import BonnibelError
p = ArgumentParser(description="Generate jsix build files")
p.add_argument("--manifest", "-m", metavar="FILE", default="assets/manifests/default.yaml",
help="File to use as the system manifest")
p.add_argument("--config", "-c", metavar="NAME", default="debug",
help="Configuration to build (eg, 'debug' or 'release')")
p.add_argument("output", metavar="DIR", default="build", nargs='?',
help="Where to create the build root")
args = p.parse_args()
try:
generate(args.output, args.config, args.manifest)
except BonnibelError as be:
import sys
print(f"Error: {be}", file=sys.stderr)
sys.exit(1)

20
definitions/memory_layout.yaml
View File

@@ -1,20 +0,0 @@
---
- name: linear
size: 64T
shared: true
- name: bitmap
size: 1T
shared: true
- name: heap
size: 64G
- name: stacks
size: 64G
- name: buffers
size: 64G
- name: slabs
size: 64G

21
definitions/objects/channel.def
View File

@@ -1,21 +0,0 @@
object channel : object {
uid 3ea38b96aa0e54c8
capabilities [
send
receive
close
]
method create [constructor]
method close [destructor cap:close]
method send [cap:send] {
param data buffer [inout]
}
method receive [cap:receive] {
param data buffer [out]
param flags uint64
}
}

22
definitions/objects/event.def
View File

@@ -1,22 +0,0 @@
object event : object {
uid f441e03da5516b1a
capabilities [
signal
wait
]
method create [constructor]
# Signal events on this object
method signal [cap:signal] {
param signals uint64 # A bitset of which events to signal
}
# Wait for signaled events on this object
method wait [cap:wait] {
param signals uint64 [out] # A bitset of which events were signaled
param timeout uint64 # Wait timeout in nanoseconds
}
}

65
definitions/objects/mailbox.def
View File

@@ -1,65 +0,0 @@
# Mailboxes are objects that enable synchronous or asynchronous
# IPC via short message-passing of bytes and handles.
object mailbox : object {
uid 99934ad04ece1e07
capabilities [
send
receive
close
]
method create [constructor]
method close [destructor cap:close]
# Asynchronously send a message to the reciever
method send [cap:send handle] {
param tag uint64
param data buffer [zero_ok]
param handles ref object [list]
}
# Receive a pending message, or block waiting for a message to
# arrive if block is true.
method receive [cap:receive] {
param tag uint64 [out]
param data buffer [out zero_ok]
param handles ref object [out list zero_ok]
param reply_tag uint16 [out optional]
param badge uint64 [out optional]
param flags uint64
}
# Send a message to the reciever, and block until a
# response is sent. Note that getting this response
# does not require the receive capability.
method call [cap:send handle] {
param tag uint64 [inout]
param data buffer [inout zero_ok]
param handles ref object [inout list zero_ok]
}
# Respond to a message sent using call. Note that this
# requires the receive capability and not the send capability.
method respond [cap:receive] {
param tag uint64
param data buffer [zero_ok]
param handles ref object [list zero_ok]
param reply_tag uint16
}
# Respond to a message sent using call, and wait for another
# message to arrive. Note that this does not require the send
# capability.
method respond_receive [cap:receive] {
param tag uint64 [inout]
param data buffer [inout zero_ok]
param data_in size
param handles ref object [inout list zero_ok]
param handles_in size
param reply_tag uint16 [inout]
param badge uint64 [out optional]
param flags uint64
}
}

14
definitions/objects/object.def
View File

@@ -1,14 +0,0 @@
# The base type of all kernel-exposed objects
object object [virtual] {
uid 667f61fb2cd57bb4
cname kobject
capabilities [
clone
]
# Get the internal kernel object id of an object
method koid {
param koid uint64 [out]
}
}

31
definitions/objects/process.def
View File

@@ -1,31 +0,0 @@
import "objects/object.def"
# Processes are a collection of handles and a virtual memory
# space inside which threads are run.
object process : object {
uid 0c69ee0b7502ba31
capabilities [
kill
create_thread
]
# Create a new empty process
method create [constructor]
# Stop all threads and exit the given process
method kill [destructor cap:kill]
# Stop all threads and exit the current process
method exit [static noreturn] {
param result int32 # The result to retrun to the parent process
}
# Give the given process a handle that points to the same
# object as the specified handle.
method give_handle {
param target ref object [handle] # A handle in the caller process to send
param received ref object [out optional] # The handle as the recipient will see it
}
}

40
definitions/objects/system.def
View File

@@ -1,40 +0,0 @@
# The system object represents a handle to kernel functionality
# needed by drivers and other priviledged services
object system : object {
uid fa72506a2cf71a30
capabilities [
get_log
bind_irq
map_phys
change_iopl
]
# Get a log line from the kernel log
method get_log [cap:get_log] {
param buffer buffer [out zero_ok] # Buffer for the log message data structure
}
# Ask the kernel to send this process messages whenever
# the given IRQ fires
method bind_irq [cap:bind_irq] {
param dest ref event # Event object that will receive messages
param irq uint # IRQ number to bind
param signal uint # Signal number on the event to bind to
}
# Create a VMA and map an area of physical memory into it,
# also mapping that VMA into the current process
method map_phys [cap:map_phys] {
param area ref vma [out] # Receives a handle to the VMA created
param phys address # The physical address of the area
param size size # Size of the area, in pages
param flags uint32 # Flags to apply to the created VMA
}
# Request the kernel change the IOPL for this process. The only values
# that make sense are 0 and 3.
method request_iopl [cap:change_iopl] {
param iopl uint # The IOPL to set for this process
}
}

23
definitions/objects/thread.def
View File

@@ -1,23 +0,0 @@
object thread : object {
uid 11f23e593d5761bd
capabilities [
kill
]
method create [constructor] {
param process ref process [cap:create_thread]
param stack_top address
param entrypoint address
}
method kill [destructor cap:kill]
method exit [static] {
param status int32
}
method sleep [static] {
param duration uint64
}
}

36
definitions/objects/vma.def
View File

@@ -1,36 +0,0 @@
import "objects/process.def"
object vma : object {
uid d6a12b63b3ed3937
cname vm_area
capabilities [
map
unmap
resize
]
method create [constructor] {
param size size
param flags uint32
}
method create_map [constructor cap:map] {
param size size
param address address
param flags uint32
}
method map [cap:map] {
param process ref process
param address address
}
method unmap [cap:unmap] {
param process ref process
}
method resize [cap:resize] {
param size size [inout]
}
}

50
definitions/syscalls.def
View File

@@ -1,50 +0,0 @@
import "objects/object.def"
import "objects/channel.def"
import "objects/event.def"
import "objects/mailbox.def"
import "objects/process.def"
import "objects/system.def"
import "objects/thread.def"
import "objects/vma.def"
interface syscalls [syscall] {
uid 01d9b6a948961097
expose ref object
expose ref system
expose ref event
expose ref process
expose ref thread
expose ref mailbox
expose ref channel
expose ref vma
# Simple no-op syscall for testing
function noop
# Write a message to the kernel log
function log {
param message string
}
# Get a list of handles owned by this process. If the
# supplied list is not big enough, will set the size
# needed in `size` and return j6_err_insufficient
function handle_list {
param handles ref object [list inout zero_ok] # A list of handles to be filled
}
# Create a clone of an existing handle, possibly with
# some capabilities masked out.
function handle_clone {
param orig ref object [handle cap:clone] # The handle to clone
param clone ref object [out] # The new handle
param mask uint32 # The capability bitmask
}
# Testing mode only: Have the kernel finish and exit QEMU with the given exit code
function test_finish [test] {
param exit_code uint32
}
}

34
definitions/sysconf.yaml
View File

@@ -1,34 +0,0 @@
---
address: 0x6a360000
vars:
- name: version_major
section: kernel
type: uint8_t
- name: version_minor
section: kernel
type: uint8_t
- name: version_patch
section: kernel
type: uint16_t
- name: version_gitsha
section: kernel
type: uint32_t
- name: page_size
section: sys
type: size_t
- name: large_page_size
section: sys
type: size_t
- name: huge_page_size
section: sys
type: size_t
- name: num_cpus
section: sys
type: uint32_t

3669
external/cpptoml/cpptoml.h vendored Normal file
View File

File diff suppressed because it is too large Load Diff

330
modules.yaml Normal file
View File

@@ -0,0 +1,330 @@
name: jsix
templates: scripts/templates
modules:
kernel:
kind: exe
output: jsix.elf
target: host
deps:
- cpu
- kutil
includes:
- src/kernel
source:
- src/kernel/apic.cpp
- src/kernel/assert.cpp
- src/kernel/boot.s
- src/kernel/clock.cpp
- src/kernel/console.cpp
- src/kernel/cpprt.cpp
- src/kernel/cpu.cpp
- src/kernel/debug.cpp
- src/kernel/debug.s
- src/kernel/device_manager.cpp
- src/kernel/frame_allocator.cpp
- src/kernel/fs/gpt.cpp
- src/kernel/gdt.cpp
- src/kernel/gdt.s
- src/kernel/hpet.cpp
- src/kernel/interrupts.cpp
- src/kernel/interrupts.s
- src/kernel/io.cpp
- src/kernel/log.cpp
- src/kernel/main.cpp
- src/kernel/memory_bootstrap.cpp
- src/kernel/msr.cpp
- src/kernel/objects/channel.cpp
- src/kernel/objects/endpoint.cpp
- src/kernel/objects/kobject.cpp
- src/kernel/objects/thread.cpp
- src/kernel/objects/process.cpp
- src/kernel/objects/system.cpp
- src/kernel/objects/vm_area.cpp
- src/kernel/page_table.cpp
- src/kernel/page_tree.cpp
- src/kernel/pci.cpp
- src/kernel/scheduler.cpp
- src/kernel/serial.cpp
- src/kernel/symbol_table.cpp
- src/kernel/syscall.cpp
- src/kernel/syscall.s
- src/kernel/syscalls/channel.cpp
- src/kernel/syscalls/endpoint.cpp
- src/kernel/syscalls/object.cpp
- src/kernel/syscalls/process.cpp
- src/kernel/syscalls/system.cpp
- src/kernel/syscalls/thread.cpp
- src/kernel/syscalls/vm_area.cpp
- src/kernel/task.s
- src/kernel/vm_space.cpp
boot:
kind: exe
target: boot
output: boot.efi
deps:
- cpu
source:
- src/boot/main.cpp
- src/boot/console.cpp
- src/boot/error.cpp
- src/boot/fs.cpp
- src/boot/hardware.cpp
- src/boot/loader.cpp
- src/boot/memory.cpp
- src/boot/paging.cpp
- src/boot/status.cpp
- src/boot/support.cpp
nulldrv:
kind: exe
target: user
output: nulldrv.elf
deps:
- libc
source:
- src/drivers/nulldrv/io.cpp
- src/drivers/nulldrv/main.cpp
- src/drivers/nulldrv/serial.cpp
fb:
kind: exe
target: user
output: fb.elf
deps:
- libc
source:
- src/drivers/fb/font.cpp
- src/drivers/fb/main.cpp
- src/drivers/fb/screen.cpp
- src/drivers/fb/scrollback.cpp
kutil:
kind: lib
output: libkutil.a
includes:
- src/libraries/kutil/include
source:
- src/libraries/kutil/assert.cpp
- src/libraries/kutil/bip_buffer.cpp
- src/libraries/kutil/heap_allocator.cpp
- src/libraries/kutil/logger.cpp
- src/libraries/kutil/memory.cpp
- src/libraries/kutil/printf.c
cpu:
kind: lib
output: libcpu.a
includes:
- src/libraries/cpu/include
source:
- src/libraries/cpu/cpu.cpp
j6:
kind: lib
output: libj6.a
includes:
- src/libraries/j6/include
target: user
source:
- src/libraries/j6/syscalls.s
libc:
kind: lib
output: libc.a
includes:
- src/libraries/libc/include
deps:
- j6
target: user
defines:
- DISABLE_SSE
- LACKS_UNISTD_H
- LACKS_FCNTL_H
- LACKS_SYS_PARAM_H
- LACKS_SYS_MMAN_H
- LACKS_SCHED_H
- LACKS_STRINGS_H
- HAVE_MMAP=0
#- LACKS_STRING_H
#- LACKS_ERRNO_H
#- LACKS_STDLIB_H
#- LACKS_TIME_H
source:
- src/libraries/libc/arch/x86_64/_Exit.s
- src/libraries/libc/arch/x86_64/crt0.s
- src/libraries/libc/arch/x86_64/init_libc.c
- src/libraries/libc/ctype/isalnum.c
- src/libraries/libc/ctype/isalpha.c
- src/libraries/libc/ctype/isblank.c
- src/libraries/libc/ctype/iscntrl.c
- src/libraries/libc/ctype/isdigit.c
- src/libraries/libc/ctype/isgraph.c
- src/libraries/libc/ctype/islower.c
- src/libraries/libc/ctype/isprint.c
- src/libraries/libc/ctype/ispunct.c
- src/libraries/libc/ctype/isspace.c
- src/libraries/libc/ctype/isupper.c
- src/libraries/libc/ctype/isxdigit.c
- src/libraries/libc/ctype/tolower.c
- src/libraries/libc/ctype/toupper.c
- src/libraries/libc/inttypes/imaxabs.c
- src/libraries/libc/inttypes/imaxdiv.c
- src/libraries/libc/inttypes/strtoimax.c
- src/libraries/libc/inttypes/strtoumax.c
- src/libraries/libc/locale/localeconv.c
- src/libraries/libc/locale/setlocale.c
- src/libraries/libc/j6libc/assert.c
- src/libraries/libc/j6libc/errno.c
- src/libraries/libc/j6libc/allocpages.c
- src/libraries/libc/j6libc/atomax.c
- src/libraries/libc/j6libc/closeall.c
- src/libraries/libc/j6libc/close.c
- src/libraries/libc/j6libc/digits.c
- src/libraries/libc/j6libc/filemode.c
- src/libraries/libc/j6libc/fillbuffer.c
- src/libraries/libc/j6libc/flushbuffer.c
- src/libraries/libc/j6libc/is_leap.c
- src/libraries/libc/j6libc/load_lc_collate.c
- src/libraries/libc/j6libc/load_lc_ctype.c
- src/libraries/libc/j6libc/load_lc_messages.c
- src/libraries/libc/j6libc/load_lc_monetary.c
- src/libraries/libc/j6libc/load_lc_numeric.c
- src/libraries/libc/j6libc/load_lc_time.c
- src/libraries/libc/j6libc/load_lines.c
- src/libraries/libc/j6libc/open.c
- src/libraries/libc/j6libc/prepread.c
- src/libraries/libc/j6libc/prepwrite.c
- src/libraries/libc/j6libc/print.c
- src/libraries/libc/j6libc/rename.c
- src/libraries/libc/j6libc/scan.c
- src/libraries/libc/j6libc/seed.c
- src/libraries/libc/j6libc/seek.c
- src/libraries/libc/j6libc/stdinit.c
- src/libraries/libc/j6libc/strtox_main.c
- src/libraries/libc/j6libc/strtox_prelim.c
- src/libraries/libc/j6libc/sbrk.c
- src/libraries/libc/signal/raise.c
- src/libraries/libc/signal/signal.c
- src/libraries/libc/stdio/clearerr.c
- src/libraries/libc/stdio/fclose.c
- src/libraries/libc/stdio/feof.c
- src/libraries/libc/stdio/ferror.c
- src/libraries/libc/stdio/fflush.c
- src/libraries/libc/stdio/fgetc.c
- src/libraries/libc/stdio/fgetpos.c
- src/libraries/libc/stdio/fgets.c
- src/libraries/libc/stdio/fopen.c
- src/libraries/libc/stdio/fprintf.c
- src/libraries/libc/stdio/fputc.c
- src/libraries/libc/stdio/fputs.c
- src/libraries/libc/stdio/fread.c
- src/libraries/libc/stdio/freopen.c
- src/libraries/libc/stdio/fscanf.c
- src/libraries/libc/stdio/fseek.c
- src/libraries/libc/stdio/fsetpos.c
- src/libraries/libc/stdio/ftell.c
- src/libraries/libc/stdio/fwrite.c
- src/libraries/libc/stdio/getc.c
- src/libraries/libc/stdio/getchar.c
- src/libraries/libc/stdio/perror.c
- src/libraries/libc/stdio/printf.c
- src/libraries/libc/stdio/putc.c
- src/libraries/libc/stdio/putchar.c
- src/libraries/libc/stdio/puts.c
- src/libraries/libc/stdio/remove.c
- src/libraries/libc/stdio/rename.c
- src/libraries/libc/stdio/rewind.c
- src/libraries/libc/stdio/scanf.c
- src/libraries/libc/stdio/setbuf.c
- src/libraries/libc/stdio/setvbuf.c
- src/libraries/libc/stdio/snprintf.c
- src/libraries/libc/stdio/sprintf.c
- src/libraries/libc/stdio/sscanf.c
- src/libraries/libc/stdio/tmpfile.c
- src/libraries/libc/stdio/tmpnam.c
- src/libraries/libc/stdio/ungetc.c
- src/libraries/libc/stdio/vfprintf.c
- src/libraries/libc/stdio/vfscanf.c
- src/libraries/libc/stdio/vprintf.c
- src/libraries/libc/stdio/vscanf.c
- src/libraries/libc/stdio/vsnprintf.c
- src/libraries/libc/stdio/vsprintf.c
- src/libraries/libc/stdio/vsscanf.c
- src/libraries/libc/stdlib/abort.c
- src/libraries/libc/stdlib/abs.c
- src/libraries/libc/stdlib/atexit.c
- src/libraries/libc/stdlib/atoi.c
- src/libraries/libc/stdlib/atol.c
- src/libraries/libc/stdlib/atoll.c
- src/libraries/libc/stdlib/bsearch.c
- src/libraries/libc/stdlib/div.c
- src/libraries/libc/stdlib/exit.c
- src/libraries/libc/stdlib/_Exit.c
- src/libraries/libc/stdlib/getenv.c
- src/libraries/libc/stdlib/labs.c
- src/libraries/libc/stdlib/ldiv.c
- src/libraries/libc/stdlib/llabs.c
- src/libraries/libc/stdlib/lldiv.c
- src/libraries/libc/stdlib/malloc.c
- src/libraries/libc/stdlib/qsort.c
- src/libraries/libc/stdlib/rand.c
- src/libraries/libc/stdlib/srand.c
- src/libraries/libc/stdlib/strtol.c
- src/libraries/libc/stdlib/strtoll.c
- src/libraries/libc/stdlib/strtoul.c
- src/libraries/libc/stdlib/strtoull.c
- src/libraries/libc/stdlib/system.c
- src/libraries/libc/string/memchr.c
- src/libraries/libc/string/memcmp.c
- src/libraries/libc/string/memcpy.c
- src/libraries/libc/string/memmove.c
- src/libraries/libc/string/memset.c
- src/libraries/libc/string/strcat.c
- src/libraries/libc/string/strchr.c
- src/libraries/libc/string/strcmp.c
- src/libraries/libc/string/strcoll.c
- src/libraries/libc/string/strcpy.c
- src/libraries/libc/string/strcspn.c
- src/libraries/libc/string/strerror.c
- src/libraries/libc/string/strlen.c
- src/libraries/libc/string/strncat.c
- src/libraries/libc/string/strncmp.c
- src/libraries/libc/string/strncpy.c
- src/libraries/libc/string/strpbrk.c
- src/libraries/libc/string/strrchr.c
- src/libraries/libc/string/strspn.c
- src/libraries/libc/string/strstr.c
- src/libraries/libc/string/strtok.c
- src/libraries/libc/string/strxfrm.c
- src/libraries/libc/time/asctime.c
- src/libraries/libc/time/clock.c
- src/libraries/libc/time/ctime.c
- src/libraries/libc/time/difftime.c
- src/libraries/libc/time/gmtime.c
- src/libraries/libc/time/localtime.c
- src/libraries/libc/time/mktime.c
- src/libraries/libc/time/strftime.c
- src/libraries/libc/time/time.c
- src/libraries/libc/time/timespec_get.c
tests:
kind: exe
target: native
output: tests
deps:
- kutil
source:
- src/tests/constexpr_hash.cpp
- src/tests/linked_list.cpp
- src/tests/logger.cpp
- src/tests/heap_allocator.cpp
- src/tests/main.cpp
- src/tests/map.cpp
- src/tests/vector.cpp
overlays:
- url: https://f000.backblazeb2.com/file/jsix-os/sysroot-llvm8-20190706.tar.bz2
path: sysroot
- url: https://f000.backblazeb2.com/file/jsix-os/sysroot-j6libc-8cb8ce7.tar.bz2
path: sysroot

34
other_software.md
View File

@@ -33,18 +33,42 @@ Boost Software license:
> ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER > ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> DEALINGS IN THE SOFTWARE. > DEALINGS IN THE SOFTWARE.
## cpptoml
jsix uses the [cpptoml][] library for parsing TOML configuration files. cpptoml
is released under the terms of the MIT license:
[cpptoml]: https://github.com/skystrife/cpptoml
> Copyright (c) 2014 Chase Geigle
>
> Permission is hereby granted, free of charge, to any person obtaining a copy of
> this software and associated documentation files (the "Software"), to deal in
> the Software without restriction, including without limitation the rights to
> use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
> the Software, and to permit persons to whom the Software is furnished to do so,
> subject to the following conditions:
>
> The above copyright notice and this permission notice shall be included in all
> copies or substantial portions of the Software.
>
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
> FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
> COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
> IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
## printf ## printf
The implementation of jsix's `*printf()` family of functions is derived from jsix uses Marco Paland's tiny [printf][] library for its `*printf()` functions,
Marco Paland and Eyal Rozenberg's tiny [printf][] library, which is also which is also released under the terms of the MIT license:
released under the terms of the MIT license:
[printf]: https://github.com/eyalroz/printf [printf]: https://github.com/mpaland/printf
> The MIT License (MIT) > The MIT License (MIT)
> >
> Copyright (c) 2014 Marco Paland > Copyright (c) 2014 Marco Paland
> Copyright (c) 2021 Eyal Rozenberg
> >
> Permission is hereby granted, free of charge, to any person obtaining a copy > Permission is hereby granted, free of charge, to any person obtaining a copy
> of this software and associated documentation files (the "Software"), to deal > of this software and associated documentation files (the "Software"), to deal

58
qemu.sh
View File

@@ -3,55 +3,37 @@
build="$(dirname $0)/build" build="$(dirname $0)/build"
assets="$(dirname $0)/assets" assets="$(dirname $0)/assets"
debug="" debug=""
isaexit='-device isa-debug-exit,iobase=0xf4,iosize=0x04'
debugtarget="${build}/jsix.elf" debugtarget="${build}/jsix.elf"
flash_name="ovmf_vars"
gfx="-nographic" gfx="-nographic"
vga="-vga none" vga="-vga none"
log=""
kvm="" kvm=""
cpu="Broadwell,+pdpe1gb" cpu="Broadwell,+pdpe1gb"
smp=4
while true; do for arg in $@; do
case "$1" in case "${arg}" in
-b | --debugboot) --debugboot)
debug="-s -S" debug="-s -S"
debugtarget="${build}/boot/boot.efi" debugtarget="${build}/boot/boot.efi"
shift flash_name="ovmf_vars_d"
;; ;;
-d | --debug) --debug)
debug="-s -S" debug="-s -S"
isaexit="" flash_name="ovmf_vars_d"
shift
;; ;;
-g | --gfx) --gfx)
gfx="" gfx=""
vga="" vga=""
shift
;; ;;
-v | --vga) --vga)
vga="" vga=""
shift
;; ;;
-k | --kvm) --kvm)
kvm="-enable-kvm" kvm="-enable-kvm"
cpu="host" cpu="host"
shift
;;
-c | --cpus)
smp=$2
shift 2
;;
-l | --log)
log="-d mmu,int,guest_errors -D jsix.log"
shift
;; ;;
*) *)
if [ -d "$1" ]; then build="${arg}"
build="$1"
shift
fi
break
;; ;;
esac esac
done done
@@ -74,25 +56,25 @@ if [[ -n $TMUX ]]; then
fi fi
elif [[ $DESKTOP_SESSION = "i3" ]]; then elif [[ $DESKTOP_SESSION = "i3" ]]; then
if [[ -n $debug ]]; then if [[ -n $debug ]]; then
i3-msg exec i3-sensible-terminal -- -e "gdb ${debugtarget}" & i3-msg exec i3-sensible-terminal -- -e "gdb ${PWD}/${build}/jsix.elf" &
else else
i3-msg exec i3-sensible-terminal -- -e 'telnet localhost 45454' & i3-msg exec i3-sensible-terminal -- -e 'telnet localhost 45454' &
fi fi
fi fi
qemu-system-x86_64 \ exec qemu-system-x86_64 \
-drive "if=pflash,format=raw,readonly,file=${assets}/ovmf/x64/ovmf_code.fd" \ -drive "if=pflash,format=raw,readonly,file=${assets}/ovmf/x64/ovmf_code.fd" \
-drive "if=pflash,format=raw,file=${build}/ovmf_vars.fd" \ -drive "if=pflash,format=raw,file=${build}/${flash_name}.fd" \
-drive "format=raw,file=${build}/jsix.img" \ -drive "format=raw,file=${build}/jsix.img" \
-device "isa-debug-exit,iobase=0xf4,iosize=0x04" \
-monitor telnet:localhost:45454,server,nowait \ -monitor telnet:localhost:45454,server,nowait \
-serial stdio \ -serial stdio \
-serial telnet:localhost:45455,server,nowait \ -serial telnet:localhost:45455,server,nowait \
-smp "${smp}" \ -smp 4 \
-m 4096 \ -m 512 \
-d mmu,int,guest_errors \
-D jsix.log \
-cpu "${cpu}" \ -cpu "${cpu}" \
-M q35 \ -M q35 \
-no-reboot \ -no-reboot \
$isaexit $log $gfx $vga $kvm $debug $gfx $vga $kvm $debug
((result = ($? >> 1) - 1))
exit $result

5
requirements.txt
View File

@@ -1,5 +0,0 @@
cogapp >= 3
ninja >= 1.10.2
peru >= 1.2.1
pyyaml >= 5.4
lark == 0.12.0

25
scripts/bonnibel/__init__.py
View File

@@ -1,25 +0,0 @@
from os.path import join
class BonnibelError(Exception):
def __init__(self, message):
self.message = message
def load_config(filename):
from yaml import safe_load
with open(filename, 'r') as infile:
return safe_load(infile.read())
def mod_rel(path):
return join("${module_dir}", path)
def target_rel(path, module=None):
if module:
return join("${target_dir}", module + ".dir", path)
else:
return join("${target_dir}", path)
def include_rel(path, module=None):
if module:
return join("${build_root}", "include", module, path)
else:
return join("${build_root}", "include", path)

103
scripts/bonnibel/manifest.py
View File

@@ -1,103 +0,0 @@
from . import BonnibelError
class Manifest:
from collections import namedtuple
Entry = namedtuple("Entry", ("module", "target", "output", "location", "description", "flags"))
flags = {
"graphical": 0x01,
"panic": 0x02,
"symbols": 0x04,
}
boot_flags = {
"debug": 0x01,
"test": 0x02,
}
def __init__(self, path, modules):
from . import load_config
config = load_config(path)
self.kernel = self.__build_entry(modules,
config.get("kernel", dict()),
name="kernel", target="kernel")
self.init = self.__build_entry(modules,
config.get("init", None))
self.programs = [self.__build_entry(modules, i)
for i in config.get("programs", tuple())]
self.flags = config.get("flags", tuple())
self.data = []
for d in config.get("data", tuple()):
self.add_data(**d)
def __build_entry(self, modules, config, target="user", name=None):
flags = tuple()
if isinstance(config, str):
name = config
elif isinstance(config, dict):
name = config.get("name", name)
target = config.get("target", target)
flags = config.get("flags", tuple())
if isinstance(flags, str):
flags = flags.split()
mod = modules.get(name)
if not mod:
raise BonnibelError(f"Manifest specifies unknown module '{name}'")
for f in flags:
if not f in Manifest.flags:
raise BonnibelError(f"Manifest specifies unknown flag '{f}'")
return Manifest.Entry(name, target, mod.output, mod.location, mod.description, flags)
def add_data(self, output, location, desc, flags=tuple()):
e = Manifest.Entry(None, None, output, location, desc, flags)
self.data.append(e)
return e
def write_boot_config(self, path):
from os.path import join
import struct
with open(path, 'wb') as outfile:
magic = "jsixboot".encode("utf-8") # magic string
version = 0
reserved = 0
bootflags = sum([Manifest.boot_flags.get(s, 0) for s in self.flags])
outfile.write(struct.pack("<8sBBHHH",
magic, version, reserved,
len(self.programs), len(self.data),
bootflags))
def write_str(s):
outfile.write(struct.pack("<H", (len(s)+1)*2))
outfile.write(s.encode("utf-16le"))
outfile.write(b"\0\0")
def write_ent(ent):
flags = 0
for f in ent.flags:
flags |= Manifest.flags[f]
outfile.write(struct.pack("<H", flags))
write_str(join(ent.location, ent.output).replace('/','\\'))
write_str(ent.description)
write_ent(self.kernel)
write_ent(self.init)
for p in self.programs:
write_ent(p)
for d in self.data:
write_ent(d)

276
scripts/bonnibel/module.py
View File

@@ -1,276 +0,0 @@
from . import include_rel, mod_rel, target_rel
def resolve(path):
if path.startswith('/') or path.startswith('$'):
return path
from pathlib import Path
return str(Path(path).resolve())
class BuildOptions:
def __init__(self, includes=tuple(), libs=tuple(), order_only=tuple()):
self.includes = list(includes)
self.libs = list(libs)
self.order_only = list(order_only)
class Module:
__fields = {
# name: (type, default)
"kind": (str, "exe"),
"output": (str, None),
"targets": (set, ()),
"deps": (set, ()),
"public_headers": (set, ()),
"includes": (tuple, ()),
"sources": (tuple, ()),
"variables": (dict, ()),
"default": (bool, False),
"location": (str, "jsix"),
"description": (str, None),
"no_libc": (bool, False),
}
def __init__(self, name, modfile, root, **kwargs):
from .source import make_source
# Required fields
self.root = root
self.name = name
self.modfile = modfile
for name, data in self.__fields.items():
ctor, default = data
value = kwargs.get(name, default)
if value is not None:
value = ctor(value)
setattr(self, name, value)
for name in kwargs:
if not name in self.__fields:
raise AttributeError(f"No attribute named {name} on Module")
# Turn strings into real Source objects
self.sources = [make_source(root, f) for f in self.sources]
self.public_headers = [make_source(root, f) for f in self.public_headers]
# Filled by Module.update
self.depmods = set()
def __str__(self):
return "Module {} {}\n\t{}".format(self.kind, self.name, "\n\t".join(map(str, self.sources)))
@property
def output(self):
if self.__output is not None:
return self.__output
if self.kind == "lib":
return f"lib{self.name}.a"
else:
return f"{self.name}.elf"
@output.setter
def output(self, value):
self.__output = value
@classmethod
def update(cls, mods):
from . import BonnibelError
def resolve(source, modlist):
resolved = set()
for dep in modlist:
if not dep in mods:
raise BonnibelError(f"module '{source.name}' references unknown module '{dep}'")
mod = mods[dep]
resolved.add(mod)
return resolved
for mod in mods.values():
mod.depmods = resolve(mod, mod.deps)
target_mods = [mod for mod in mods.values() if mod.targets]
for mod in target_mods:
closed = set()
children = set(mod.depmods)
while children:
child = children.pop()
closed.add(child)
child.targets |= mod.targets
children |= {m for m in child.depmods if not m in closed}
def generate(self, output):
from pathlib import Path
from collections import defaultdict
from ninja.ninja_syntax import Writer
def walk_deps(deps):
open_set = set(deps)
closed_set = set()
while open_set:
dep = open_set.pop()
closed_set.add(dep)
open_set |= {m for m in dep.depmods if not m in closed_set}
return closed_set
all_deps = walk_deps(self.depmods)
def gather_phony(build, deps, child_rel, add_headers=False):
phony = ".headers.phony"
child_phony = [child_rel(phony, module=c.name)
for c in all_deps]
header_targets = []
if add_headers:
if not self.no_libc:
header_targets.append(f"${{build_root}}/include/libc/{phony}")
if self.public_headers:
header_targets.append(f"${{build_root}}/include/{self.name}/{phony}")
build.build(
rule = "touch",
outputs = [mod_rel(phony)],
implicit = child_phony + header_targets,
order_only = list(map(mod_rel, deps)),
)
filename = str(output / f"headers.{self.name}.ninja")
with open(filename, "w") as buildfile:
build = Writer(buildfile)
build.comment("This file is automatically generated by bonnibel")
build.newline()
build.variable("module_dir", f"${{build_root}}/include/{self.name}")
header_deps = []
inputs = []
headers = set(self.public_headers)
while headers:
source = headers.pop()
headers.update(source.next)
if source.action:
build.newline()
build.build(rule=source.action, **source.args)
if source.gather:
header_deps += list(source.outputs)
if source.input:
inputs.extend(map(mod_rel, source.outputs))
build.newline()
gather_phony(build, header_deps, include_rel)
filename = str(output / f"module.{self.name}.ninja")
with open(filename, "w") as buildfile:
build = Writer(buildfile)
build.comment("This file is automatically generated by bonnibel")
build.newline()
build.variable("module_dir", target_rel(self.name + ".dir"))
modopts = BuildOptions(
includes = [self.root, "${module_dir}"],
)
if self.public_headers:
modopts.includes += [f"${{build_root}}/include/{self.name}"]
for key, value in self.variables.items():
build.variable(key, value)
build.newline()
for include in self.includes:
p = Path(include)
if p.is_absolute():
if not p in modopts.includes:
modopts.includes.append(str(p.resolve()))
elif include != ".":
incpath = self.root / p
destpath = mod_rel(p)
for header in incpath.rglob("*.h"):
dest_header = f"{destpath}/" + str(header.relative_to(incpath))
modopts.includes.append(str(incpath))
modopts.includes.append(destpath)
all_deps = walk_deps(self.depmods)
for dep in all_deps:
if dep.public_headers:
modopts.includes += [f"${{build_root}}/include/{dep.name}"]
if dep.kind == "lib":
modopts.libs.append(target_rel(dep.output))
else:
modopts.order_only.append(target_rel(dep.output))
if modopts.includes:
build.variable("ccflags", ["${ccflags}"] + [f"-I{i}" for i in modopts.includes])
build.variable("asflags", ["${asflags}"] + [f"-I{i}" for i in modopts.includes])
if modopts.libs:
build.variable("libs", ["${libs}"] + modopts.libs)
header_deps = []
inputs = []
sources = set(self.sources)
while sources:
source = sources.pop()
sources.update(source.next)
if source.action:
build.newline()
build.build(rule=source.action, **source.args)
if source.gather:
header_deps += list(source.outputs)
if source.input:
inputs.extend(map(mod_rel, source.outputs))
build.newline()
gather_phony(build, header_deps, target_rel, add_headers=True)
output = target_rel(self.output)
dump = output + ".dump"
build.newline()
build.build(
rule = self.kind,
outputs = output,
inputs = inputs,
implicit = modopts.libs,
order_only = modopts.order_only,
)
build.newline()
build.build(
rule = "dump",
outputs = dump,
inputs = output,
variables = {"name": self.name},
)
if self.default:
build.newline()
build.default(output)
build.default(dump)
def add_input(self, path, **kwargs):
from .source import Source
s = Source(self.root, path, **kwargs)
self.sources.append(s)
return s.outputs
def add_depends(self, paths, deps):
for source in self.sources:
if source.path in paths:
source.add_deps(deps)
for source in self.public_headers:
if source.path in paths:
source.add_deps(deps)

234
scripts/bonnibel/project.py
View File

@@ -1,234 +0,0 @@
from . import BonnibelError
class Project:
def __init__(self, root):
from .version import git_version
self.root = root
self.version = git_version(root)
def __str__(self):
return f"{self.name} {self.version.major}.{self.version.minor}.{self.version.patch}-{self.version.sha}"
def generate(self, root, output, modules, config, manifest_file):
import sys
import bonnibel
from os.path import join
from ninja.ninja_syntax import Writer
from .target import Target
targets = set()
for mod in modules.values():
targets.update({Target.load(root, t, config) for t in mod.targets})
with open(output / "build.ninja", "w") as buildfile:
build = Writer(buildfile)
build.comment("This file is automatically generated by bonnibel")
build.variable("ninja_required_version", "1.3")
build.variable("build_root", output)
build.variable("source_root", root)
build.newline()
build.include(root / "configs" / "rules.ninja")
build.newline()
build.variable("version_major", self.version.major)
build.variable("version_minor", self.version.minor)
build.variable("version_patch", self.version.patch)
build.variable("version_sha", self.version.sha)
build.newline()
build.variable("cogflags", [
"-I", "${source_root}/scripts",
"-D", "definitions_path=${source_root}/definitions",
])
build.newline()
for target in targets:
build.subninja(output / target.name / "target.ninja")
build.newline()
for mod in modules.values():
build.subninja(output / f"headers.{mod.name}.ninja")
build.newline()
build.build(
rule = "touch",
outputs = "${build_root}/.all_headers",
implicit = [f"${{build_root}}/include/{m.name}/.headers.phony"
for m in modules.values() if m.public_headers],
)
build.build(
rule = "phony",
outputs = ["all-headers"],
inputs = ["${build_root}/.all_headers"])
debugroot = output / ".debug"
debugroot.mkdir(exist_ok=True)
fatroot = output / "fatroot"
fatroot.mkdir(exist_ok=True)
fatroot_content = []
def add_fatroot(source, entry):
output = join(entry.location, entry.output)
fatroot_output = f"${{build_root}}/fatroot/{output}"
build.build(
rule = "cp",
outputs = [fatroot_output],
inputs = [source],
variables = {
"name": f"Installing {output}",
})
fatroot_content.append(fatroot_output)
build.newline()
def add_fatroot_exe(entry):
input_path = f"${{build_root}}/{entry.target}/{entry.output}"
intermediary = f"${{build_root}}/{entry.output}"
build.build(
rule = "strip",
outputs = [intermediary],
inputs = [input_path],
implicit = [f"{input_path}.dump"],
variables = {
"name": f"Stripping {entry.module}",
"debug": f"${{build_root}}/.debug/{entry.output}.debug",
})
add_fatroot(intermediary, entry)
from .manifest import Manifest
manifest = Manifest(manifest_file, modules)
add_fatroot_exe(manifest.kernel)
add_fatroot_exe(manifest.init)
for program in manifest.programs:
add_fatroot_exe(program)
syms = manifest.add_data("symbol_table.dat",
manifest.kernel.location, "Symbol table", ("symbols",))
sym_out = f"${{build_root}}/symbol_table.dat"
build.build(
rule = "makest",
outputs = [sym_out],
inputs = [f"${{build_root}}/{modules['kernel'].output}"],
)
add_fatroot(sym_out, syms)
bootloader = "${build_root}/fatroot/efi/boot/bootx64.efi"
build.build(
rule = "cp",
outputs = [bootloader],
inputs = ["${build_root}/boot/boot.efi"],
variables = {
"name": "Installing bootloader",
})
build.newline()
boot_config = join(fatroot, "jsix_boot.dat")
manifest.write_boot_config(boot_config)
build.build(
rule = "makefat",
outputs = ["${build_root}/jsix.img"],
inputs = ["${source_root}/assets/diskbase.img"],
implicit = fatroot_content + [bootloader],
variables = {"name": "jsix.img"},
)
build.newline()
default_assets = {
"UEFI Variables": ("ovmf/x64/ovmf_vars.fd", "ovmf_vars.fd"),
"GDB Debug Helpers": ("debugging/jsix.elf-gdb.py", "jsix.elf-gdb.py"),
}
for name, assets in default_assets.items():
p = root / "assets" / assets[0]
out = f"${{build_root}}/{assets[1]}"
build.build(
rule = "cp",
outputs = [out],
inputs = [str(p)],
variables = {"name": name},
)
build.default([out])
build.newline()
compdb = "${source_root}/compile_commands.json"
build.rule("regen",
command = " ".join([str(root / 'configure')] + sys.argv[1:]),
description = "Regenerate build files",
generator = True,
)
build.newline()
regen_implicits = \
[f"{self.root}/configure", str(manifest_file)] + \
[str(mod.modfile) for mod in modules.values()]
for target in targets:
regen_implicits += target.depfiles
build.build(
rule = "compdb",
outputs = [compdb],
implicit = regen_implicits,
)
build.default([compdb])
build.newline()
build.build(
rule = "regen",
outputs = ['build.ninja'],
implicit = regen_implicits,
implicit_outputs =
[f"module.{mod.name}.ninja" for mod in modules.values()] +
[f"{target.name}/target.ninja" for target in targets] +
[boot_config],
)
build.newline()
build.default(["${build_root}/jsix.img"])
for target in targets:
mods = [m.name for m in modules.values() if target.name in m.targets]
targetdir = output / target.name
targetdir.mkdir(exist_ok=True)
buildfilename = str(targetdir / "target.ninja")
with open(buildfilename, "w") as buildfile:
build = Writer(buildfile)
build.comment("This file is automatically generated by bonnibel")
build.newline()
build.variable("target", target.name)
build.variable("target_dir", output / target.name)
build.newline()
for name, value in target.items():
build.variable(name, value)
build.newline()
for kind in ('defs', 'run'):
for lang in ('c', 'cpp'):
deffile = str(output / target.name / f"{lang}.{kind}")
build.build(
rule = f"dump_{lang}_{kind}",
outputs = [deffile],
implicit = [buildfilename],
)
build.default(deffile)
build.newline()
for mod in mods:
build.subninja(f"module.{mod}.ninja")

119
scripts/bonnibel/source.py
View File

@@ -1,119 +0,0 @@
from os.path import join, splitext
from . import mod_rel
def _resolve(path):
if path.startswith('/') or path.startswith('$'):
return path
from pathlib import Path
return str(Path(path).resolve())
def _dynamic_action(name):
def prop(self):
root, suffix = splitext(self.path)
return f"{name}{suffix}"
return prop
class Source:
next = tuple()
action = None
args = dict()
gather = False
outputs = tuple()
input = False
def __init__(self, path, root = "${module_dir}", deps=tuple()):
self.path = path
self.root = root
self.deps = deps
def add_deps(self, deps):
self.deps += tuple(deps)
@property
def fullpath(self):
return join(self.root, self.path)
class ParseSource(Source):
action = property(_dynamic_action("parse"))
@property
def output(self):
root, _ = splitext(self.path)
return root
@property
def outputs(self):
return (self.output,)
@property
def gather(self):
_, suffix = splitext(self.output)
return suffix in (".h", ".inc")
@property
def next(self):
_, suffix = splitext(self.output)
nextType = {
".s": CompileSource,
".cpp": CompileSource,
}.get(suffix)
if nextType:
return (nextType(self.output),)
return tuple()
@property
def args(self):
return dict(
outputs = list(map(mod_rel, self.outputs)),
inputs = [self.fullpath],
implicit = list(map(_resolve, self.deps)),
variables = dict(name=self.path),
)
class HeaderSource(Source):
action = "cp"
gather = True
@property
def outputs(self):
return (self.path,)
@property
def args(self):
return dict(
outputs = [mod_rel(self.path)],
inputs = [join(self.root, self.path)],
implicit = list(map(_resolve, self.deps)),
variables = dict(name=self.path),
)
class CompileSource(Source):
action = property(_dynamic_action("compile"))
input = True
@property
def outputs(self):
return (self.path + ".o",)
@property
def args(self):
return dict(
outputs = list(map(mod_rel, self.outputs)),
inputs = [join(self.root, self.path)],
implicit = list(map(_resolve, self.deps)) + [mod_rel(".headers.phony")],
variables = dict(name=self.path),
)
def make_source(root, path):
_, suffix = splitext(path)
if suffix in (".s", ".c", ".cpp"):
return CompileSource(path, root)
elif suffix in (".cog",):
return ParseSource(path, root)
elif suffix in (".h", ".inc"):
return HeaderSource(path, root)
else:
raise RuntimeError(f"{path} has no Source type")

50
scripts/bonnibel/target.py
View File

@@ -1,50 +0,0 @@
class Target(dict):
__targets = {}
@classmethod
def load(cls, root, name, config=None):
from . import load_config
if (name, config) in cls.__targets:
return cls.__targets[(name, config)]
configs = root / "configs"
dicts = []
depfiles = []
basename = name
if config:
basename += f"-{config}"
while basename is not None:
filename = str(configs / (basename + ".yaml"))
depfiles.append(filename)
desc = load_config(filename)
basename = desc.get("extends")
dicts.append(desc.get("variables", dict()))
t = Target(name, config, depfiles)
for d in reversed(dicts):
for k, v in d.items():
if isinstance(v, (list, tuple)):
t[k] = t.get(k, list()) + list(v)
elif isinstance(v, dict):
t[k] = t.get(k, dict())
t[k].update(v)
else:
t[k] = v
cls.__targets[(name, config)] = t
return t
def __init__(self, name, config, depfiles):
self.__name = name
self.__config = config
self.__depfiles = tuple(depfiles)
def __hash__(self):
return hash((self.__name, self.__config))
name = property(lambda self: self.__name)
config = property(lambda self: self.__config)
depfiles = property(lambda self: self.__depfiles)

41
scripts/bonnibel/version.py
View File

@@ -1,41 +0,0 @@
from collections import namedtuple as _namedtuple
version = _namedtuple('version', [
'major',
'minor',
'patch',
'sha',
'dirty',
])
def _run_git(root, *args):
from subprocess import run
git = run(['git', '-C', root] + list(args),
check=True, capture_output=True)
return git.stdout.decode('utf-8').strip()
def git_version(root):
full_version = _run_git(root, 'describe', '--dirty')
full_sha = _run_git(root, 'rev-parse', 'HEAD')
dirty = False
parts1 = full_version.split('-')
if parts1[-1] == "dirty":
dirty = True
parts1 = parts1[:-1]
if parts1[0][0] == 'v':
parts1[0] = parts1[0][1:]
parts2 = parts1[0].split('.')
return version(
parts2[0],
parts2[1],
parts2[2],
full_sha[:7],
dirty)

43
scripts/build_symbol_table.py
View File

@@ -4,37 +4,37 @@
# is as follows: # is as follows:
# #
# <num_entires> : 8 bytes # <num_entires> : 8 bytes
# <index> : 24 * N bytes # <index> : 16 * N bytes
# <name data> : variable # <name data> : variable
# #
# Each index entry has the format # Each index entry has the format
# <symbol address> : 8 bytes # <symbol address> : 8 bytes
# <symbol size> : 8 bytes
# <offset of name> : 8 bytes # <offset of name> : 8 bytes
# #
# Name offsets are from the start of the symbol table as a whole. (ie, # Name offsets are from the start of the symbol table as a whole. (ie,
# where <num_entries> is located.) # where <num_entries> is located.)
import re
sym_re = re.compile(r'([0-9a-fA-F]{16}) ([0-9a-fA-F]{16}) [tTvVwW] (.*)')
def parse_syms(infile): def parse_syms(infile):
"""Take the output of the `nm` command, and parse it into a tuple """Take the output of the `nm` command, and parse it into a tuple
representing the symbols in the text segment of the binary. Returns representing the symbols in the text segment of the binary. Returns
a list of (address, symbol_name).""" a list of (address, symbol_name)."""
from cxxfilt import demangle, InvalidName
syms = [] syms = []
for line in sys.stdin: for line in sys.stdin:
match = sym_re.match(line) addr, t, mangled = line.split()
if not match: continue if t not in "tTvVwW": continue
addr = int(match.group(1), base=16) try:
size = int(match.group(2), base=16) name = demangle(mangled)
name = match.group(3) except InvalidName:
syms.append([addr, size, name, 0]) continue
return syms addr = int(addr, base=16)
syms.append((addr, name))
return sorted(syms)
def write_table(syms, outfile): def write_table(syms, outfile):
@@ -46,26 +46,29 @@ def write_table(syms, outfile):
outfile.write(struct.pack("@Q", len(syms))) outfile.write(struct.pack("@Q", len(syms)))
index_pos = outfile.tell() index_pos = outfile.tell()
outfile.seek(struct.calcsize("@QQQ") * len(syms), 1) outfile.seek(struct.calcsize("@QQ") * len(syms), 1)
nul = b'\0' nul = b'\0'
positions = {}
for s in syms: for s in syms:
s[3] = outfile.tell() addr, name = s
outfile.write(s[2].encode('utf-8')) positions[addr] = outfile.tell()
data = name.encode('utf-8')
outfile.write(name.encode('utf-8'))
outfile.write(nul) outfile.write(nul)
outfile.seek(index_pos) outfile.seek(index_pos)
for s in syms: for s in syms:
addr = s[0] addr = s[0]
size = s[1] pos = positions[addr]
pos = s[3] outfile.write(struct.pack("@QQ", addr, pos))
outfile.write(struct.pack("@QQQ", addr, size, pos))
if __name__ == "__main__": if __name__ == "__main__":
import sys import sys
if len(sys.argv) != 2: if len(sys.argv) != 2:
print(f"Usage: nm -n -S --demangle | {sys.argv[0]} <output>") print(f"Usage: {sys.argv[0]} <output>")
sys.exit(1) sys.exit(1)
outfile = open(sys.argv[1], "wb") outfile = open(sys.argv[1], "wb")

131
scripts/build_sysroot.sh Executable file
View File

@@ -0,0 +1,131 @@
#!/usr/bin/env bash
TARGET="x86_64-elf"
LLVM_BRANCH="release_80"
TOOLS="clang lld" # lld libunwind libcxxabi libcxx"
PROJECTS="compiler-rt libcxxabi libcxx libunwind"
#RUNTIMES="compiler-rt libcxxabi libcxx libunwind"
set -e
README=$(realpath "$(dirname $0)/readme_for_prebuilt_sysroots.md")
SYSROOT=$(realpath "$(dirname $0)/../sysroot")
WORK=$(realpath "$(dirname $0)/sysroot")
mkdir -p "${SYSROOT}"
mkdir -p "${WORK}"
export CC=clang
export CXX=clang++
if [[ ! -d "${WORK}/llvm" ]]; then
echo "Downloading LLVM..."
git clone -q \
--branch "${LLVM_BRANCH}" \
--depth 1 \
"https://git.llvm.org/git/llvm.git" "${WORK}/llvm"
fi
for tool in ${TOOLS}; do
if [[ ! -d "${WORK}/llvm/tools/${tool}" ]]; then
echo "Downloading ${tool}..."
git clone -q \
--branch "${LLVM_BRANCH}" \
--depth 1 \
"https://git.llvm.org/git/${tool}.git" "${WORK}/llvm/tools/${tool}"
fi
done
if [[ ! -d "${WORK}/llvm/tools/clang/tools/extra" ]]; then
echo "Downloading clang-tools-extra..."
git clone -q \
--branch "${LLVM_BRANCH}" \
--depth 1 \
"https://git.llvm.org/git/clang-tools-extra.git" "${WORK}/llvm/tools/clang/tools/extra"
fi
for proj in ${PROJECTS}; do
if [[ ! -d "${WORK}/llvm/projects/${proj}" ]]; then
echo "Downloading ${proj}..."
git clone -q \
--branch "${LLVM_BRANCH}" \
--depth 1 \
"https://git.llvm.org/git/${proj}.git" "${WORK}/llvm/projects/${proj}"
fi
done
for proj in ${RUNTIMES}; do
if [[ ! -d "${WORK}/llvm/runtimes/${proj}" ]]; then
echo "Downloading ${proj}..."
git clone -q \
--branch "${LLVM_BRANCH}" \
--depth 1 \
"https://git.llvm.org/git/${proj}.git" "${WORK}/llvm/runtime/${proj}"
fi
done
mkdir -p "${WORK}/build/llvm"
pushd "${WORK}/build/llvm"
echo "Configuring LLVM..."
cmake -G Ninja \
-DCLANG_DEFAULT_RTLIB=compiler-rt \
-DCLANG_DEFAULT_STD_C=c11 \
-DCLANG_DEFAULT_STD_CXX=cxx14 \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER="clang" \
-DCMAKE_CXX_COMPILER="clang++" \
-DCMAKE_CXX_FLAGS="-Wno-unused-parameter -D_LIBCPP_HAS_NO_ALIGNED_ALLOCATION -D_LIBUNWIND_IS_BAREMETAL=1 -U_LIBUNWIND_SUPPORT_DWARF_UNWIND" \
-DCMAKE_INSTALL_PREFIX="${SYSROOT}" \
-DCMAKE_MAKE_PROGRAM=`which ninja` \
-DDEFAULT_SYSROOT="${SYSROOT}" \
-DLIBCXX_CXX_ABI=libcxxabi \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS="${WORK}/llvm/projects/libcxxabi/include" \
-DLIBCXX_CXX_ABI_LIBRARY_PATH=lib \
-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=OFF \
-DLIBCXX_ENABLE_NEW_DELETE_DEFINITIONS=ON \
-DLIBCXX_ENABLE_SHARED=OFF \
-DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=ON \
-DLIBCXX_ENABLE_THREADS=OFF \
-DLIBCXX_INCLUDE_BENCHMARKS=OFF \
-DLIBCXX_USE_COMPILER_RT=ON \
-DLIBCXXABI_ENABLE_NEW_DELETE_DEFINITIONS=OFF \
-DLIBCXXABI_ENABLE_SHARED=OFF \
-DLIBCXXABI_ENABLE_STATIC_UNWINDER=ON \
-DLIBCXXABI_ENABLE_THREADS=OFF \
-DLIBCXXABI_LIBCXX_PATH="${WORK}/llvm/projects/libcxx" \
-DLIBCXXABI_USE_COMPILER_RT=ON \
-DLIBCXXABI_USE_LLVM_UNWINDER=ON \
-DLIBUNWIND_ENABLE_SHARED=OFF \
-DLIBUNWIND_ENABLE_THREADS=OFF \
-DLIBUNWIND_USE_COMPILER_RT=ON \
-DLLVM_CONFIG_PATH="${SYSROOT}/bin/llvm-config" \
-DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-unknown-elf" \
-DLLVM_ENABLE_LIBCXX=ON \
-DLLVM_ENABLE_LLD=ON \
-DLLVM_ENABLE_PIC=OFF \
-DLLVM_ENABLE_THREADS=OFF \
-DLLVM_INSTALL_BINUTILS_SYMLINKS=ON \
-DLLVM_TARGETS_TO_BUILD="X86" \
${WORK}/llvm > cmake_configure.log
# -DCMAKE_ASM_COMPILER=nasm \
# -DCMAKE_LINKER="${SYSROOT}/bin/ld.lld" \
# -DCOMPILER_RT_ENABLE_LLD=ON \
# -DLIBCXX_ENABLE_LLD=ON \
# -DLIBCXX_ENABLE_STATIC_UNWINDER=ON \
# -DLIBCXXABI_ENABLE_LLD=ON \
# -DLIBUNWIND_ENABLE_LLD=ON \
# -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi;libunwind;compiler-rt" \
# -DCOMPILER_RT_BAREMETAL_BUILD=ON \
# -DLIBCXXABI_BAREMETAL=ON \
echo "Building LLVM..."
ninja && ninja install
ninja cxx cxxabi compiler-rt
ninja install-compiler-rt install-cxx install-cxxabi
popd
cp "${README}" "${SYSROOT}/README.md"

0
scripts/definitions/__init__.py
View File

83
scripts/definitions/context.py
View File

@@ -1,83 +0,0 @@
class NotFound(Exception): pass
class Context:
def __init__(self, path, verbose=False):
if isinstance(path, str):
self.__paths = [path]
else:
self.__paths = path
self.__closed = set()
self.__verbose = verbose
self.__deps = {}
self.objects = dict()
self.interfaces = dict()
verbose = property(lambda self: self.__verbose)
def find(self, filename):
from os.path import exists, isabs, join
if exists(filename) or isabs(filename):
return filename
for path in self.__paths:
full = join(path, filename)
if exists(full):
return full
raise NotFound(filename)
def parse(self, filename):
pending = set()
pending.add(filename)
from .parser import LarkError
from .parser import Lark_StandAlone as Parser
from .transformer import DefTransformer
objects = {}
interfaces = {}
while pending:
name = pending.pop()
self.__closed.add(name)
path = self.find(name)
parser = Parser(transformer=DefTransformer(name))
try:
imps, objs, ints = parser.parse(open(path, "r").read())
except LarkError as e:
import sys
import textwrap
print(f"\nError parsing {name}:", file=sys.stderr)
print(textwrap.indent(str(e), " "), file=sys.stderr)
sys.exit(1)
objects.update(objs)
interfaces.update(ints)
self.__deps[name] = imps
pending.update(imps.difference(self.__closed))
from .types import ObjectRef
ObjectRef.connect(objects)
for obj in objects.values():
for method in obj.methods:
caps = method.options.get("cap", list())
for cap in caps:
if not cap in obj.caps:
from .errors import UnknownCapError
raise UnknownCapError(f"Unknown capability {cap} on {obj.name}::{method.name}")
self.objects.update(objects)
self.interfaces.update(interfaces)
def deps(self):
return {self.find(k): tuple(map(self.find, v)) for k, v in self.__deps.items()}

3
scripts/definitions/errors.py
View File

@@ -1,3 +0,0 @@
class InvalidType(Exception): pass
class UnknownTypeError(Exception): pass
class UnknownCapError(Exception): pass

2905
scripts/definitions/parser.py
View File

File diff suppressed because one or more lines are too long

23
scripts/definitions/templates/syscall/client/private/syscall_trampolines.s
View File

@@ -1,23 +0,0 @@
%macro SYSCALL 2
global __syscall_%1
__syscall_%1:
push rbp
mov rbp, rsp
; args should already be in rdi, etc, but rcx will
; get stomped, so stash it in r10, which isn't a
; callee-saved register, but also isn't used in the
; function call ABI.
mov r10, rcx
mov rax, %2
syscall
; result is now already in rax, so just return
pop rbp
ret
%endmacro
{% for id, scope, method in interface.methods %}
SYSCALL {% if scope %}{{ scope.name }}_{% endif %}{{ method.name }} {{ id }}
{% endfor %}

41
scripts/definitions/templates/syscall/client/public/_object_.hh
View File

@@ -1,41 +0,0 @@
#pragma once
/// \file {{filename}}
{% if object.super %}
#include <j6/{{ object.super.name }}.hh>
{% endif %}
namespace j6 {
{% if object.desc %}
{{ object.desc|indent('/// ', first=True) }}
{% endif %}
class {{ object.name }}
{% if object.super %} : public {{ object.super.name }}
{% endif %}
{
public:
{% macro argument(type, name, first, options=False) -%}
{%- for ctype, suffix in type.c_names(options) -%}
{%- if not first or not loop.first %}, {% endif %}{{ ctype }} {{ name }}{{ suffix }}
{%- endfor -%}
{%- endmacro -%}
{% for method in object.methods %}
{% if method.desc %} /// {{ method.desc|indent(' /// ') }}{% endif %}
{% for param in method.params %}
{% if param.desc %} /// \arg {{ "%-10s"|format(param.name) }} {{ param.desc }}
{% endif %}
{% endfor %}
{%+ if method.static %}static {% endif %}j6_status_t {{ method.name }} (
{%- for param in method.params %}{{ argument(param.type, param.name, loop.first, options=param.options) }}{% endfor -%});
{% endfor %}
~{{ object.name }}();
private:
{{ object.name }}(j6_handle_t handle) : m_handle {handle} {}
j6_handle_t m_handle;
}
} // namespace j6

25
scripts/definitions/templates/syscall/client/public/syscalls.h
View File

@@ -1,25 +0,0 @@
#pragma once
#include <j6/types.h>
#ifdef __cplusplus
extern "C" {
#endif
{% macro argument(type, name, first, options=False) -%}
{%- for ctype, suffix in type.c_names(options) -%}
{%- if not first or not loop.first %}, {% endif %}{{ ctype }} {{ name }}{{ suffix }}
{%- endfor -%}
{%- endmacro %}
{% for id, scope, method in interface.methods %}
j6_status_t __syscall_{% if scope %}{{ scope.name }}_{% endif %}{{ method.name }} (
{%- if not method.static -%}{{ argument(scope.reftype, "self", True) }}{% endif -%}
{%- set first = method.static -%}
{%- for param in method.params %}{{ argument(param.type, param.name, first, options=param.options) }}{% set first = False %}{% endfor -%}
);
{% endfor %}
#ifdef __cplusplus
}
#endif

156
scripts/definitions/transformer.py
View File

@@ -1,156 +0,0 @@
from .parser import Transformer, v_args
def get_opts(args):
from .types import Caps, CName, Description, Options, Type, UID
kinds = {
Description: "desc",
Options: "opts",
CName: "cname",
Caps: "caps",
UID: "uid",
Type: "typename",
}
result = dict()
outargs = []
for a in args:
for kind, name in kinds.items():
if isinstance(a, kind):
result[name] = a
break
else:
outargs.append(a)
return result, outargs
class DefTransformer(Transformer):
def __init__(self, filename):
self.filename = filename
def start(self, args):
from .types import Import, Interface, Object
imports = set()
objects = dict()
interfaces = dict()
for o in args:
if isinstance(o, Object):
objects[o.name] = o
elif isinstance(o, Interface):
interfaces[o.name] = o
elif isinstance(o, Import):
imports.add(o)
return imports, objects, interfaces
@v_args(inline=True)
def import_statement(self, path):
from .types import Import
return Import(path)
def object(self, args):
from .types import Object
specials, args = get_opts(args)
name, args = args[0], args[1:]
return Object(name, children=args, **specials)
def interface(self, args):
from .types import Interface
specials, args = get_opts(args)
name, args = args[0], args[1:]
return Interface(name, children=args, **specials)
def method(self, args):
from .types import Method
specials, args = get_opts(args)
name, args = args[0], args[1:]
return Method(name, children=args, **specials)
def function(self, args):
from .types import Function
specials, args = get_opts(args)
name, args = args[0], args[1:]
return Function(name, children=args, **specials)
def param(self, args):
from .types import Param
specials, args = get_opts(args)
name = args[0]
return Param(name, **specials)
@v_args(inline=True)
def expose(self, s):
from .types import Expose
return Expose(s)
@v_args(inline=True)
def uid(self, s):
return s
@v_args(inline=True)
def cname(self, s):
from .types import CName
return CName(s)
@v_args(inline=True)
def name(self, s):
return s
@v_args(inline=True)
def type(self, s):
return s
@v_args(inline=True)
def super(self, s):
from .types import ObjectRef
return ObjectRef(s, self.filename)
def options(self, args):
from .types import Options
return Options([str(s) for s in args])
def capabilities(self, args):
from .types import Caps
return Caps([str(s) for s in args])
def description(self, s):
from .types import Description
return Description("\n".join(s))
@v_args(inline=True)
def object_name(self, n):
from .types import ObjectRef
return ObjectRef(n, self.filename)
def PRIMITIVE(self, s):
from .types import get_primitive
return get_primitive(s)
def UID(self, s):
from .types import UID
return UID(int(s, base=16))
def INT_TYPE(self, s):
return s
def NUMBER(self, s):
if s.startswith("0x"):
return int(s,16)
return int(s)
def COMMENT(self, s):
return s[2:].strip()
def OPTION(self, s):
return str(s)
def IDENTIFIER(self, s):
return str(s)
def PATH(self, s):
return str(s[1:-1])

26
scripts/definitions/types/__init__.py
View File

@@ -1,26 +0,0 @@
def _indent(x):
from textwrap import indent
return indent(str(x), ' ')
class CName(str): pass
class Description(str): pass
class Import(str): pass
class Caps(list): pass
class Options(dict):
def __init__(self, opts = tuple()):
for opt in opts:
parts = opt.split(":", 1)
self[parts[0]] = self.get(parts[0], []) + ["".join(parts[1:])]
class UID(int):
def __str__(self):
return f"{self:016x}"
from .object import Object
from .interface import Interface, Expose
from .function import Function, Method, Param
from .type import Type
from .primitive import get_primitive
from .objref import ObjectRef

68
scripts/definitions/types/function.py
View File

@@ -1,68 +0,0 @@
from . import _indent
from . import Options
def _hasopt(opt):
def test(self):
return opt in self.options
return test
class Function:
typename = "function"
def __init__(self, name, opts=Options(), desc="", children=tuple()):
self.name = name
self.options = opts
self.desc = desc
self.params = [c for c in children if isinstance(c, Param)]
self.id = -1
def __str__(self):
parts = ["{} {}".format(self.typename, self.name)]
if self.desc:
parts.append(_indent(self.desc))
if self.options:
parts.append(f" Options: {self.options}")
parts.extend(map(_indent, self.params))
return "\n".join(parts)
static = property(lambda x: True)
constructor = property(lambda x: False)
class Method(Function):
typename = "method"
static = property(_hasopt("static"))
constructor = property(_hasopt("constructor"))
class Param:
def __init__(self, name, typename, opts=Options(), desc=""):
self.name = name
self.type = typename
self.options = opts
self.desc = desc
self.caps = set()
for key, values in opts.items():
if key == "cap":
self.caps.update(values)
def __str__(self):
return "param {} {} {} {}".format(
self.name, repr(self.type), self.options, self.desc or "")
@property
def outparam(self):
return "out" in self.options or "inout" in self.options
@property
def refparam(self):
return self.type.reference or self.outparam
@property
def optional(self):
if "optional" in self.options: return "optional"
elif "zero_ok" in self.options: return "zero_ok"
else: return "required"

46
scripts/definitions/types/interface.py
View File

@@ -1,46 +0,0 @@
from . import _indent
from . import Options
class Expose(object):
def __init__(self, type):
self.type = type
def __repr__(self):
return f'expose {repr(self.type)}'
class Interface:
def __init__(self, name, uid, opts=Options(), desc="", children=tuple()):
from .function import Function
self.name = name
self.uid = uid
self.options = opts
self.desc = desc
self.functions = [c for c in children if isinstance(c, Function)]
self.__exposes = [e.type for e in children if isinstance(e, Expose)]
def __str__(self):
parts = [f"interface {self.name}: {self.uid}"]
if self.desc:
parts.append(_indent(self.desc))
if self.options:
parts.append(f" Options: {self.options}")
parts.extend(map(_indent, self.exposes))
parts.extend(map(_indent, self.functions))
return "\n".join(parts)
@property
def methods(self):
mm = [(i, None, self.functions[i]) for i in range(len(self.functions))]
base = len(mm)
for o in self.exposes:
mm.extend([(base + i, o, o.methods[i]) for i in range(len(o.methods))])
base += len(o.methods)
return mm
@property
def exposes(self):
return [ref.object for ref in self.__exposes]

33
scripts/definitions/types/object.py
View File

@@ -1,33 +0,0 @@
from . import _indent
from . import Caps, Options
class Object:
def __init__(self, name, uid, typename=None, opts=Options(), caps=Caps(), desc="", children=tuple(), cname=None):
self.name = name
self.uid = uid
self.options = opts
self.desc = desc
self.methods = children
self.cname = cname or name
self.caps = caps
self.__super = typename
from . import ObjectRef
self.__ref = ObjectRef(name)
def __str__(self):
parts = [f"object {self.name}: {self.uid}"]
if self.desc:
parts.append(_indent(self.desc))
if self.options:
parts.append(f" Options: {self.options}")
parts.extend(map(_indent, self.methods))
return "\n".join(parts)
reftype = property(lambda self: self.__ref)
@property
def super(self):
if self.__super is not None:
return self.__super.object

49
scripts/definitions/types/objref.py
View File

@@ -1,49 +0,0 @@
from .type import Type
class ObjectRef(Type):
all_refs = {}
def __init__(self, name, filename=None):
super().__init__(name)
self.__c_type = "j6_handle_t"
self.__object = None
ObjectRef.all_refs[self] = filename
def __repr__(self):
return f'ObjectRef({self.name})'
object = property(lambda self: self.__object)
def c_names(self, options):
one = self.__c_type
out = bool({"out", "inout"}.intersection(options))
if "list" in options:
two = "size_t"
one = f"{one} *"
if out:
two += " *"
return ((one, ""), (two, "_count"))
else:
if out:
one += " *"
return ((one, ""),)
def cxx_names(self, options):
if not self.needs_object(options):
return self.c_names(options)
return ((f"obj::{self.name} *", ""),)
def needs_object(self, options):
return not bool({"out", "inout", "list", "handle"}.intersection(options))
@classmethod
def connect(cls, objects):
for ref, filename in cls.all_refs.items():
ref.__object = objects.get(ref.name)
if ref.__object is None:
from ..errors import UnknownTypeError
raise UnknownTypeError(ref.name, filename)

72
scripts/definitions/types/primitive.py
View File

@@ -1,72 +0,0 @@
from .type import Type
class Primitive(Type):
def __init__(self, name, c_type):
super().__init__(name)
self.c_type = c_type
def __repr__(self):
return f'Primitive({self.name})'
def c_names(self, options=dict()):
one = self.c_type
if "out" in options or "inout" in options:
one += " *"
return ((one, ""),)
def cxx_names(self, options):
return self.c_names(options)
class PrimitiveRef(Primitive):
def __init__(self, name, c_type, counted=False):
super().__init__(name, c_type)
self.__counted = counted
reference = property(lambda self: True)
def c_names(self, options=dict()):
one = f"{self.c_type} *"
two = "size_t"
if "out" in options or "inout" in options:
two += " *"
else:
one = "const " + one
if self.__counted:
return ((one, ""), (two, "_len"))
else:
return ((one, ""),)
def cxx_names(self, options):
return self.c_names(options)
_primitives = {
"string": PrimitiveRef("string", "char"),
"buffer": PrimitiveRef("buffer", "void", counted=True),
"int": Primitive("int", "int"),
"uint": Primitive("uint", "unsigned"),
"size": Primitive("size", "size_t"),
"address": Primitive("address", "uintptr_t"),
"int8": Primitive("int8", "int8_t"),
"uint8": Primitive("uint8", "uint8_t"),
"int16": Primitive("int16", "int16_t"),
"uint16": Primitive("uint16", "uint16_t"),
"int32": Primitive("int32", "int32_t"),
"uint32": Primitive("uint32", "uint32_t"),
"int64": Primitive("int64", "int64_t"),
"uint64": Primitive("uint64", "uint64_t"),
}
def get_primitive(name):
p = _primitives.get(name)
if not p:
from ..errors import InvalidType
raise InvalidType(name)
return p

15
scripts/definitions/types/type.py
View File

@@ -1,15 +0,0 @@
class Type:
def __init__(self, name):
self.__name = name
name = property(lambda self: self.__name)
reference = property(lambda self: False)
def c_names(self, options):
raise NotImplemented("Call to base Type.c_names")
def cxx_names(self, options):
raise NotImplemented("Call to base Type.c_names")
def needs_object(self, options):
return False

11
scripts/idgen
View File

@@ -1,11 +0,0 @@
#!/usr/bin/env python3
def hashid(s):
from hashlib import shake_128 as sh
return sh(s.encode('utf-8')).hexdigest(8)
import sys
for arg in sys.argv[1:]:
id = hashid(arg)
print(f"{arg}: {id}")

56
scripts/j6libc.py
View File

@@ -1,56 +0,0 @@
import cog
supported_architectures = {
"amd64": "__amd64__",
}
def arch_includes(header):
prefix = "if"
for arch, define in supported_architectures.items():
cog.outl(f"#{prefix} defined({define})")
cog.outl(f"#include <__j6libc/arch/{arch}/{header}>")
prefix = "elif"
cog.outl("#endif")
int_widths = (8, 16, 32, 64)
int_mods = ("fast", "least")
int_types = {
# type: abbrev
"char": "char",
"short": "short",
"int": "int",
"long": "long",
"long long": "llong",
}
def definition(kind, name, val, width=24):
cog.outl(f"{kind} {name:{width}} {val}")
atomic_types = {
"_Bool": "bool",
"signed char": "schar",
"char16_t": "char16_t",
"char32_t": "char32_t",
"wchar_t": "wchar_t",
"wchar_t": "wchar_t",
"size_t": "size_t",
"ptrdiff_t": "ptrdiff_t",
"intptr_t": "intptr_t",
"uintptr_t": "uintptr_t",
"intmax_t": "intmax_t",
"uintmax_t": "uintmax_t",
}
for name, abbrev in int_types.items():
atomic_types.update({name: abbrev, f"unsigned {name}": f"u{abbrev}"})
for width in int_widths:
atomic_types.update({t: t for t in (
f"int_least{width}_t",
f"uint_least{width}_t",
f"int_fast{width}_t",
f"uint_fast{width}_t")})

32
scripts/memory.py
View File

@@ -1,32 +0,0 @@
class Layout:
from collections import namedtuple
Region = namedtuple("Region", ("name", "start", "size", "shared"))
sizes = {'G': 1024 ** 3, 'T': 1024 ** 4}
@staticmethod
def get_size(desc):
size, mag = int(desc[:-1]), desc[-1]
try:
mult = Layout.sizes[mag]
except KeyError:
raise RuntimeError(f"No magnitude named '{mag}'.")
return size * mult
def __init__(self, path):
from yaml import safe_load
regions = []
addr = 1 << 64
with open(path, 'r') as infile:
data = safe_load(infile.read())
for r in data:
size = Layout.get_size(r["size"])
addr -= size
regions.append(Layout.Region(r["name"], addr, size,
r.get("shared", False)))
self.regions = tuple(regions)

10
scripts/parse_syms.py Executable file
View File

@@ -0,0 +1,10 @@
#!/usr/bin/env python3
def parse_elf(filename):
import struct
with open(filename, 'rb') as elf:
if __name__ == "__main__":
import sys
for arg in sys.argv[1:]:
parse_elf(arg)

27
scripts/psf_to_cpp.py Executable file
View File

@@ -0,0 +1,27 @@
#!/usr/bin/env python3
from fontpsf import PSF2
def print_header(filename):
font = PSF2.load(filename)
print("#pragma once")
print(f"// This file was autogenerated by psf_to_cpp.py from {font.filename}\n")
print(f"const uint8_t font_glyph_size = {font.charsize};")
print(f"const uint8_t font_glyph_width = {font.dimension[0]};")
print(f"const uint8_t font_glyph_height = {font.dimension[1]};")
print(f"const uint16_t font_glyph_count = {font.count};\n")
print('const uint8_t font_glyph_data[] = {')
for glyph in font:
print(" ", "".join([f"0x{b:02x}," for b in glyph.data]), end="")
print(" // {}".format(glyph.description()))
print("};")
if __name__ == "__main__":
import sys
for filename in sys.argv[1:]:
print_header(filename)

14
scripts/readme_for_prebuilt_sysroots.md Normal file
View File

@@ -0,0 +1,14 @@
# jsix OS sysroot
This is a pre-built sysroot for building the jsix operating system kernel,
bootloader, and utilities. This package is provided as a convenience, and
contains software from the following repositories.
## The LLVM toolchain
The LLVM sources as downloaded via git from [llvm.org][llvm] under the terms of
the [Apache License v2.0][apache2], modified [as described here][llvmlic].
[llvm]: https://llvm.org
[apache2]: https://www.apache.org/licenses/LICENSE-2.0
[llvmlic]: https://llvm.org/docs/DeveloperPolicy.html#new-llvm-project-license-framework

17
scripts/sysconf.py
View File

@@ -1,17 +0,0 @@
class Sysconf:
from collections import namedtuple
Var = namedtuple("Var", ("name", "section", "type"))
def __init__(self, path):
from yaml import safe_load
sys_vars = []
with open(path, 'r') as infile:
data = safe_load(infile.read())
self.address = data["address"]
for v in data["vars"]:
sys_vars.append(Sysconf.Var(v["name"], v["section"], v["type"]))
self.vars = tuple(sys_vars)

214
scripts/templates/build.ninja.j2 Normal file
View File

@@ -0,0 +1,214 @@
ninja_required_version = 1.3
builddir = {{ buildroot }}
srcroot = {{ srcroot }}
modulefile = {{ modulefile }}
{%- for var, value in vars %}
{{ var }} = {{ value }}
{%- endfor %}
warnflags = $
-Wformat=2 $
-Winit-self $
-Wfloat-equal $
-Winline $
-Wmissing-format-attribute $
-Wmissing-include-dirs $
-Wswitch $
-Wundef $
-Wdisabled-optimization $
-Wpointer-arith $
-Wno-attributes $
-Wno-sign-compare $
-Wno-multichar $
-Wno-div-by-zero $
-Wno-endif-labels $
-Wno-pragmas $
-Wno-format-extra-args $
-Wno-unused-result $
-Wno-deprecated-declarations $
-Wno-unused-function $
-Wno-address-of-packed-member $
-Werror
ccflags = $
-I${srcroot}/src/include $
-I${srcroot}/src/include/x86_64 $
-fcolor-diagnostics $
-DVERSION_MAJOR={{ version_major }} $
-DVERSION_MINOR={{ version_minor }} $
-DVERSION_PATCH={{ version_patch }} $
-DVERSION_GITSHA=0x0{{ version_sha }} $
-DGIT_VERSION=\"{{ version }}\" $
-DGIT_VERSION_WIDE=L\"{{ version }}\" $
$warnflags
asflags = $
-DVERSION_MAJOR={{ version_major }} $
-DVERSION_MINOR={{ version_minor }} $
-DVERSION_PATCH={{ version_patch }} $
-DVERSION_GITSHA=0x{{ version_sha }} $
-I${srcroot}/src/include
cflags = -std=c11
cxxflags = -std=c++17
libs =
rule c
deps = gcc
depfile = $out.d
description = Compiling $name
command = $cc -MMD -MF $out.d $ccflags $cflags -o $out -c $in
rule dump_c_defs
description = Dumping C defines for $target
command = echo "" | $cc $ccflags $cflags -dM -E - > $out
rule dump_c_run
description = Dumping C arguments for $target
command = $
echo "#!/bin/bash" > $out; $
echo '$cc $ccflags $cflags $$*' >> $out; $
chmod a+x $out
rule cpp
deps = gcc
depfile = $out.d
description = Compiling $name
command = $cxx -MMD -MF $out.d $cxxflags $ccflags -o $out -c $in
rule dump_cpp_defs
description = Dumping C++ defines for $target
command = echo "" | $cxx -x c++ $cxxflags $ccflags -dM -E - > $out
rule dump_cpp_run
description = Dumping C++ arguments for $target
command = $
echo "#!/bin/bash" > $out; $
echo '$cc $cxxflags $ccflags $$*' >> $out; $
chmod a+x $out
rule s
deps = gcc
depfile = $out.d
description = Assembling $name
command = $nasm -o $out -felf64 -MD $out.d $asflags $in
rule exe
description = Linking $name
command = $ld $ldflags -o $out $in $libs
rule lib
description = Archiving $name
command = $ar qcs $out $in
rule regen
generator = true
description = Regenrating build files
command = $
{{ generator }} $
--file $modulefile $
--dir $builddir $
generate
rule cp
description = Copying $name
command = cp $in $out
rule dump
description = Dumping decompiled $name
command = objdump -DSC -M intel $in > $out
rule makest
description = Making symbol table
command = nm $in | ${srcroot}/scripts/build_symbol_table.py $out
rule makeefi
description = Converting $name
command = objcopy $
-j .text $
-j .sdata $
-j .data $
-j .dynamic $
-j .dynsym $
-j .rel $
-j .rela $
-j .reloc $
--target=efi-app-x86_64 $
$in $out
rule makefat
description = Creating $name
command = $
cp $srcroot/assets/diskbase.img $out; $
mcopy -s -D o -i $out@@1M $builddir/fatroot/* ::/
rule strip
description = Stripping $name
command = $
cp $in $out; $
objcopy --only-keep-debug $out $out.debug; $
strip -g $out; $
objcopy --add-gnu-debuglink=$out.debug $out
{% for target in targets %}
subninja {{ target }}/target.ninja
{% endfor %}
build $
{%- for buildfile in buildfiles %}
{{ buildfile }} $
{%- endfor %}
: regen | $
{%- for template in templates %}
{{ template }} $
{%- endfor %}
$modulefile $
{{ generator }}
build $builddir/ovmf_vars.fd : cp $srcroot/assets/ovmf/x64/ovmf_vars.fd
name = ovmf_vars.fd
build $builddir/ovmf_vars_d.fd : cp $srcroot/assets/ovmf/x64/ovmf_vars_d.fd
name = ovmf_vars_d.fd
build $builddir/jsix.elf | $builddir/jsix.elf.debug : strip $builddir/host/jsix.elf
name = kernel
build $builddir/jsix.dump : dump $builddir/host/jsix.elf
name = kernel
build $builddir/jsix.elf-gdb.py : cp ${srcroot}/assets/debugging/jsix.elf-gdb.py
name = kernel debug python scripts
build $builddir/fatroot/jsix.elf : cp $builddir/jsix.elf
name = kernel to FAT image
build $builddir/fatroot/efi/boot/bootx64.efi : cp $builddir/boot/boot.efi
name = bootloader to FAT image
build $builddir/fatroot/nulldrv.elf : cp $builddir/user/nulldrv.elf
name = null driver to FAT image
build $builddir/fatroot/fb.elf : cp $builddir/user/fb.elf
name = fb driver to FAT image
build ${builddir}/fatroot/symbol_table.dat : makest ${builddir}/jsix.elf
build $builddir/jsix.img : makefat | $
$builddir/fatroot/symbol_table.dat $
$builddir/fatroot/nulldrv.elf $
$builddir/fatroot/fb.elf $
$builddir/fatroot/jsix.elf $
$builddir/fatroot/efi/boot/bootx64.efi
name = jsix.img
default $
$builddir/ovmf_vars.fd $
$builddir/ovmf_vars_d.fd $
$builddir/jsix.dump $
$builddir/jsix.elf-gdb.py $
$builddir/jsix.img
# vim: ft=ninja et ts=4 sts=4 sw=4

14
scripts/templates/exe.boot.j2 Normal file
View File

@@ -0,0 +1,14 @@
{% extends "exe.default.j2" %}
{% block variables %}
{{ super() }}
ccflags = $ccflags $
-g3 $
-DKERNEL_FILENAME=L\"jsix.elf\" $
-I${srcroot}/external/include $
-I${srcroot}/external/include/X64
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

8
scripts/templates/exe.default.j2 Normal file
View File

@@ -0,0 +1,8 @@
{% extends "module.base.j2" %}
{% block variables %}
{{ super() }}
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

13
scripts/templates/exe.kernel.j2 Normal file
View File

@@ -0,0 +1,13 @@
{% extends "exe.default.j2" %}
{% block variables %}
{{ super() }}
asflags = $asflags -I${srcroot}/src/kernel/
libs = $libs
ldflags = $ldflags -T ${srcroot}/src/arch/x86_64/kernel.ld
ccflags = $ccflags -I${srcroot}/external
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

10
scripts/templates/exe.makerd.j2 Normal file
View File

@@ -0,0 +1,10 @@
{% extends "exe.default.j2" %}
{% block variables %}
{{ super() }}
ccflags = $ccflags -I${srcroot}/external/cpptoml
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

10
scripts/templates/exe.tests.j2 Normal file
View File

@@ -0,0 +1,10 @@
{% extends "exe.default.j2" %}
{% block variables %}
{{ super() }}
ccflags = $ccflags -ggdb -I${srcroot}/external/catch
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

4
scripts/templates/lib.default.j2 Normal file
View File

@@ -0,0 +1,4 @@
{% extends "module.base.j2" %}
# vim: ft=ninja et ts=4 sts=4 sw=4

45
scripts/templates/module.base.j2 Normal file
View File

@@ -0,0 +1,45 @@
moddir = ${builddir}/{{ name }}.dir
{% block variables %}
ccflags = $ccflags $
{%- for dep in depmods %}
{%- for inc in dep.includes %}
-I${srcroot}/{{ inc }} $
{%- endfor %}
{%- endfor %}
{%- for inc in module.includes %}
-I${srcroot}/{{ inc }} $
{%- endfor %}
{%- for define in module.defines %}
-D{{ define }} $
{%- endfor %}
{% endblock %}
{% for source in module.source %}
build ${moddir}/{{ source.output }} : {{ source.action }} ${srcroot}/{{ source.input }} || {{ buildfile }}
name = {{ source.name }}
{% endfor %}
build ${builddir}/{{ module.output }} : {{ module.kind }} $
{%- for source in module.source %}
${moddir}/{{ source.output }} $
{%- endfor -%}
{%- for dep in deplibs %}
${builddir}/{{ dep.output }} $
{%- endfor %}
| $
{%- for dep in depexes %}
${builddir}/{{ dep.output }} $
{%- endfor %}
{{ buildfile }}
name = {{ name }}
{% if module.default %}
default ${builddir}/{{ module.output }}
{% endif %}
{% block extra %}
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

38
scripts/templates/target.boot.j2 Normal file
View File

@@ -0,0 +1,38 @@
{% extends "target.default.j2" %}
{% block binaries %}
cc = clang
cxx = clang++
ld = clang++
ar = ar
nasm = nasm
objcopy = objcopy
{% endblock %}
{% block variables %}
ccflags = $ccflags $
-I $srcroot/external $
--target=x86_64-unknown-windows $
-ffreestanding $
-mno-red-zone $
-fshort-wchar $
-fno-omit-frame-pointer $
-ggdb
cxxflags = $cxxflags $
-fno-rtti $
-fno-exceptions
ldflags = $ldflags $
--target=x86_64-unknown-windows $
-nostdlib $
-Wl,-entry:efi_main $
-Wl,-subsystem:efi_application $
-fuse-ld=lld-link $
-g
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

31
scripts/templates/target.default.j2 Normal file
View File

@@ -0,0 +1,31 @@
builddir = $builddir/{{ target }}
target = {{ target }}
{% block variables %}
{% endblock %}
{% block binaries %}
cc = clang
cxx = clang++
ld = ld
ar = ar
nasm = nasm
objcopy = objcopy
{% endblock %}
{% for module in modules %}
subninja {{ module }}.ninja
{% endfor %}
build ${builddir}/c.defs : dump_c_defs | {{ buildfile }}
build ${builddir}/cpp.defs : dump_cpp_defs | {{ buildfile }}
build ${builddir}/c.run : dump_c_run | {{ buildfile }}
build ${builddir}/cpp.run : dump_cpp_run | {{ buildfile }}
default ${builddir}/c.defs
default ${builddir}/cpp.defs
default ${builddir}/c.run
default ${builddir}/cpp.run
# vim: ft=ninja et ts=4 sts=4 sw=4

44
scripts/templates/target.host.j2 Normal file
View File

@@ -0,0 +1,44 @@
{% extends "target.default.j2" %}
{% block binaries %}
cc = ${srcroot}/sysroot/bin/clang
cxx = ${srcroot}/sysroot/bin/clang++
ld = ${srcroot}/sysroot/bin/ld.lld
ar = ${srcroot}/sysroot/bin/ar
nasm = nasm
objcopy = ${srcroot}/sysroot/bin/objcopy
{% endblock %}
{% block variables %}
ccflags = $ccflags $
-nostdlib $
-ffreestanding $
-nodefaultlibs $
-fno-builtin $
-mno-sse $
-fno-omit-frame-pointer $
-mno-red-zone $
-g $
-mcmodel=large $
-D__ELF__ $
-D__JSIX__ $
-isystem${srcroot}/sysroot/include $
-isystem${srcroot}/src/libraries/libc/include $
--sysroot="${srcroot}/sysroot"
cxxflags = $cxxflags $
-fno-exceptions $
-fno-rtti $
-isystem${srcroot}/sysroot/include/c++/v1
ldflags = $ldflags $
-g $
-nostdlib $
-Bsymbolic $
-Bstatic
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

15
scripts/templates/target.native.j2 Normal file
View File

@@ -0,0 +1,15 @@
{% extends "target.default.j2" %}
{% block binaries %}
{{ super() }}
ld = clang++
{% endblock %}
{% block variables %}
ccflags = $ccflags -g -ggdb
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

45
scripts/templates/target.user.j2 Normal file
View File

@@ -0,0 +1,45 @@
{% extends "target.default.j2" %}
{% block binaries %}
cc = ${srcroot}/sysroot/bin/clang
cxx = ${srcroot}/sysroot/bin/clang++
ld = ${srcroot}/sysroot/bin/ld.lld
ar = ${srcroot}/sysroot/bin/ar
nasm = nasm
objcopy = ${srcroot}/sysroot/bin/objcopy
{% endblock %}
{% block variables %}
ccflags = $ccflags $
-nostdlib $
-nodefaultlibs $
-fno-builtin $
-mno-sse $
-fno-omit-frame-pointer $
-mno-red-zone $
-g $
-mcmodel=large $
-D__ELF__ $
-D__JSIX__ $
-isystem${srcroot}/sysroot/include $
-isystem${srcroot}/src/libraries/libc/include $
--sysroot="${srcroot}/sysroot"
cxxflags = $cxxflags $
-fno-exceptions $
-fno-rtti $
-isystem${srcroot}/sysroot/include/c++/v1
ldflags = $ldflags $
-g $
-nostdlib $
-Bsymbolic $
-Bstatic $
--sysroot="${srcroot}/sysroot" $
-L "${srcroot}/sysroot/lib" $
{% endblock %}
# vim: ft=ninja et ts=4 sts=4 sw=4

6
src/arch/x86_64/config.mk Normal file
View File

@@ -0,0 +1,6 @@
AS := nasm
ASFLAGS := -felf64
LDFLAGS := -m elf_x86_64
CFLAGS := -march=nocona -m64
# vim:ft=make

1
src/arch/x86_64/details.c Normal file
View File

@@ -0,0 +1 @@
const char *KERNEL_PLATFORM = "x86_64";

13
src/kernel/kernel.ld → src/arch/x86_64/kernel.ld
View File

@@ -1,4 +1,4 @@
ENTRY(_kernel_start.real) ENTRY(_kernel_start)
SECTIONS SECTIONS
{ {
. = 0xFFFF800000000000; . = 0xFFFF800000000000;
@@ -10,25 +10,24 @@ SECTIONS
} }
.text ALIGN(4096) : { .text ALIGN(4096) : {
*(.text*) *(.text)
KEEP(*(.isrs)) *(.isrs)
} }
.data ALIGN(4096) : { .data ALIGN(4096) : {
*(.data*) *(.data)
*(.rodata*) *(.rodata)
} }
.ctors : ALIGN(8) { .ctors : ALIGN(8) {
__ctors = .; __ctors = .;
KEEP(*(.ctors)) KEEP(*(.ctors))
KEEP(*(.init_array))
__ctors_end = .; __ctors_end = .;
} }
.bss ALIGN(4096) : { .bss ALIGN(4096) : {
__bss_start = .; __bss_start = .;
*(.bss*) *(.bss)
__bss_end = .; __bss_end = .;
} }

173
src/boot/allocator.cpp
View File

@@ -1,173 +0,0 @@
#include <uefi/boot_services.h>
#include <uefi/types.h>
#include <bootproto/init.h>
#include <bootproto/kernel.h>
#include <util/no_construct.h>
#include <util/pointers.h>
#include "allocator.h"
#include "error.h"
#include "memory.h"
namespace boot {
util::no_construct<memory::allocator> __g_alloc_storage;
memory::allocator &g_alloc = __g_alloc_storage.value;
namespace memory {
using bootproto::allocation_register;
using bootproto::module;
using bootproto::page_allocation;
static_assert(sizeof(allocation_register) == page_size);
void
allocator::init(
allocation_register *&allocs,
modules_page *&modules,
uefi::boot_services *bs)
{
new (&g_alloc) allocator(*bs);
allocs = g_alloc.m_register;
modules = g_alloc.m_modules;
}
allocator::allocator(uefi::boot_services &bs) :
m_bs(bs),
m_register(nullptr),
m_modules(nullptr)
{
add_register();
add_modules();
}
void
allocator::add_register()
{
allocation_register *reg = nullptr;
try_or_raise(
m_bs.allocate_pages(uefi::allocate_type::any_pages,
uefi::memory_type::loader_data, 1, reinterpret_cast<void**>(&reg)),
L"Failed allocating allocation register page");
m_bs.set_mem(reg, sizeof(allocation_register), 0);
if (m_register)
m_register->next = reg;
m_register = reg;
return;
}
void
allocator::add_modules()
{
modules_page *mods = reinterpret_cast<modules_page*>(
allocate_pages(1, alloc_type::init_args, true));
if (m_modules)
m_modules->next = reinterpret_cast<uintptr_t>(mods);
mods->modules = reinterpret_cast<module*>(mods + 1);
m_modules = mods;
m_next_mod = mods->modules;
return;
}
void *
allocator::allocate_pages(size_t count, alloc_type type, bool zero)
{
if (count & ~0xffffffffull) {
error::raise(uefi::status::unsupported,
L"Cannot allocate more than 16TiB in pages at once.",
__LINE__);
}
if (!m_register || m_register->count == 0xff)
add_register();
void *pages = nullptr;
try_or_raise(
m_bs.allocate_pages(uefi::allocate_type::any_pages,
uefi::memory_type::loader_data, count, &pages),
L"Failed allocating usable pages");
page_allocation &ent = m_register->entries[m_register->count++];
ent.address = reinterpret_cast<uintptr_t>(pages);
ent.count = count;
ent.type = type;
if (zero)
m_bs.set_mem(pages, count * page_size, 0);
return pages;
}
module *
allocator::allocate_module_untyped(size_t size)
{
size_t remaining =
reinterpret_cast<uintptr_t>(m_modules) + page_size
- reinterpret_cast<uintptr_t>(m_next_mod);
if (size > remaining)
add_modules();
++m_modules->count;
module *m = m_next_mod;
m_next_mod = util::offset_pointer(m_next_mod, size);
m->mod_length = size;
return m;
}
void *
allocator::allocate(size_t size, bool zero)
{
void *p = nullptr;
try_or_raise(
m_bs.allocate_pool(uefi::memory_type::loader_data, size, &p),
L"Could not allocate pool memory");
if (zero)
m_bs.set_mem(p, size, 0);
return p;
}
void
allocator::free(void *p)
{
try_or_raise(
m_bs.free_pool(p),
L"Freeing pool memory");
}
void
allocator::memset(void *start, size_t size, uint8_t value)
{
m_bs.set_mem(start, size, value);
}
void
allocator::copy(void *to, void *from, size_t size)
{
m_bs.copy_mem(to, from, size);
}
} // namespace memory
} // namespace boot
void * operator new (size_t size, void *p) { return p; }
void * operator new(size_t size) { return boot::g_alloc.allocate(size); }
void * operator new [] (size_t size) { return boot::g_alloc.allocate(size); }
void operator delete (void *p) noexcept { return boot::g_alloc.free(p); }
void operator delete [] (void *p) noexcept { return boot::g_alloc.free(p); }

76
src/boot/allocator.h
View File

@@ -1,76 +0,0 @@
#pragma once
/// \file allocator.h
/// Page allocator class definition
namespace uefi {
class boot_services;
}
namespace bootproto {
enum class allocation_type : uint8_t;
struct allocation_register;
struct module;
struct modules_page;
}
namespace boot {
namespace memory {
using alloc_type = bootproto::allocation_type;
class allocator
{
public:
using allocation_register = bootproto::allocation_register;
using module = bootproto::module;
using modules_page = bootproto::modules_page;
allocator(uefi::boot_services &bs);
void * allocate(size_t size, bool zero = false);
void free(void *p);
void * allocate_pages(size_t count, alloc_type type, bool zero = false);
template <typename M>
M * allocate_module(size_t extra = 0) {
return static_cast<M*>(allocate_module_untyped(sizeof(M) + extra));
}
void memset(void *start, size_t size, uint8_t value);
void copy(void *to, void *from, size_t size);
inline void zero(void *start, size_t size) { memset(start, size, 0); }
/// Initialize the global allocator
/// \arg allocs [out] Poiinter to the initial allocation register
/// \arg modules [out] Pointer to the initial modules_page
/// \arg bs UEFI boot services
static void init(
allocation_register *&allocs,
modules_page *&modules,
uefi::boot_services *bs);
private:
void add_register();
void add_modules();
module * allocate_module_untyped(size_t size);
uefi::boot_services &m_bs;
allocation_register *m_register;
modules_page *m_modules;
module *m_next_mod;
};
} // namespace memory
extern memory::allocator &g_alloc;
} // namespace boot
void * operator new (size_t size, void *p);
void * operator new(size_t size);
void * operator new [] (size_t size);
void operator delete (void *p) noexcept;
void operator delete [] (void *p) noexcept;

23
src/boot/boot.module
View File

@@ -1,23 +0,0 @@
# vim: ft=python
boot = module("boot",
kind = "exe",
output = "boot.efi",
targets = [ "boot" ],
deps = [ "cpu", "elf", "util", "bootproto" ],
sources = [
"allocator.cpp",
"bootconfig.cpp",
"console.cpp",
"error.cpp",
"fs.cpp",
"hardware.cpp",
"loader.cpp",
"main.cpp",
"memory.cpp",
"memory_map.cpp",
"paging.cpp",
"status.cpp",
"support.cpp",
"video.cpp",
])

63
src/boot/bootconfig.cpp
View File

@@ -1,63 +0,0 @@
#include <uefi/boot_services.h>
#include <uefi/types.h>
#include "bootconfig.h"
#include "error.h"
#include "fs.h"
#include "status.h"
namespace boot {
constexpr uint64_t jsixboot = 0x746f6f627869736a; // "jsixboot"
static const wchar_t *
read_string(util::buffer &data)
{
uint16_t size = *util::read<uint16_t>(data);
const wchar_t *string = reinterpret_cast<const wchar_t*>(data.pointer);
data += size;
return string;
}
static void
read_descriptor(descriptor &e, util::buffer &data)
{
e.flags = static_cast<desc_flags>(*util::read<uint16_t>(data));
e.path = read_string(data);
e.desc = read_string(data);
}
bootconfig::bootconfig(util::buffer data, uefi::boot_services *bs)
{
status_line status {L"Loading boot config"};
if (*util::read<uint64_t>(data) != jsixboot)
error::raise(uefi::status::load_error, L"Bad header in jsix_boot.dat");
const uint8_t version = *util::read<uint8_t>(data);
if (version != 0)
error::raise(uefi::status::incompatible_version, L"Bad version in jsix_boot.dat");
data += 1; // reserved byte
uint16_t num_programs = *util::read<uint16_t>(data);
uint16_t num_data = *util::read<uint16_t>(data);
m_flags = *util::read<uint16_t>(data);
read_descriptor(m_kernel, data);
read_descriptor(m_init, data);
m_programs.count = num_programs;
m_programs.pointer = new descriptor [num_programs];
for (unsigned i = 0; i < num_programs; ++i)
read_descriptor(m_programs[i], data);
m_data.count = num_programs;
m_data.pointer = new descriptor [num_data];
for (unsigned i = 0; i < num_data; ++i)
read_descriptor(m_data[i], data);
}
} // namespace boot

45
src/boot/bootconfig.h
View File

@@ -1,45 +0,0 @@
/// \file bootconfig.h
/// Definitions for reading the jsix bootconfig file
#pragma once
#include <bootproto/bootconfig.h>
#include <util/counted.h>
namespace uefi {
struct boot_services;
}
namespace boot {
using desc_flags = bootproto::desc_flags;
struct descriptor {
desc_flags flags;
wchar_t const *path;
wchar_t const *desc;
};
/// A bootconfig is a manifest of potential files.
class bootconfig
{
public:
using descriptors = util::counted<descriptor>;
/// Constructor. Loads bootconfig from the given buffer.
bootconfig(util::buffer data, uefi::boot_services *bs);
inline uint16_t flags() const { return m_flags; }
inline const descriptor & kernel() const { return m_kernel; }
inline const descriptor & init() const { return m_init; }
descriptors programs() { return m_programs; }
descriptors data() { return m_data; }
private:
uint16_t m_flags;
descriptor m_kernel;
descriptor m_init;
descriptors m_programs;
descriptors m_data;
};
} // namespace boot

109
src/boot/console.cpp
View File

@@ -1,8 +1,9 @@
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
#include <uefi/protos/simple_text_output.h>
#include <uefi/types.h> #include <uefi/types.h>
#include <uefi/graphics.h>
#include <uefi/protos/graphics_output.h>
#include "console.h" #include "console.h"
#include "error.h" #include "error.h"
@@ -22,7 +23,7 @@ static const wchar_t digits[] = {u'0', u'1', u'2', u'3', u'4', u'5',
u'6', u'7', u'8', u'9', u'a', u'b', u'c', u'd', u'e', u'f'}; u'6', u'7', u'8', u'9', u'a', u'b', u'c', u'd', u'e', u'f'};
size_t static size_t
wstrlen(const wchar_t *s) wstrlen(const wchar_t *s)
{ {
size_t count = 0; size_t count = 0;
@@ -31,23 +32,22 @@ wstrlen(const wchar_t *s)
} }
console::console(uefi::protos::simple_text_output *out) : console::console(uefi::boot_services *bs, uefi::protos::simple_text_output *out) :
m_rows {0}, m_rows {0},
m_cols {0}, m_cols {0},
m_out {out} m_out {out},
m_fb {0, 0}
{ {
pick_mode(bs);
try_or_raise( try_or_raise(
m_out->query_mode(m_out->mode->mode, &m_cols, &m_rows), m_out->query_mode(m_out->mode->mode, &m_cols, &m_rows),
L"Failed to get text output mode."); L"Failed to get text output mode.");
try_or_raise( try_or_raise(
m_out->clear_screen(), m_out->clear_screen(),
L"Failed to clear screen"); L"Failed to clear screen");
s_console = this;
}
void
console::announce()
{
m_out->set_attribute(uefi::attribute::light_cyan); m_out->set_attribute(uefi::attribute::light_cyan);
m_out->output_string(L"jsix loader "); m_out->output_string(L"jsix loader ");
@@ -56,6 +56,97 @@ console::announce()
m_out->set_attribute(uefi::attribute::light_gray); m_out->set_attribute(uefi::attribute::light_gray);
m_out->output_string(L" booting...\r\n"); m_out->output_string(L" booting...\r\n");
if (m_fb.type != kernel::args::fb_type::none) {
wchar_t const * type = nullptr;
switch (m_fb.type) {
case kernel::args::fb_type::rgb8:
type = L"rgb8";
break;
case kernel::args::fb_type::bgr8:
type = L"bgr8";
break;
default:
type = L"unknown";
}
printf(L"Found framebuffer: %dx%d[%d] type %s @0x%x\r\n",
m_fb.horizontal, m_fb.vertical, m_fb.scanline, type, m_fb.phys_addr);
} else {
printf(L"No framebuffer found.\r\n");
}
s_console = this;
}
void
console::pick_mode(uefi::boot_services *bs)
{
uefi::status status;
uefi::protos::graphics_output *gfx_out_proto;
uefi::guid guid = uefi::protos::graphics_output::guid;
m_fb.type = kernel::args::fb_type::none;
uefi::status has_gop = bs->locate_protocol(&guid, nullptr,
(void **)&gfx_out_proto);
if (has_gop != uefi::status::success)
// No video output found, skip it
return;
const uint32_t modes = gfx_out_proto->mode->max_mode;
uint32_t best = gfx_out_proto->mode->mode;
uefi::graphics_output_mode_info *info =
(uefi::graphics_output_mode_info *)gfx_out_proto->mode;
uint32_t res = info->horizontal_resolution * info->vertical_resolution;
int pixmode = static_cast<int>(info->pixel_format);
for (uint32_t i = 0; i < modes; ++i) {
size_t size = 0;
try_or_raise(
gfx_out_proto->query_mode(i, &size, &info),
L"Failed to find a graphics mode the driver claimed to support");
#ifdef MAX_HRES
if (info->horizontal_resolution > MAX_HRES) continue;
#endif
const uint32_t new_res = info->horizontal_resolution * info->vertical_resolution;
int new_pixmode = static_cast<int>(info->pixel_format);
if (new_pixmode <= pixmode && new_res >= res) {
best = i;
res = new_res;
pixmode = new_pixmode;
}
}
try_or_raise(
gfx_out_proto->set_mode(best),
L"Failed to set graphics mode");
if (pixmode <= static_cast<int>(uefi::pixel_format::bgr8)) {
m_fb.phys_addr = gfx_out_proto->mode->frame_buffer_base;
m_fb.size = gfx_out_proto->mode->frame_buffer_size;
m_fb.vertical = gfx_out_proto->mode->info->vertical_resolution;
m_fb.horizontal = gfx_out_proto->mode->info->horizontal_resolution;
m_fb.scanline = gfx_out_proto->mode->info->pixels_per_scanline;
switch (gfx_out_proto->mode->info->pixel_format) {
case uefi::pixel_format::rgb8:
m_fb.type = kernel::args::fb_type::rgb8;
break;
case uefi::pixel_format::bgr8:
m_fb.type = kernel::args::fb_type::bgr8;
break;
default:
m_fb.type = kernel::args::fb_type::none;
}
}
} }
size_t size_t

23
src/boot/console.h
View File

@@ -1,15 +1,12 @@
#pragma once
/// \file console.h /// \file console.h
/// Text output handler /// Text output handler
#pragma once
#include <stdarg.h> #include <stdarg.h>
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <uefi/boot_services.h>
#include <uefi/protos/simple_text_output.h>
namespace uefi { #include "kernel_args.h"
namespace protos { #include "types.h"
struct simple_text_output;
}}
namespace boot { namespace boot {
@@ -17,9 +14,9 @@ namespace boot {
class console class console
{ {
public: public:
console(uefi::protos::simple_text_output *out); using framebuffer = kernel::args::framebuffer;
void announce(); console(uefi::boot_services *bs, uefi::protos::simple_text_output *out);
size_t print_hex(uint32_t n) const; size_t print_hex(uint32_t n) const;
size_t print_dec(uint32_t n) const; size_t print_dec(uint32_t n) const;
@@ -27,20 +24,22 @@ public:
size_t print_long_dec(uint64_t n) const; size_t print_long_dec(uint64_t n) const;
size_t printf(const wchar_t *fmt, ...) const; size_t printf(const wchar_t *fmt, ...) const;
const framebuffer & fb() const { return m_fb; };
static console & get() { return *s_console; } static console & get() { return *s_console; }
static size_t print(const wchar_t *fmt, ...); static size_t print(const wchar_t *fmt, ...);
private: private:
friend class status_line; friend class status_line;
void pick_mode(uefi::boot_services *bs);
size_t vprintf(const wchar_t *fmt, va_list args) const; size_t vprintf(const wchar_t *fmt, va_list args) const;
size_t m_rows, m_cols; size_t m_rows, m_cols;
uefi::protos::simple_text_output *m_out; uefi::protos::simple_text_output *m_out;
framebuffer m_fb;
static console *s_console; static console *s_console;
}; };
size_t wstrlen(const wchar_t *s);
} // namespace boot } // namespace boot

83
src/boot/elf.h Normal file
View File

@@ -0,0 +1,83 @@
/// \file elf.h
/// Definitions and related constants for ELF64 structures
#pragma once
#include <stdint.h>
namespace boot {
namespace elf {
constexpr uint8_t version = 1;
constexpr uint8_t word_size = 2;
constexpr uint8_t endianness = 1;
constexpr uint8_t os_abi = 0;
constexpr uint16_t machine = 0x3e;
const unsigned PT_LOAD = 1;
const unsigned ST_PROGBITS = 1;
const unsigned ST_NOBITS = 8;
const unsigned long SHF_ALLOC = 0x2;
struct header
{
char magic[4];
uint8_t word_size;
uint8_t endianness;
uint8_t header_version;
uint8_t os_abi;
uint64_t reserved;
uint16_t type;
uint16_t machine;
uint32_t version;
uint64_t entrypoint;
uint64_t ph_offset;
uint64_t sh_offset;
uint32_t flags;
uint16_t eh_size;
uint16_t ph_entsize;
uint16_t ph_num;
uint16_t sh_entsize;
uint16_t sh_num;
uint16_t sh_str_idx;
} __attribute__ ((packed));
struct program_header
{
uint32_t type;
uint32_t flags;
uint64_t offset;
uint64_t vaddr;
uint64_t paddr;
uint64_t file_size;
uint64_t mem_size;
uint64_t align;
} __attribute__ ((packed));
struct section_header
{
uint32_t name;
uint32_t type;
uint64_t flags;
uint64_t addr;
uint64_t offset;
uint64_t size;
uint32_t link;
uint32_t info;
uint64_t align;
uint64_t entry_size;
} __attribute__ ((packed));
} // namespace elf
} // namespace boot

1
src/boot/error.cpp
View File

@@ -1,5 +1,6 @@
#include "error.h" #include "error.h"
#include "console.h" #include "console.h"
#include "kernel_args.h"
#include "status.h" #include "status.h"
namespace boot { namespace boot {

36
src/boot/fs.cpp
View File

@@ -1,37 +1,34 @@
#include <uefi/boot_services.h>
#include <uefi/types.h> #include <uefi/types.h>
#include <uefi/protos/file.h> #include <uefi/protos/file.h>
#include <uefi/protos/file_info.h> #include <uefi/protos/file_info.h>
#include <uefi/protos/loaded_image.h> #include <uefi/protos/loaded_image.h>
#include <uefi/protos/simple_file_system.h> #include <uefi/protos/simple_file_system.h>
#include <bootproto/kernel.h> #include "fs.h"
#include "allocator.h"
#include "console.h" #include "console.h"
#include "error.h" #include "error.h"
#include "fs.h"
#include "memory.h" #include "memory.h"
#include "status.h" #include "status.h"
namespace boot { namespace boot {
namespace fs { namespace fs {
using memory::alloc_type; file::file(uefi::protos::file *f, uefi::boot_services *bs) :
m_file(f),
file::file(uefi::protos::file *f) : m_bs(bs)
m_file(f)
{ {
} }
file::file(file &o) : file::file(file &o) :
m_file(o.m_file) m_file(o.m_file),
m_bs(o.m_bs)
{ {
o.m_file = nullptr; o.m_file = nullptr;
} }
file::file(file &&o) : file::file(file &&o) :
m_file(o.m_file) m_file(o.m_file),
m_bs(o.m_bs)
{ {
o.m_file = nullptr; o.m_file = nullptr;
} }
@@ -51,11 +48,11 @@ file::open(const wchar_t *path)
m_file->open(&fh, path, uefi::file_mode::read, uefi::file_attr::none), m_file->open(&fh, path, uefi::file_mode::read, uefi::file_attr::none),
L"Could not open relative path to file"); L"Could not open relative path to file");
return file(fh); return file(fh, m_bs);
} }
util::buffer buffer
file::load() file::load(uefi::memory_type mem_type)
{ {
uint8_t info_buf[sizeof(uefi::protos::file_info) + 100]; uint8_t info_buf[sizeof(uefi::protos::file_info) + 100];
size_t size = sizeof(info_buf); size_t size = sizeof(info_buf);
@@ -69,14 +66,19 @@ file::load()
reinterpret_cast<uefi::protos::file_info*>(&info_buf); reinterpret_cast<uefi::protos::file_info*>(&info_buf);
size_t pages = memory::bytes_to_pages(info->file_size); size_t pages = memory::bytes_to_pages(info->file_size);
void *data = g_alloc.allocate_pages(pages, alloc_type::file); void *data = nullptr;
try_or_raise(
m_bs->allocate_pages(
uefi::allocate_type::any_pages,
mem_type, pages, &data),
L"Could not allocate pages to load file");
size = info->file_size; size = info->file_size;
try_or_raise( try_or_raise(
m_file->read(&size, data), m_file->read(&size, data),
L"Could not read from file"); L"Could not read from file");
return { .pointer = data, .count = size }; return { .size = size, .data = data };
} }
file file
@@ -104,7 +106,7 @@ get_boot_volume(uefi::handle image, uefi::boot_services *bs)
fs->open_volume(&f), fs->open_volume(&f),
L"Could not open the boot volume"); L"Could not open the boot volume");
return file(f); return file(f, bs);
} }
} // namespace fs } // namespace fs

15
src/boot/fs.h
View File

@@ -3,13 +3,9 @@
#pragma once #pragma once
#include <uefi/types.h> #include <uefi/types.h>
#include <util/counted.h> #include <uefi/boot_services.h>
#include <uefi/protos/file.h>
namespace uefi { #include "types.h"
struct boot_services;
namespace protos {
struct file;
}}
namespace boot { namespace boot {
namespace fs { namespace fs {
@@ -27,14 +23,15 @@ public:
file open(const wchar_t *path); file open(const wchar_t *path);
/// Load the contents of this file into memory. /// Load the contents of this file into memory.
/// \arg mem_type The UEFI memory type to use for allocation
/// \returns A buffer describing the loaded memory. The /// \returns A buffer describing the loaded memory. The
/// memory will be page-aligned. /// memory will be page-aligned.
util::buffer load(); buffer load(uefi::memory_type mem_type = uefi::memory_type::loader_data);
private: private:
friend file get_boot_volume(uefi::handle, uefi::boot_services*); friend file get_boot_volume(uefi::handle, uefi::boot_services*);
file(uefi::protos::file *f); file(uefi::protos::file *f, uefi::boot_services *bs);
uefi::protos::file *m_file; uefi::protos::file *m_file;
uefi::boot_services *m_bs; uefi::boot_services *m_bs;

28
src/boot/hardware.cpp
View File

@@ -1,7 +1,6 @@
#include "console.h"
#include "cpu/cpu_id.h"
#include "error.h"
#include "hardware.h" #include "hardware.h"
#include "console.h"
#include "error.h"
#include "status.h" #include "status.h"
namespace boot { namespace boot {
@@ -54,6 +53,7 @@ wrmsr(uint32_t addr, uint64_t value)
__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high)); __asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
} }
void void
setup_control_regs() setup_control_regs()
{ {
@@ -77,27 +77,5 @@ setup_control_regs()
wrmsr(IA32_EFER, efer); wrmsr(IA32_EFER, efer);
} }
void
check_cpu_supported()
{
status_line status {L"Checking CPU features"};
cpu::cpu_id cpu;
uint64_t missing = cpu.missing();
if (missing) {
#define CPU_FEATURE_OPT(...)
#define CPU_FEATURE_REQ(name, ...) \
if (!cpu.has_feature(cpu::feature::name)) { \
status::fail(L"CPU required feature " L ## #name, uefi::status::unsupported); \
}
#include "cpu/features.inc"
#undef CPU_FEATURE_REQ
#undef CPU_FEATURE_OPT
error::raise(uefi::status::unsupported, L"CPU not supported");
}
}
} // namespace hw } // namespace hw
} // namespace boot } // namespace boot

3
src/boot/hardware.h
View File

@@ -16,8 +16,5 @@ void * find_acpi_table(uefi::system_table *st);
/// Enable CPU options in CR4 etc for the kernel starting state. /// Enable CPU options in CR4 etc for the kernel starting state.
void setup_control_regs(); void setup_control_regs();
/// Check that all required cpu features are supported
void check_cpu_supported();
} // namespace hw } // namespace hw
} // namespace boot } // namespace boot

187
src/boot/loader.cpp
View File

@@ -1,155 +1,116 @@
#include <uefi/boot_services.h> #include <uefi/boot_services.h>
#include <uefi/types.h> #include <uefi/types.h>
#include <bootproto/init.h> #include "loader.h"
#include <elf/file.h>
#include <elf/headers.h>
#include <util/pointers.h>
#include "allocator.h"
#include "bootconfig.h"
#include "console.h" #include "console.h"
#include "elf.h"
#include "error.h" #include "error.h"
#include "fs.h" #include "fs.h"
#include "loader.h"
#include "memory.h" #include "memory.h"
#include "paging.h" #include "paging.h"
#include "status.h" #include "status.h"
namespace args = kernel::args;
namespace boot { namespace boot {
namespace loader { namespace loader {
using memory::alloc_type; buffer
util::buffer
load_file( load_file(
fs::file &disk, fs::file &disk,
const descriptor &desc) const wchar_t *name,
const wchar_t *path,
uefi::memory_type type)
{ {
status_line status(L"Loading file", desc.path); status_line status(L"Loading file", name);
fs::file file = disk.open(desc.path); fs::file file = disk.open(path);
util::buffer b = file.load(); buffer b = file.load(type);
//console::print(L" Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size); //console::print(L" Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
return b; return b;
} }
static void static bool
create_module(util::buffer data, const descriptor &desc, bool loaded) is_elfheader_valid(const elf::header *header)
{ {
size_t path_len = wstrlen(desc.path); return
bootproto::module_program *mod = g_alloc.allocate_module<bootproto::module_program>(path_len); header->magic[0] == 0x7f &&
mod->mod_type = bootproto::module_type::program; header->magic[1] == 'E' &&
mod->base_address = reinterpret_cast<uintptr_t>(data.pointer); header->magic[2] == 'L' &&
mod->size = data.count; header->magic[3] == 'F' &&
if (loaded) header->word_size == elf::word_size &&
mod->mod_flags = static_cast<bootproto::module_flags>( header->endianness == elf::endianness &&
static_cast<uint8_t>(mod->mod_flags) | header->os_abi == elf::os_abi &&
static_cast<uint8_t>(bootproto::module_flags::no_load)); header->machine == elf::machine &&
header->header_version == elf::version;
// TODO: support non-ascii path characters and do real utf-16 to utf-8
// conversion
for (int i = 0; i < path_len; ++i) {
char c = desc.path[i];
mod->filename[i] = c == '\\' ? '/' : c;
}
mod->filename[path_len] = 0;
} }
bootproto::program * void
load_program( load_program(
fs::file &disk, args::program &program,
const descriptor &desc, const wchar_t *name,
bool add_module) buffer data,
uefi::boot_services *bs)
{ {
status_line status(L"Loading program", desc.desc); status_line status(L"Loading program:", name);
const elf::header *header = reinterpret_cast<const elf::header*>(data.data);
util::buffer data = load_file(disk, desc); if (data.size < sizeof(elf::header) || !is_elfheader_valid(header))
if (add_module)
create_module(data, desc, true);
elf::file program(data.pointer, data.count);
if (!program.valid())
error::raise(uefi::status::load_error, L"ELF file not valid"); error::raise(uefi::status::load_error, L"ELF file not valid");
size_t num_sections = 0; uintptr_t prog_base = uintptr_t(-1);
for (auto &seg : program.programs()) { uintptr_t prog_end = 0;
if (seg.type == elf::segment_type::load)
++num_sections;
}
bootproto::program_section *sections = new bootproto::program_section [num_sections]; for (int i = 0; i < header->ph_num; ++i) {
ptrdiff_t offset = header->ph_offset + i * header->ph_entsize;
const elf::program_header *pheader =
offset_ptr<elf::program_header>(data.data, offset);
size_t next_section = 0; if (pheader->type != elf::PT_LOAD)
for (auto &seg : program.programs()) {
if (seg.type != elf::segment_type::load)
continue; continue;
bootproto::program_section &section = sections[next_section++]; uintptr_t end = pheader->vaddr + pheader->mem_size;
if (pheader->vaddr < prog_base) prog_base = pheader->vaddr;
size_t page_count = memory::bytes_to_pages(seg.mem_size); if (end > prog_end) prog_end = end;
if (seg.mem_size > seg.file_size) {
void *pages = g_alloc.allocate_pages(page_count, alloc_type::program, true);
void *source = util::offset_pointer(data.pointer, seg.offset);
g_alloc.copy(pages, source, seg.file_size);
section.phys_addr = reinterpret_cast<uintptr_t>(pages);
} else {
section.phys_addr = program.base() + seg.offset;
} }
section.virt_addr = seg.vaddr; size_t total_size = prog_end - prog_base;
section.size = seg.mem_size; size_t num_pages = memory::bytes_to_pages(total_size);
section.type = static_cast<bootproto::section_flags>(seg.flags); void *pages = nullptr;
try_or_raise(
bs->allocate_pages(uefi::allocate_type::any_pages,
uefi::memory_type::loader_data, num_pages, &pages),
L"Failed allocating space for program");
bs->set_mem(pages, total_size, 0);
program.base = prog_base;
program.total_size = total_size;
program.num_sections = 0;
for (int i = 0; i < header->ph_num; ++i) {
ptrdiff_t offset = header->ph_offset + i * header->ph_entsize;
const elf::program_header *pheader =
offset_ptr<elf::program_header>(data.data, offset);
if (pheader->type != elf::PT_LOAD)
continue;
args::program_section &section = program.sections[program.num_sections++];
void *src_start = offset_ptr<void>(data.data, pheader->offset);
void *dest_start = offset_ptr<void>(pages, pheader->vaddr - prog_base);
bs->copy_mem(dest_start, src_start, pheader->file_size);
section.phys_addr = reinterpret_cast<uintptr_t>(dest_start);
section.virt_addr = pheader->vaddr;
section.size = pheader->mem_size;
section.type = static_cast<args::section_flags>(pheader->flags);
} }
bootproto::program *prog = new bootproto::program; program.entrypoint = header->entrypoint;
prog->sections = { .pointer = sections, .count = num_sections };
prog->phys_base = program.base();
prog->entrypoint = program.entrypoint();
return prog;
}
void
load_module(
fs::file &disk,
const descriptor &desc)
{
status_line status(L"Loading module", desc.desc);
util::buffer data = load_file(disk, desc);
create_module(data, desc, false);
}
void
verify_kernel_header(bootproto::program &program)
{
status_line status(L"Verifying kernel header");
const bootproto::header *header =
reinterpret_cast<const bootproto::header *>(program.sections[0].phys_addr);
if (header->magic != bootproto::header_magic)
error::raise(uefi::status::load_error, L"Bad kernel magic number");
if (header->length < sizeof(bootproto::header))
error::raise(uefi::status::load_error, L"Bad kernel header length");
if (header->version < bootproto::min_header_version)
error::raise(uefi::status::unsupported, L"Kernel header version not supported");
console::print(L" Loaded kernel vserion: %d.%d.%d %x\r\n",
header->version_major, header->version_minor, header->version_patch,
header->version_gitsha);
/*
for (auto &section : program.sections)
console::print(L" Section: p:0x%lx v:0x%lx fs:0x%x ms:0x%x\r\n",
section.phys_addr, section.virt_addr, section.file_size, section.mem_size);
*/
} }
} // namespace loader } // namespace loader

53
src/boot/loader.h
View File

@@ -2,53 +2,40 @@
/// Definitions for loading the kernel into memory /// Definitions for loading the kernel into memory
#pragma once #pragma once
#include <util/counted.h> #include <uefi/boot_services.h>
namespace bootproto { #include "kernel_args.h"
struct program; #include "memory.h"
struct module; #include "types.h"
}
namespace boot { namespace boot {
class descriptor; namespace fs { class file; }
namespace fs {
class file;
}
namespace loader { namespace loader {
/// Load a file from disk into memory. /// Load a file from disk into memory.
/// \arg disk The opened UEFI filesystem to load from /// \arg disk The opened UEFI filesystem to load from
/// \arg desc The program descriptor identifying the file /// \arg name Name of the module (informational only)
util::buffer /// \arg path Path on `disk` of the file to load
/// \arg type Memory type to use for allocation
buffer
load_file( load_file(
fs::file &disk, fs::file &disk,
const descriptor &desc); const wchar_t *name,
const wchar_t *path,
uefi::memory_type type = uefi::memory_type::loader_data);
/// Parse and load an ELF file in memory into a loaded image. /// Parse and load an ELF file in memory into a loaded image.
/// \arg disk The opened UEFI filesystem to load from /// \arg program The program structure to fill
/// \arg desc The descriptor identifying the program /// \arg data Buffer of the ELF file in memory
/// \arg add_module Also create a module for this loaded program /// \arg bs Boot services
bootproto::program * void
load_program( load_program(
fs::file &disk, kernel::args::program &program,
const descriptor &desc, const wchar_t *name,
bool add_module = false); buffer data,
uefi::boot_services *bs);
/// Load a file from disk into memory, creating an init args module
/// \arg disk The opened UEFI filesystem to load from
/// \arg desc The program descriptor identifying the file
void
load_module(
fs::file &disk,
const descriptor &desc);
/// Verify that a loaded ELF has the j6 kernel header
/// \arg program The program to check for a header
void
verify_kernel_header(bootproto::program &program);
} // namespace loader } // namespace loader
} // namespace boot } // namespace boot

246
src/boot/main.cpp
View File

@@ -7,134 +7,161 @@
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
#include <bootproto/bootconfig.h>
#include <bootproto/kernel.h>
#include <bootproto/memory.h>
#include <util/counted.h>
#include <util/pointers.h>
#include "allocator.h"
#include "bootconfig.h"
#include "console.h" #include "console.h"
#include "cpu/cpu.h"
#include "error.h" #include "error.h"
#include "fs.h" #include "fs.h"
#include "hardware.h" #include "hardware.h"
#include "loader.h" #include "loader.h"
#include "memory.h" #include "memory.h"
#include "memory_map.h"
#include "paging.h" #include "paging.h"
#include "status.h" #include "status.h"
#include "video.h"
#include "kernel_args.h"
namespace kernel {
#include "kernel_memory.h"
}
namespace args = kernel::args;
namespace boot { namespace boot {
constexpr int max_modules = 5; // Max modules to allocate room for
constexpr int max_programs = 5; // Max programs to allocate room for
struct program_desc
{
const wchar_t *name;
const wchar_t *path;
};
const program_desc program_list[] = {
{L"kernel", L"jsix.elf"},
{L"null driver", L"nulldrv.elf"},
{L"fb driver", L"fb.elf"},
};
/// Change a pointer to point to the higher-half linear-offset version /// Change a pointer to point to the higher-half linear-offset version
/// of the address it points to. /// of the address it points to.
template <typename T> template <typename T>
void change_pointer(T *&pointer) void change_pointer(T *&pointer)
{ {
pointer = util::offset_pointer(pointer, bootproto::mem::linear_offset); pointer = offset_ptr<T>(pointer, kernel::memory::page_offset);
} }
/// The main procedure for the portion of the loader that runs while /// Allocate space for kernel args. Allocates enough space from pool
/// UEFI is still in control of the machine. (ie, while the loader still /// memory for the args header and the module and program headers.
/// has access to boot services.) args::header *
bootproto::args * allocate_args_structure(
uefi_preboot(uefi::handle image, uefi::system_table *st) uefi::boot_services *bs,
size_t max_modules,
size_t max_programs)
{ {
uefi::boot_services *bs = st->boot_services; status_line status {L"Setting up kernel args memory"};
uefi::runtime_services *rs = st->runtime_services;
status_line status {L"Performing UEFI pre-boot"}; args::header *args = nullptr;
hw::check_cpu_supported(); size_t args_size =
memory::init_pointer_fixup(bs, rs); sizeof(args::header) + // The header itself
max_modules * sizeof(args::module) + // The module structures
max_programs * sizeof(args::program); // The program structures
bootproto::args *args = new bootproto::args; try_or_raise(
g_alloc.zero(args, sizeof(bootproto::args)); bs->allocate_pool(uefi::memory_type::loader_data, args_size,
reinterpret_cast<void**>(&args)),
L"Could not allocate argument memory");
args->magic = bootproto::args_magic; bs->set_mem(args, args_size, 0);
args->version = bootproto::args_version;
args->runtime_services = rs;
args->acpi_table = hw::find_acpi_table(st);
memory::mark_pointer_fixup(&args->runtime_services);
paging::allocate_tables(args); args->modules =
reinterpret_cast<args::module*>(args + 1);
args->num_modules = 0;
args->programs =
reinterpret_cast<args::program*>(args->modules + max_modules);
args->num_programs = 0;
return args; return args;
} }
/// Load the kernel and other programs from disk /// Add a module to the kernel args list
void inline void
load_resources(bootproto::args *args, video::screen *screen, uefi::handle image, uefi::boot_services *bs) add_module(args::header *args, args::mod_type type, buffer &data)
{ {
status_line status {L"Loading programs"}; args::module &m = args->modules[args->num_modules++];
m.type = type;
m.location = data.data;
m.size = data.size;
}
/// Check that all required cpu features are supported
void
check_cpu_supported()
{
status_line status {L"Checking CPU features"};
cpu::cpu_id cpu;
uint64_t missing = cpu.missing();
if (missing) {
#define CPU_FEATURE_OPT(...)
#define CPU_FEATURE_REQ(name, ...) \
if (!cpu.has_feature(cpu::feature::name)) { \
status::fail(L"CPU required feature " L ## #name, uefi::status::unsupported); \
}
#include "cpu/features.inc"
#undef CPU_FEATURE_REQ
#undef CPU_FEATURE_OPT
error::raise(uefi::status::unsupported, L"CPU not supported");
}
}
/// The main procedure for the portion of the loader that runs while
/// UEFI is still in control of the machine. (ie, while the loader still
/// has access to boot services.
args::header *
uefi_preboot(uefi::handle image, uefi::system_table *st)
{
status_line status {L"Performing UEFI pre-boot"};
uefi::boot_services *bs = st->boot_services;
uefi::runtime_services *rs = st->runtime_services;
memory::init_pointer_fixup(bs, rs);
args::header *args =
allocate_args_structure(bs, max_modules, max_programs);
args->magic = args::magic;
args->version = args::version;
args->runtime_services = rs;
args->acpi_table = hw::find_acpi_table(st);
paging::allocate_tables(args, bs);
memory::mark_pointer_fixup(&args->runtime_services);
fs::file disk = fs::get_boot_volume(image, bs); fs::file disk = fs::get_boot_volume(image, bs);
fs::file bc_data = disk.open(L"jsix_boot.dat");
bootconfig bc {bc_data.load(), bs};
args->kernel = loader::load_program(disk, bc.kernel(), true); buffer symbols = loader::load_file(disk, L"symbol table", L"symbol_table.dat",
args->init = loader::load_program(disk, bc.init()); uefi::memory_type::loader_data);
args->flags = static_cast<bootproto::boot_flags>(bc.flags()); add_module(args, args::mod_type::symbol_table, symbols);
namespace bits = util::bits; for (auto &desc : program_list) {
using bootproto::desc_flags; buffer buf = loader::load_file(disk, desc.name, desc.path);
args::program &program = args->programs[args->num_programs++];
if (screen) { loader::load_program(program, desc.name, buf, bs);
video::make_module(screen);
// Go through the screen-specific descriptors first to
// give them priority
for (const descriptor &d : bc.programs()) {
if (!bits::has(d.flags, desc_flags::graphical))
continue;
if (bits::has(d.flags, desc_flags::panic))
args->panic = loader::load_program(disk, d);
else
loader::load_module(disk, d);
}
} }
// Load the non-graphical descriptors return args;
for (const descriptor &d : bc.programs()) {
if (bits::has(d.flags, desc_flags::graphical))
continue;
if (bits::has(d.flags, desc_flags::panic) && !args->panic)
args->panic = loader::load_program(disk, d);
else
loader::load_module(disk, d);
}
// For now the only data we load is the symbol table
for (const descriptor &d : bc.data()) {
if (!bits::has(d.flags, desc_flags::symbols))
continue;
util::buffer symbol_table = loader::load_file(disk, d);
args->symbol_table = symbol_table.pointer;
break;
}
loader::verify_kernel_header(*args->kernel);
} }
memory::efi_mem_map memory::efi_mem_map
uefi_exit(bootproto::args *args, uefi::handle image, uefi::boot_services *bs) uefi_exit(args::header *args, uefi::handle image, uefi::boot_services *bs)
{ {
status_line status {L"Exiting UEFI", nullptr, false}; status_line status {L"Exiting UEFI", nullptr, false};
memory::efi_mem_map map; memory::efi_mem_map map =
map.update(*bs); memory::build_kernel_mem_map(args, bs);
args->mem_map = memory::build_kernel_map(map);
args->frame_blocks = memory::build_frame_blocks(args->mem_map);
map.update(*bs);
status.do_blank();
try_or_raise( try_or_raise(
bs->exit_boot_services(image, map.key), bs->exit_boot_services(image, map.key),
@@ -150,49 +177,34 @@ extern "C" uefi::status
efi_main(uefi::handle image, uefi::system_table *st) efi_main(uefi::handle image, uefi::system_table *st)
{ {
using namespace boot; using namespace boot;
console con(st->boot_services, st->con_out);
check_cpu_supported();
uefi::boot_services *bs = st->boot_services; args::header *args = uefi_preboot(image, st);
console con(st->con_out);
bootproto::allocation_register *allocs = nullptr;
bootproto::modules_page *modules = nullptr;
memory::allocator::init(allocs, modules, bs);
video::screen *screen = video::pick_mode(bs);
con.announce();
bootproto::args *args = uefi_preboot(image, st);
load_resources(args, screen, image, bs);
memory::efi_mem_map map = uefi_exit(args, image, st->boot_services); memory::efi_mem_map map = uefi_exit(args, image, st->boot_services);
args->allocations = allocs; args->video = con.fb();
args->modules = reinterpret_cast<uintptr_t>(modules); status_bar status {con.fb()}; // Switch to fb status display
status_bar status {screen}; // Switch to fb status display // Map the kernel to the appropriate address
args::program &kernel = args->programs[0];
// Map the kernel and panic handler to the appropriate addresses for (auto &section : kernel.sections)
paging::map_program(args, *args->kernel); if (section.size)
paging::map_program(args, *args->panic); paging::map_section(args, section);
memory::fix_frame_blocks(args); memory::fix_frame_blocks(args);
bootproto::entrypoint kentry = kernel::entrypoint kentry =
reinterpret_cast<bootproto::entrypoint>(args->kernel->entrypoint); reinterpret_cast<kernel::entrypoint>(kernel.entrypoint);
//status.next(); status.next();
hw::setup_control_regs(); hw::setup_control_regs();
memory::virtualize(args->pml4, map, st->runtime_services); memory::virtualize(args->pml4, map, st->runtime_services);
//status.next(); status.next();
change_pointer(args);
change_pointer(args->pml4); change_pointer(args->pml4);
status.next();
change_pointer(args->kernel);
change_pointer(args->kernel->sections.pointer);
change_pointer(args->init);
change_pointer(args->init->sections.pointer);
//status.next();
kentry(args); kentry(args);
debug_break(); debug_break();

319
src/boot/memory.cpp
View File

@@ -1,24 +1,67 @@
#include <stddef.h> #include <stddef.h>
#include <uefi/boot_services.h>
#include <uefi/runtime_services.h>
#include <uefi/types.h> #include <uefi/types.h>
#include <bootproto/memory.h> #include "kernel_memory.h"
#include "console.h" #include "console.h"
#include "error.h" #include "error.h"
#include "memory.h" #include "memory.h"
#include "memory_map.h"
#include "paging.h" #include "paging.h"
#include "status.h" #include "status.h"
namespace boot { namespace boot {
namespace memory { namespace memory {
using mem_entry = kernel::args::mem_entry;
using mem_type = kernel::args::mem_type;
using frame_block = kernel::args::frame_block;
using kernel::args::frames_per_block;
size_t fixup_pointer_index = 0; size_t fixup_pointer_index = 0;
void **fixup_pointers[64]; void **fixup_pointers[64];
static const wchar_t *memory_type_names[] = {
L"reserved memory type",
L"loader code",
L"loader data",
L"boot services code",
L"boot services data",
L"runtime services code",
L"runtime services data",
L"conventional memory",
L"unusable memory",
L"acpi reclaim memory",
L"acpi memory nvs",
L"memory mapped io",
L"memory mapped io port space",
L"pal code",
L"persistent memory"
};
static const wchar_t *kernel_memory_type_names[] = {
L"free",
L"pending",
L"acpi",
L"uefi_runtime",
L"mmio",
L"persistent"
};
static const wchar_t *
memory_type_name(uefi::memory_type t)
{
if (t < uefi::memory_type::max_memory_type)
return memory_type_names[static_cast<uint32_t>(t)];
return L"Bad Type Value";
}
static const wchar_t *
kernel_memory_type_name(kernel::args::mem_type t)
{
return kernel_memory_type_names[static_cast<uint32_t>(t)];
}
void void
update_marked_addresses(uefi::event, void *context) update_marked_addresses(uefi::event, void *context)
{ {
@@ -56,13 +99,275 @@ mark_pointer_fixup(void **p)
fixup_pointers[fixup_pointer_index++] = p; fixup_pointers[fixup_pointer_index++] = p;
} }
bool
can_merge(mem_entry &prev, mem_type type, uefi::memory_descriptor *next)
{
return
prev.type == type &&
prev.start + (page_size * prev.pages) == next->physical_start &&
prev.attr == (next->attribute & 0xffffffff);
}
void
get_uefi_mappings(efi_mem_map &map, uefi::boot_services *bs)
{
size_t length = map.total;
uefi::status status = bs->get_memory_map(
&length, map.entries, &map.key, &map.size, &map.version);
map.length = length;
if (status == uefi::status::success)
return;
if (status != uefi::status::buffer_too_small)
error::raise(status, L"Error getting memory map size");
if (map.entries) {
try_or_raise(
bs->free_pool(reinterpret_cast<void*>(map.entries)),
L"Freeing previous memory map space");
}
map.total = length + 10*map.size;
try_or_raise(
bs->allocate_pool(
uefi::memory_type::loader_data, map.total,
reinterpret_cast<void**>(&map.entries)),
L"Allocating space for memory map");
map.length = map.total;
try_or_raise(
bs->get_memory_map(&map.length, map.entries, &map.key, &map.size, &map.version),
L"Getting UEFI memory map");
}
inline size_t bitmap_size(size_t frames) { return (frames + 63) / 64; }
inline size_t num_blocks(size_t frames) { return (frames + (frames_per_block-1)) / frames_per_block; }
void
build_kernel_frame_blocks(const mem_entry *map, size_t nent, kernel::args::header *args, uefi::boot_services *bs)
{
status_line status {L"Creating kernel frame accounting map"};
size_t block_count = 0;
size_t total_bitmap_size = 0;
for (size_t i = 0; i < nent; ++i) {
const mem_entry &ent = map[i];
if (ent.type != mem_type::free)
continue;
block_count += num_blocks(ent.pages);
total_bitmap_size += bitmap_size(ent.pages) * sizeof(uint64_t);
}
size_t total_size = block_count * sizeof(frame_block) + total_bitmap_size;
frame_block *blocks = nullptr;
try_or_raise(
bs->allocate_pages(
uefi::allocate_type::any_pages,
uefi::memory_type::loader_data,
bytes_to_pages(total_size),
reinterpret_cast<void**>(&blocks)),
L"Error allocating kernel frame block space");
frame_block *next_block = blocks;
for (size_t i = 0; i < nent; ++i) {
const mem_entry &ent = map[i];
if (ent.type != mem_type::free)
continue;
size_t page_count = ent.pages;
uintptr_t base_addr = ent.start;
while (page_count) {
frame_block *blk = next_block++;
bs->set_mem(blk, sizeof(frame_block), 0);
blk->attrs = ent.attr;
blk->base = base_addr;
base_addr += frames_per_block * page_size;
if (page_count >= frames_per_block) {
page_count -= frames_per_block;
blk->count = frames_per_block;
blk->map1 = ~0ull;
bs->set_mem(blk->map2, sizeof(blk->map2), 0xff);
} else {
blk->count = page_count;
unsigned i = 0;
uint64_t b1 = (page_count + 4095) / 4096;
blk->map1 = (1 << b1) - 1;
uint64_t b2 = (page_count + 63) / 64;
uint64_t b2q = b2 / 64;
uint64_t b2r = b2 % 64;
bs->set_mem(blk->map2, b2q, 0xff);
blk->map2[b2q] = (1 << b2r) - 1;
break;
}
}
}
uint64_t *bitmap = reinterpret_cast<uint64_t*>(next_block);
bs->set_mem(bitmap, total_bitmap_size, 0);
for (unsigned i = 0; i < block_count; ++i) {
frame_block &blk = blocks[i];
blk.bitmap = bitmap;
size_t b = blk.count / 64;
size_t r = blk.count % 64;
bs->set_mem(blk.bitmap, b*8, 0xff);
blk.bitmap[b] = (1 << r) - 1;
bitmap += bitmap_size(blk.count);
}
args->frame_block_count = block_count;
args->frame_block_pages = bytes_to_pages(total_size);
args->frame_blocks = blocks;
}
void
fix_frame_blocks(kernel::args::header *args)
{
// Map the frame blocks to the appropriate address
paging::map_pages(args,
reinterpret_cast<uintptr_t>(args->frame_blocks),
::memory::bitmap_start,
args->frame_block_pages,
true, false);
uintptr_t offset = ::memory::bitmap_start -
reinterpret_cast<uintptr_t>(args->frame_blocks);
for (unsigned i = 0; i < args->frame_block_count; ++i) {
frame_block &blk = args->frame_blocks[i];
blk.bitmap = reinterpret_cast<uint64_t*>(
reinterpret_cast<uintptr_t>(blk.bitmap) + offset);
}
}
efi_mem_map
build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
{
status_line status {L"Creating kernel memory map"};
efi_mem_map map;
get_uefi_mappings(map, bs);
size_t map_size = map.num_entries() * sizeof(mem_entry);
kernel::args::mem_entry *kernel_map = nullptr;
try_or_raise(
bs->allocate_pages(
uefi::allocate_type::any_pages,
uefi::memory_type::loader_data,
bytes_to_pages(map_size),
reinterpret_cast<void**>(&kernel_map)),
L"Error allocating kernel memory map module space");
bs->set_mem(kernel_map, map_size, 0);
get_uefi_mappings(map, bs);
size_t nent = 0;
bool first = true;
for (auto desc : map) {
/*
// EFI map dump
console::print(L" eRange %lx (%lx) %x(%s) [%lu]\r\n",
desc->physical_start, desc->attribute, desc->type, memory_type_name(desc->type), desc->number_of_pages);
*/
mem_type type;
switch (desc->type) {
case uefi::memory_type::reserved:
case uefi::memory_type::unusable_memory:
case uefi::memory_type::acpi_memory_nvs:
case uefi::memory_type::pal_code:
continue;
case uefi::memory_type::loader_code:
case uefi::memory_type::boot_services_code:
case uefi::memory_type::boot_services_data:
case uefi::memory_type::conventional_memory:
case uefi::memory_type::loader_data:
type = mem_type::free;
break;
case uefi::memory_type::runtime_services_code:
case uefi::memory_type::runtime_services_data:
type = mem_type::uefi_runtime;
break;
case uefi::memory_type::acpi_reclaim_memory:
type = mem_type::acpi;
break;
case uefi::memory_type::memory_mapped_io:
case uefi::memory_type::memory_mapped_io_port_space:
type = mem_type::mmio;
break;
case uefi::memory_type::persistent_memory:
type = mem_type::persistent;
break;
default:
error::raise(
uefi::status::invalid_parameter,
L"Got an unexpected memory type from UEFI memory map");
}
// TODO: validate uefi's map is sorted
if (first) {
first = false;
mem_entry &ent = kernel_map[nent++];
ent.start = desc->physical_start;
ent.pages = desc->number_of_pages;
ent.type = type;
ent.attr = (desc->attribute & 0xffffffff);
continue;
}
mem_entry &prev = kernel_map[nent - 1];
if (can_merge(prev, type, desc)) {
prev.pages += desc->number_of_pages;
} else {
mem_entry &next = kernel_map[nent++];
next.start = desc->physical_start;
next.pages = desc->number_of_pages;
next.type = type;
next.attr = (desc->attribute & 0xffffffff);
}
}
// Give just the actually-set entries in the header
args->mem_map = kernel_map;
args->map_count = nent;
/*
// kernel map dump
for (unsigned i = 0; i < nent; ++i) {
const kernel::args::mem_entry &e = kernel_map[i];
console::print(L" kRange %lx (%lx) %x(%s) [%lu]\r\n",
e.start, e.attr, e.type, kernel_memory_type_name(e.type), e.pages);
}
*/
build_kernel_frame_blocks(kernel_map, nent, args, bs);
get_uefi_mappings(map, bs);
return map;
}
void void
virtualize(void *pml4, efi_mem_map &map, uefi::runtime_services *rs) virtualize(void *pml4, efi_mem_map &map, uefi::runtime_services *rs)
{ {
paging::add_current_mappings(reinterpret_cast<paging::page_table*>(pml4)); paging::add_current_mappings(reinterpret_cast<paging::page_table*>(pml4));
for (auto &desc : map) for (auto desc : map)
desc.virtual_start = desc.physical_start + bootproto::mem::linear_offset; desc->virtual_start = desc->physical_start + ::memory::page_offset;
// Write our new PML4 pointer to CR3 // Write our new PML4 pointer to CR3
asm volatile ( "mov %0, %%cr3" :: "r" (pml4) ); asm volatile ( "mov %0, %%cr3" :: "r" (pml4) );

52
src/boot/memory.h
View File

@@ -1,19 +1,15 @@
#pragma once
/// \file memory.h /// \file memory.h
/// Memory-related constants and functions. /// Memory-related constants and functions.
#pragma once
#include <uefi/boot_services.h>
#include <uefi/runtime_services.h>
#include <stdint.h> #include <stdint.h>
#include "kernel_args.h"
namespace uefi { #include "pointer_manipulation.h"
struct boot_services;
struct runtime_services;
}
namespace boot { namespace boot {
namespace memory { namespace memory {
class efi_mem_map;
/// UEFI specifies that pages are always 4 KiB. /// UEFI specifies that pages are always 4 KiB.
constexpr size_t page_size = 0x1000; constexpr size_t page_size = 0x1000;
@@ -37,6 +33,44 @@ void mark_pointer_fixup(void **p);
/// @} /// @}
/// Struct that represents UEFI's memory map. Contains a pointer to the map data
/// as well as the data on how to read it.
struct efi_mem_map
{
using desc = uefi::memory_descriptor;
using iterator = offset_iterator<desc>;
size_t length; ///< Total length of the map data
size_t total; ///< Total allocated space for map data
size_t size; ///< Size of an entry in the array
size_t key; ///< Key for detecting changes
uint32_t version; ///< Version of the `memory_descriptor` struct
desc *entries; ///< The array of UEFI descriptors
efi_mem_map() : length(0), total(0), size(0), key(0), version(0), entries(nullptr) {}
/// Get the count of entries in the array
inline size_t num_entries() const { return length / size; }
/// Return an iterator to the beginning of the array
iterator begin() { return iterator(entries, size); }
/// Return an iterator to the end of the array
iterator end() { return offset_ptr<desc>(entries, length); }
};
/// Add the kernel's memory map as a module to the kernel args.
/// \returns The uefi memory map used to build the kernel map
efi_mem_map build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs);
/// Create the kernel frame allocation maps
void build_kernel_frame_blocks(
const kernel::args::mem_entry *map, size_t nent,
kernel::args::header *args, uefi::boot_services *bs);
/// Map the frame allocation maps to the right spot and fix up pointers
void fix_frame_blocks(kernel::args::header *args);
/// Activate the given memory mappings. Sets the given page tables live as well /// Activate the given memory mappings. Sets the given page tables live as well
/// as informs UEFI runtime services of the new mappings. /// as informs UEFI runtime services of the new mappings.
/// \arg pml4 The root page table for the new mappings /// \arg pml4 The root page table for the new mappings

306
src/boot/memory_map.cpp
View File

@@ -1,306 +0,0 @@
#include <uefi/boot_services.h>
#include <uefi/types.h>
#include <bootproto/kernel.h>
#include <bootproto/memory.h>
#include <util/pointers.h>
#include "allocator.h"
#include "error.h"
#include "memory.h"
#include "memory_map.h"
#include "paging.h"
#include "status.h"
namespace boot {
namespace memory {
using bootproto::frame_block;
using bootproto::frames_per_block;
using bootproto::mem_entry;
using bootproto::mem_type;
void
efi_mem_map::update(uefi::boot_services &bs)
{
size_t l = total;
uefi::status status = bs.get_memory_map(
&l, entries, &key, &size, &version);
length = l;
if (status == uefi::status::success)
return;
if (status != uefi::status::buffer_too_small)
error::raise(status, L"Error getting memory map size");
if (entries) {
try_or_raise(
bs.free_pool(reinterpret_cast<void*>(entries)),
L"Freeing previous memory map space");
}
total = length + 10 * size;
try_or_raise(
bs.allocate_pool(
uefi::memory_type::loader_data, total,
reinterpret_cast<void**>(&entries)),
L"Allocating space for memory map");
length = total;
try_or_raise(
bs.get_memory_map(&length, entries, &key, &size, &version),
L"Getting UEFI memory map");
}
static const wchar_t *memory_type_names[] = {
L"reserved memory type",
L"loader code",
L"loader data",
L"boot services code",
L"boot services data",
L"runtime services code",
L"runtime services data",
L"conventional memory",
L"unusable memory",
L"acpi reclaim memory",
L"acpi memory nvs",
L"memory mapped io",
L"memory mapped io port space",
L"pal code",
L"persistent memory"
};
static const wchar_t *kernel_memory_type_names[] = {
L"free",
L"pending",
L"acpi",
L"uefi_runtime",
L"mmio",
L"persistent"
};
static const wchar_t *
memory_type_name(uefi::memory_type t)
{
if (t < uefi::memory_type::max_memory_type)
return memory_type_names[static_cast<uint32_t>(t)];
return L"Bad Type Value";
}
static const wchar_t *
kernel_memory_type_name(bootproto::mem_type t)
{
return kernel_memory_type_names[static_cast<uint32_t>(t)];
}
inline bool
can_merge(mem_entry &prev, mem_type type, uefi::memory_descriptor &next)
{
return
prev.type == type &&
prev.start + (page_size * prev.pages) == next.physical_start &&
prev.attr == (next.attribute & 0xffffffff);
}
util::counted<mem_entry>
build_kernel_map(efi_mem_map &map)
{
status_line status {L"Creating kernel memory map"};
size_t map_size = map.num_entries() * sizeof(mem_entry);
size_t num_pages = bytes_to_pages(map_size);
mem_entry *kernel_map = reinterpret_cast<mem_entry*>(
g_alloc.allocate_pages(num_pages, alloc_type::mem_map, true));
size_t nent = 0;
bool first = true;
for (auto &desc : map) {
/*
// EFI map dump
console::print(L" eRange %lx (%lx) %x(%s) [%lu]\r\n",
desc.physical_start, desc.attribute, desc.type, memory_type_name(desc.type), desc.number_of_pages);
*/
mem_type type;
switch (desc.type) {
case uefi::memory_type::reserved:
case uefi::memory_type::unusable_memory:
case uefi::memory_type::acpi_memory_nvs:
case uefi::memory_type::pal_code:
continue;
case uefi::memory_type::loader_code:
case uefi::memory_type::boot_services_code:
case uefi::memory_type::boot_services_data:
case uefi::memory_type::conventional_memory:
case uefi::memory_type::loader_data:
type = mem_type::free;
break;
case uefi::memory_type::runtime_services_code:
case uefi::memory_type::runtime_services_data:
type = mem_type::uefi_runtime;
break;
case uefi::memory_type::acpi_reclaim_memory:
type = mem_type::acpi;
break;
case uefi::memory_type::memory_mapped_io:
case uefi::memory_type::memory_mapped_io_port_space:
type = mem_type::mmio;
break;
case uefi::memory_type::persistent_memory:
type = mem_type::persistent;
break;
default:
error::raise(
uefi::status::invalid_parameter,
L"Got an unexpected memory type from UEFI memory map");
}
// TODO: validate uefi's map is sorted
if (first) {
first = false;
mem_entry &ent = kernel_map[nent++];
ent.start = desc.physical_start;
ent.pages = desc.number_of_pages;
ent.type = type;
ent.attr = (desc.attribute & 0xffffffff);
continue;
}
mem_entry &prev = kernel_map[nent - 1];
if (can_merge(prev, type, desc)) {
prev.pages += desc.number_of_pages;
} else {
mem_entry &next = kernel_map[nent++];
next.start = desc.physical_start;
next.pages = desc.number_of_pages;
next.type = type;
next.attr = (desc.attribute & 0xffffffff);
}
}
/*
// kernel map dump
for (unsigned i = 0; i < nent; ++i) {
const mem_entry &e = kernel_map[i];
console::print(L" kRange %lx (%lx) %x(%s) [%lu]\r\n",
e.start, e.attr, e.type, kernel_memory_type_name(e.type), e.pages);
}
*/
return { .pointer = kernel_map, .count = nent };
}
inline size_t bitmap_size(size_t frames) { return (frames + 63) / 64; }
inline size_t num_blocks(size_t frames) { return (frames + (frames_per_block-1)) / frames_per_block; }
util::counted<bootproto::frame_block>
build_frame_blocks(const util::counted<bootproto::mem_entry> &kmap)
{
status_line status {L"Creating kernel frame accounting map"};
size_t block_count = 0;
size_t total_bitmap_size = 0;
for (size_t i = 0; i < kmap.count; ++i) {
const mem_entry &ent = kmap[i];
if (ent.type != mem_type::free)
continue;
block_count += num_blocks(ent.pages);
total_bitmap_size += bitmap_size(ent.pages) * sizeof(uint64_t);
}
size_t total_size = block_count * sizeof(frame_block) + total_bitmap_size;
frame_block *blocks = reinterpret_cast<frame_block*>(
g_alloc.allocate_pages(bytes_to_pages(total_size), alloc_type::frame_map, true));
frame_block *next_block = blocks;
for (size_t i = 0; i < kmap.count; ++i) {
const mem_entry &ent = kmap[i];
if (ent.type != mem_type::free)
continue;
size_t page_count = ent.pages;
uintptr_t base_addr = ent.start;
while (page_count) {
frame_block *blk = next_block++;
blk->flags = static_cast<bootproto::frame_flags>(ent.attr);
blk->base = base_addr;
base_addr += frames_per_block * page_size;
if (page_count >= frames_per_block) {
page_count -= frames_per_block;
blk->count = frames_per_block;
blk->map1 = ~0ull;
g_alloc.memset(blk->map2, sizeof(blk->map2), 0xff);
} else {
blk->count = page_count;
unsigned i = 0;
uint64_t b1 = (page_count + 4095) / 4096;
blk->map1 = (1 << b1) - 1;
uint64_t b2 = (page_count + 63) / 64;
uint64_t b2q = b2 / 64;
uint64_t b2r = b2 % 64;
g_alloc.memset(blk->map2, b2q, 0xff);
blk->map2[b2q] = (1 << b2r) - 1;
break;
}
}
}
uint64_t *bitmap = reinterpret_cast<uint64_t*>(next_block);
for (unsigned i = 0; i < block_count; ++i) {
frame_block &blk = blocks[i];
blk.bitmap = bitmap;
size_t b = blk.count / 64;
size_t r = blk.count % 64;
g_alloc.memset(blk.bitmap, b*8, 0xff);
blk.bitmap[b] = (1 << r) - 1;
bitmap += bitmap_size(blk.count);
}
return { .pointer = blocks, .count = block_count };
}
void
fix_frame_blocks(bootproto::args *args)
{
util::counted<frame_block> &blocks = args->frame_blocks;
size_t size = blocks.count * sizeof(frame_block);
for (unsigned i = 0; i < blocks.count; ++i)
size += bitmap_size(blocks[i].count) * sizeof(uint64_t);
size_t pages = bytes_to_pages(size);
uintptr_t addr = reinterpret_cast<uintptr_t>(blocks.pointer);
// Map the frame blocks to the appropriate address
paging::map_pages(args, addr,
bootproto::mem::bitmap_offset, pages, true, false);
uintptr_t offset = bootproto::mem::bitmap_offset - addr;
for (unsigned i = 0; i < blocks.count; ++i) {
frame_block &blk = blocks[i];
blk.bitmap = util::offset_pointer(blk.bitmap, offset);
}
}
} // namespace memory
} // namespace boot

62
src/boot/memory_map.h
View File

@@ -1,62 +0,0 @@
#pragma once
/// \file memory_map.h
/// Memory-map related types and functions
#include <util/counted.h>
#include <util/pointers.h>
namespace uefi {
struct boot_services;
struct memory_descriptor;
}
namespace bootproto {
struct args;
struct frame_block;
struct mem_entry;
}
namespace boot {
namespace memory {
/// Struct that represents UEFI's memory map. Contains a pointer to the map data
/// as well as the data on how to read it.
struct efi_mem_map
{
using desc = uefi::memory_descriptor;
using iterator = util::offset_iterator<desc>;
size_t length; ///< Total length of the map data
size_t total; ///< Total allocated space for map data
size_t size; ///< Size of an entry in the array
size_t key; ///< Key for detecting changes
uint32_t version; ///< Version of the `memory_descriptor` struct
desc *entries; ///< The array of UEFI descriptors
efi_mem_map() : length(0), total(0), size(0), key(0), version(0), entries(nullptr) {}
/// Update the map from UEFI
void update(uefi::boot_services &bs);
/// Get the count of entries in the array
inline size_t num_entries() const { return length / size; }
/// Return an iterator to the beginning of the array
inline iterator begin() { return iterator(entries, size); }
/// Return an iterator to the end of the array
inline iterator end() { return util::offset_pointer(entries, length); }
};
/// Add the kernel's memory map as a module to the kernel args.
/// \returns The uefi memory map used to build the kernel map
util::counted<bootproto::mem_entry> build_kernel_map(efi_mem_map &map);
/// Create the kernel frame allocation maps
util::counted<bootproto::frame_block> build_frame_blocks(const util::counted<bootproto::mem_entry> &kmap);
/// Map the frame allocation maps to the right spot and fix up pointers
void fix_frame_blocks(bootproto::args *args);
} // namespace boot
} // namespace memory

117
src/boot/paging.cpp
View File

@@ -1,21 +1,19 @@
#include <arch/memory.h> #include "kernel_memory.h"
#include <bootproto/memory.h>
#include <util/counted.h>
#include <util/pointers.h>
#include "allocator.h"
#include "console.h" #include "console.h"
#include "error.h" #include "error.h"
#include "loader.h" #include "loader.h"
#include "memory.h" #include "memory.h"
#include "paging.h" #include "paging.h"
#include "pointer_manipulation.h"
#include "status.h" #include "status.h"
namespace boot { namespace boot {
namespace paging { namespace paging {
using memory::alloc_type;
using memory::page_size; using memory::page_size;
using ::memory::pml4e_kernel;
using ::memory::table_entries;
// Flags: 0 0 0 1 0 0 0 0 0 0 0 1 = 0x0101 // Flags: 0 0 0 1 0 0 0 0 0 0 0 1 = 0x0101
// IGN | | | | | | | | +- Present // IGN | | | | | | | | +- Present
@@ -56,19 +54,6 @@ constexpr uint64_t huge_page_flags = 0x18b;
/// Page table entry flags for entries pointing at another table /// Page table entry flags for entries pointing at another table
constexpr uint64_t table_flags = 0x003; constexpr uint64_t table_flags = 0x003;
inline void *
pop_pages(util::counted<void> &pages, size_t count)
{
if (count > pages.count)
error::raise(uefi::status::out_of_resources, L"Page table cache empty", 0x7ab1e5);
void *next = pages.pointer;
pages.pointer = util::offset_pointer(pages.pointer, count*page_size);
pages.count -= count;
return next;
}
/// Iterator over page table entries. /// Iterator over page table entries.
template <unsigned D = 4> template <unsigned D = 4>
class page_entry_iterator class page_entry_iterator
@@ -77,12 +62,15 @@ public:
/// Constructor. /// Constructor.
/// \arg virt Virtual address this iterator is starting at /// \arg virt Virtual address this iterator is starting at
/// \arg pml4 Root of the page tables to iterate /// \arg pml4 Root of the page tables to iterate
/// \arg pages Cache of usable table pages /// \arg page_cache Pointer to pages that can be used for page tables
/// \arg page_count Number of pages pointed to by `page_cache`
page_entry_iterator( page_entry_iterator(
uintptr_t virt, uintptr_t virt,
page_table *pml4, page_table *pml4,
util::counted<void> &pages) : void *&page_cache,
m_pages(pages) size_t &cache_count) :
m_page_cache(page_cache),
m_cache_count(cache_count)
{ {
m_table[0] = pml4; m_table[0] = pml4;
for (unsigned i = 0; i < D; ++i) { for (unsigned i = 0; i < D; ++i) {
@@ -129,7 +117,12 @@ private:
uint64_t & parent_ent = entry(level - 1); uint64_t & parent_ent = entry(level - 1);
if (!(parent_ent & 1)) { if (!(parent_ent & 1)) {
page_table *table = reinterpret_cast<page_table*>(pop_pages(m_pages, 1)); if (!m_cache_count--)
error::raise(uefi::status::out_of_resources, L"Page table cache empty", 0x7ab1e5);
page_table *table = reinterpret_cast<page_table*>(m_page_cache);
m_page_cache = offset_ptr<void>(m_page_cache, page_size);
parent_ent = (reinterpret_cast<uintptr_t>(table) & ~0xfffull) | table_flags; parent_ent = (reinterpret_cast<uintptr_t>(table) & ~0xfffull) | table_flags;
m_table[level] = table; m_table[level] = table;
} else { } else {
@@ -137,25 +130,29 @@ private:
} }
} }
util::counted<void> &m_pages; void *&m_page_cache;
size_t &m_cache_count;
page_table *m_table[D]; page_table *m_table[D];
uint16_t m_index[D]; uint16_t m_index[D];
}; };
static void static void
add_offset_mappings(page_table *pml4, util::counted<void> &pages) add_offset_mappings(page_table *pml4, void *&page_cache, size_t &num_pages)
{ {
uintptr_t phys = 0; uintptr_t phys = 0;
uintptr_t virt = bootproto::mem::linear_offset; // Start of offset-mapped area uintptr_t virt = ::memory::page_offset; // Start of offset-mapped area
size_t page_count = 64 * 1024; // 64 TiB of 1 GiB pages size_t pages = 64 * 1024; // 64 TiB of 1 GiB pages
constexpr size_t GiB = 0x40000000ull; constexpr size_t GiB = 0x40000000ull;
page_entry_iterator<2> iterator{virt, pml4, pages}; page_entry_iterator<2> iterator{
virt, pml4,
page_cache,
num_pages};
while (true) { while (true) {
*iterator = phys | huge_page_flags; *iterator = phys | huge_page_flags;
if (--page_count == 0) if (--pages == 0)
break; break;
iterator.increment(); iterator.increment();
@@ -164,13 +161,13 @@ add_offset_mappings(page_table *pml4, util::counted<void> &pages)
} }
static void static void
add_kernel_pds(page_table *pml4, util::counted<void> &pages) add_kernel_pds(page_table *pml4, void *&page_cache, size_t &num_pages)
{ {
constexpr unsigned start = arch::kernel_root_index; for (unsigned i = pml4e_kernel; i < table_entries; ++i) {
constexpr unsigned end = arch::table_entries; pml4->set(i, page_cache, table_flags);
page_cache = offset_ptr<void>(page_cache, page_size);
for (unsigned i = start; i < end; ++i) num_pages--;
pml4->set(i, pop_pages(pages, 1), table_flags); }
} }
void void
@@ -181,8 +178,7 @@ add_current_mappings(page_table *new_pml4)
asm volatile ( "mov %%cr3, %0" : "=r" (old_pml4) ); asm volatile ( "mov %%cr3, %0" : "=r" (old_pml4) );
// Only copy mappings in the lower half // Only copy mappings in the lower half
constexpr unsigned halfway = arch::kernel_root_index; for (int i = 0; i < ::memory::pml4e_kernel; ++i) {
for (int i = 0; i < halfway; ++i) {
uint64_t entry = old_pml4->entries[i]; uint64_t entry = old_pml4->entries[i];
if (entry & 1) if (entry & 1)
new_pml4->entries[i] = entry; new_pml4->entries[i] = entry;
@@ -190,7 +186,7 @@ add_current_mappings(page_table *new_pml4)
} }
void void
allocate_tables(bootproto::args *args) allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
{ {
status_line status(L"Allocating initial page tables"); status_line status(L"Allocating initial page tables");
@@ -202,16 +198,28 @@ allocate_tables(bootproto::args *args)
static constexpr size_t tables_needed = kernel_tables + extra_tables; static constexpr size_t tables_needed = kernel_tables + extra_tables;
void *addr = g_alloc.allocate_pages(tables_needed, alloc_type::page_table, true); void *addr = nullptr;
try_or_raise(
bs->allocate_pages(
uefi::allocate_type::any_pages,
uefi::memory_type::loader_data,
tables_needed,
&addr),
L"Error allocating page table pages.");
bs->set_mem(addr, tables_needed*page_size, 0);
page_table *pml4 = reinterpret_cast<page_table*>(addr); page_table *pml4 = reinterpret_cast<page_table*>(addr);
args->pml4 = pml4; args->pml4 = pml4;
args->page_tables = { .pointer = pml4 + 1, .count = tables_needed - 1 }; args->table_pages = tables_needed;
args->table_count = tables_needed - 1;
args->page_tables = offset_ptr<void>(addr, page_size);
console::print(L" First page (pml4) at: 0x%lx\r\n", pml4); console::print(L" First page (pml4) at: 0x%lx\r\n", pml4);
add_kernel_pds(pml4, args->page_tables); add_kernel_pds(pml4, args->page_tables, args->table_count);
add_offset_mappings(pml4, args->page_tables); add_offset_mappings(pml4, args->page_tables, args->table_count);
//console::print(L" Set up initial mappings, %d spare tables.\r\n", args->table_count); //console::print(L" Set up initial mappings, %d spare tables.\r\n", args->table_count);
} }
@@ -225,7 +233,7 @@ constexpr bool has_flag(E set, E flag) {
void void
map_pages( map_pages(
bootproto::args *args, kernel::args::header *args,
uintptr_t phys, uintptr_t virt, uintptr_t phys, uintptr_t virt,
size_t count, bool write_flag, bool exe_flag) size_t count, bool write_flag, bool exe_flag)
{ {
@@ -235,7 +243,10 @@ map_pages(
paging::page_table *pml4 = paging::page_table *pml4 =
reinterpret_cast<paging::page_table*>(args->pml4); reinterpret_cast<paging::page_table*>(args->pml4);
page_entry_iterator<4> iterator{virt, pml4, args->page_tables}; page_entry_iterator<4> iterator{
virt, pml4,
args->page_tables,
args->table_count};
uint64_t flags = page_flags; uint64_t flags = page_flags;
if (!exe_flag) if (!exe_flag)
@@ -255,28 +266,22 @@ map_pages(
void void
map_section( map_section(
bootproto::args *args, kernel::args::header *args,
const bootproto::program_section &section) const kernel::args::program_section &section)
{ {
using bootproto::section_flags; using kernel::args::section_flags;
size_t pages = memory::bytes_to_pages(section.size);
map_pages( map_pages(
args, args,
section.phys_addr, section.phys_addr,
section.virt_addr, section.virt_addr,
memory::bytes_to_pages(section.size), pages,
has_flag(section.type, section_flags::write), has_flag(section.type, section_flags::write),
has_flag(section.type, section_flags::execute)); has_flag(section.type, section_flags::execute));
} }
void
map_program(
bootproto::args *args,
bootproto::program &program)
{
for (auto &section : program.sections)
paging::map_section(args, section);
}
} // namespace paging } // namespace paging
} // namespace boot } // namespace boot

Some files were not shown because too many files have changed in this diff Show More