[docs] Update the README with roadmap info

I had failed to specify in inline asm that an input variable was the
same as the output variable.

.gitignore

@@ -3,8 +3,10 @@
 *.bak
 tags
 jsix.log
+*.out
 *.o
 *.a
 sysroot
 .gdb_history
 .peru
+__pycache__

README.md

@@ -1,15 +1,18 @@
-# jsix: A toy OS kernel
+![jsix](assets/jsix.svg)
 
-**jsix** is the kernel for the hobby OS that I am currently building. It's
-far from finished, or even being usable. Instead, it's a sandbox for me to play
-with kernel-level code and explore architectures.
+# The jsix operating system
+
+**jsix** is a custom multi-core x64 operating system that I am building from
+scratch. It's far from finished, or even being usable - see the *Status and
+Roadmap* section, below.
 
 The design goals of the project are:
 
 * Modernity - I'm not interested in designing for legacy systems, or running on
   all hardware out there. My target is only 64 bit architecutres, and modern
   commodity hardware. Currently that means x64 systems with Nehalem or newer
-  CPUs and UEFI firmware. Eventually I'd like to work on an AArch64 port,
+  CPUs and UEFI firmware. (See [this list][cpu_features] for the currently
+  required CPU features.) Eventually I'd like to work on an AArch64 port,
   partly to force myself to factor out the architecture-dependent pieces of the
   code base.
 
@@ -17,13 +20,11 @@ The design goals of the project are:
   processes as possible, in the microkernel fashion. A sub-goal of this is to
   explore where the bottlenecks of such a microkernel are now, and whether
   eschewing legacy hardware will let me design a system that's less bogged down
-  by the traditional microkernel problems. Given that there are no processes
-  yet, the kernel is monolithic by default.
+  by the traditional microkernel problems.
 
 * Exploration - I'm really mostly doing this to have fun learning and exploring
-  modern OS development. Modular design may be tossed out (hopefully
-  temporarily) in some places to allow me to play around with the related
-  hardware.
+  modern OS development. Initial feature implementations may temporarily throw
+  away modular design to allow for exploration of the related hardware.
 
 A note on the name: This kernel was originally named Popcorn, but I have since
 discovered that the Popcorn Linux project is also developing a kernel with that
@@ -31,6 +32,69 @@ name, started around the same time as this project. So I've renamed this kernel
 jsix (Always styled _jsix_ or `j6`, never capitalized) as an homage to L4, xv6,
 and my wonderful wife.
 
+[cpu_features]: https://github.com/justinian/jsix/blob/master/src/libraries/cpu/include/cpu/features.inc
+
+## Status and Roadmap
+
+The following major feature areas are targets for jsix development:
+
+#### UEFI boot loader
+
+_Done._ The bootloader loads the kernel and initial userspace programs, and
+sets up necessary kernel arguments about the memory map and EFI GOP
+framebuffer. Possible future ideas:
+
+- take over more init-time functions from the kernel
+- rewrite it in Zig
+
+#### Memory
+
+_Virtual memory: Sufficient._ The kernel manages virtual memory with a number
+of kinds of `vm_area` objects representing mapped areas, which can belong to
+one or more `vm_space` objects which represent a whole virtual memory space.
+(Each process has a `vm_space`, and so does the kernel itself.)
+
+Remaining to do:
+
+- TLB shootdowns
+- Page swapping
+
+_Physical page allocation: Sufficient._ The current physical page allocator
+implementation suses a group of block representing up-to-1GiB areas of usable
+memory as defined by the bootloader. Each block has a three-level bitmap
+denoting free/used pages.
+
+#### Multitasking
+
+_Sufficient._ The global scheduler object keeps separate ready/blocked lists
+per core. Cores periodically attempt to balance load via work stealing.
+
+User-space tasks are able to create threads as well as other processes.
+
+Several kernel-only tasks exist, though I'm trying to reduce that. Eventually
+only the timekeeping task should be a separate kernel-only thread.
+
+#### API
+
+_In progress._ User-space tasks are able to make syscalls to the kernel via
+fast SYSCALL/SYSRET instructions.
+
+Major tasks still to do:
+
+- The process initialization protocol needs to be re-built entirely.
+- Processes' handles to kernel objects need the ability to check capabilities
+
+#### Hardware Support
+
+  * Framebuffer driver: _In progress._ Currently on machines with a video
+	device accessible by UEFI, jsix starts a user-space framebuffer driver that
+	only prints out kernel logs.
+  * Serial driver: _To do._ Machines without a video device should have a
+	user-space log output task like the framebuffer driver, but currently this
+	is done inside the kernel.
+  * USB driver: _To do_
+  * AHCI (SATA) driver: _To do_
+
 ## Building
 
 jsix uses the [Ninja][] build tool, and generates the build files for it with a
@@ -38,7 +102,7 @@ custom tool called [Bonnibel][]. Bonnibel can be installed with [Cargo][], or
 downloaded as a prebuilt binary from its Github repository.
 
 [Ninja]:    https://ninja-build.org
-[Bonnibel]: https://github.com/justinian/bonnibel
+[Bonnibel]: https://github.com/justinian/bonnibel_rs
 [Cargo]:    https://crates.io/crates/bonnibel
 
 Requrirements:
@@ -71,8 +135,8 @@ I personally run this either from a real debian amd64 testing/buster machine or
 a windows WSL debian testing/buster installation. The following should be
 enough to set up such a system to build the kernel:
 
-    sudo apt install qemu-system-x86 nasm clang-6.0 mtools curl
-    sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-6.0 1000
-    sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-6.0 1000
-	curl -L -o pb https://github.com/justinian/bonnibel/releases/download/2.0.0/pb_linux_amd64 && chmod a+x pb
+    sudo apt install qemu-system-x86 nasm clang-10 mtools curl ninja-build
+    sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-10 1000
+    sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-10 1000
+	curl -L -o pb https://github.com/justinian/bonnibel_rs/releases/download/v2.3.0/pb-linux-amd64 && chmod a+x pb
 

assets/debugging/jsix.elf-gdb.py

@@ -16,10 +16,14 @@ class PrintStackCommand(gdb.Command):
             depth = int(args[1])
 
         for i in range(depth-1, -1, -1):
-            offset = i * 8
-            base_addr = gdb.parse_and_eval(base)
-            value = gdb.parse_and_eval(f"*(uint64_t*)({base} + {offset})")
-            print("{:016x} (+{:04x}): {:016x}".format(int(base_addr) + offset, offset, int(value)))
+            try:
+                offset = i * 8
+                base_addr = gdb.parse_and_eval(base)
+                value = gdb.parse_and_eval(f"*(uint64_t*)({base} + {offset})")
+                print("{:016x} (+{:04x}): {:016x}".format(int(base_addr) + offset, offset, int(value)))
+            except Exception as e:
+                print(e)
+                continue
 
 
 class PrintBacktraceCommand(gdb.Command):
@@ -29,13 +33,13 @@ class PrintBacktraceCommand(gdb.Command):
     def invoke(self, arg, from_tty):
         args = gdb.string_to_argv(arg)
 
-        frame = "$rbp"
-        if len(args) > 0:
-            frame = args[0]
-
         depth = 30
+        if len(args) > 0:
+            depth = int(args[0])
+
+        frame = "$rbp"
         if len(args) > 1:
-            depth = int(args[1])
+            frame = args[1]
 
         for i in range(depth-1, -1, -1):
             ret = gdb.parse_and_eval(f"*(uint64_t*)({frame} + 8)")
@@ -52,8 +56,77 @@ class PrintBacktraceCommand(gdb.Command):
                 return
 
 
+class TableWalkCommand(gdb.Command):
+    def __init__(self):
+        super().__init__("j6tw", gdb.COMMAND_DATA)
+
+    def invoke(self, arg, from_tty):
+        args = gdb.string_to_argv(arg)
+        if len(args) < 2:
+            raise Exception("Must be: j6tw <pml4> <addr>")
+
+        pml4 = int(gdb.parse_and_eval(args[0]))
+        addr = int(gdb.parse_and_eval(args[1]))
+
+        indices = [
+            (addr >> 39) & 0x1ff,
+            (addr >> 30) & 0x1ff,
+            (addr >> 21) & 0x1ff,
+            (addr >> 12) & 0x1ff,
+            ]
+
+        names = ["PML4", "PDP", "PD", "PT"]
+
+        table_flags = [
+            (0x0001, "present"),
+            (0x0002, "write"),
+            (0x0004, "user"),
+            (0x0008, "pwt"),
+            (0x0010, "pcd"),
+            (0x0020, "accessed"),
+            (0x0040, "dirty"),
+            (0x0080, "largepage"),
+            (0x0100, "global"),
+            (0x1080, "pat"),
+            ((1<<63), "xd"),
+        ]
+
+        page_flags = [
+            (0x0001, "present"),
+            (0x0002, "write"),
+            (0x0004, "user"),
+            (0x0008, "pwt"),
+            (0x0010, "pcd"),
+            (0x0020, "accessed"),
+            (0x0040, "dirty"),
+            (0x0080, "pat"),
+            (0x0100, "global"),
+            ((1<<63), "xd"),
+        ]
+
+        flagsets = [table_flags, table_flags, table_flags, page_flags]
+
+        table = pml4
+        entry = 0
+        for i in range(len(indices)):
+            entry = int(gdb.parse_and_eval(f'((uint64_t*){table})[{indices[i]}]'))
+            flagset = flagsets[i]
+            flag_names = " | ".join([f[1] for f in flagset if (entry & f[0]) == f[0]])
+
+            print(f"{names[i]:>4}: {table:016x}")
+            print(f"      index: {indices[i]:3} {entry:016x}")
+            print(f"      flags: {flag_names}")
+
+            if (entry & 1) == 0 or (i < 3 and (entry & 0x80)):
+                break
+
+            table = (entry & 0x7ffffffffffffe00) | 0xffffc00000000000
+
+
+
 PrintStackCommand()
 PrintBacktraceCommand()
+TableWalkCommand()
 
 gdb.execute("target remote :1234")
 gdb.execute("display/i $rip")

assets/jsix.svg

@@ -0,0 +1 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?><!-- Generator: Gravit.io --><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" style="isolation:isolate" viewBox="176.562 356.069 211.11 113" width="211.11pt" height="113pt"><g><g><rect x="176.562" y="356.069" width="211.11" height="113" transform="matrix(1,0,0,1,0,0)" fill="rgb(255,255,255)"/><g><path d=" M 212.981 372.36 L 219.564 376.16 L 226.147 379.961 L 226.147 387.563 L 226.147 395.164 L 219.564 398.965 L 212.981 402.766 L 206.398 398.965 L 199.815 395.164 L 199.815 387.563 L 199.815 379.961 L 206.398 376.16 L 212.981 372.36 L 212.981 372.36 L 212.981 372.36 Z  M 256.292 397.366 L 262.875 401.166 L 269.458 404.967 L 269.458 412.569 L 269.458 420.17 L 262.875 423.971 L 256.292 427.772 L 249.709 423.971 L 243.126 420.17 L 243.126 412.569 L 243.126 404.967 L 249.709 401.166 L 256.292 397.366 L 256.292 397.366 Z  M 183.622 387.283 L 205.52 374.64 L 227.418 361.997 L 249.316 374.64 L 271.214 387.283 L 271.214 412.569 L 271.214 437.854 L 249.316 450.497 L 227.418 463.14 L 205.52 450.497 L 183.622 437.854 L 183.622 412.569 L 183.622 387.283 L 183.622 387.283 L 183.622 387.283 Z  M 241.855 372.36 L 248.438 376.16 L 255.021 379.961 L 255.021 387.563 L 255.021 395.164 L 248.438 398.965 L 241.855 402.766 L 235.272 398.965 L 228.689 395.164 L 228.689 387.563 L 228.689 379.961 L 235.272 376.16 L 241.855 372.36 Z " fill-rule="evenodd" fill="rgb(49,79,128)"/><path d=" M 298.642 379.579 L 291.621 379.579 L 291.621 372.558 L 298.642 372.558 L 298.642 379.579 Z  M 285.214 446.718 L 285.214 441.452 L 287.32 441.452 L 287.32 441.452 Q 289.339 441.452 290.524 440.092 L 290.524 440.092 L 290.524 440.092 Q 291.708 438.731 291.708 436.625 L 291.708 436.625 L 291.708 387.039 L 298.729 387.039 L 298.729 436.011 L 298.729 436.011 Q 298.729 440.925 295.921 443.822 L 295.921 443.822 L 295.921 443.822 Q 293.113 446.718 288.286 446.718 L 288.286 446.718 L 285.214 446.718 Z  M 306.628 432.676 L 306.628 427.41 L 314.088 427.41 L 314.088 427.41 Q 317.862 427.41 319.573 425.347 L 319.573 425.347 L 319.573 425.347 Q 321.285 423.285 321.285 419.95 L 321.285 419.95 L 321.285 419.95 Q 321.285 417.317 319.705 415.474 L 319.705 415.474 L 319.705 415.474 Q 318.125 413.631 314.966 411.174 L 314.966 411.174 L 314.966 411.174 Q 312.245 408.98 310.621 407.356 L 310.621 407.356 L 310.621 407.356 Q 308.998 405.732 307.813 403.319 L 307.813 403.319 L 307.813 403.319 Q 306.628 400.905 306.628 397.746 L 306.628 397.746 L 306.628 397.746 Q 306.628 393.095 309.744 390.067 L 309.744 390.067 L 309.744 390.067 Q 312.859 387.039 318.125 387.039 L 318.125 387.039 L 325.76 387.039 L 325.76 392.305 L 319.441 392.305 L 319.441 392.305 Q 313.21 392.305 313.21 398.185 L 313.21 398.185 L 313.21 398.185 Q 313.21 400.467 314.615 402.134 L 314.615 402.134 L 314.615 402.134 Q 316.019 403.802 319.003 406.083 L 319.003 406.083 L 319.003 406.083 Q 321.723 408.19 323.479 409.901 L 323.479 409.901 L 323.479 409.901 Q 325.234 411.613 326.463 414.202 L 326.463 414.202 L 326.463 414.202 Q 327.691 416.791 327.691 420.301 L 327.691 420.301 L 327.691 420.301 Q 327.691 426.532 324.4 429.604 L 324.4 429.604 L 324.4 429.604 Q 321.109 432.676 315.141 432.676 L 315.141 432.676 L 306.628 432.676 Z  M 342.611 379.579 L 335.59 379.579 L 335.59 372.558 L 342.611 372.558 L 342.611 379.579 Z  M 342.611 432.676 L 335.59 432.676 L 335.59 387.039 L 342.611 387.039 L 342.611 432.676 Z  M 356.126 432.676 L 348.754 432.676 L 361.392 409.77 L 349.632 387.039 L 356.39 387.039 L 364.639 403.187 L 372.977 387.039 L 379.735 387.039 L 367.974 409.77 L 380.612 432.676 L 373.24 432.676 L 364.639 416.001 L 356.126 432.676 Z " fill="rgb(49,79,128)"/></g></g></g></svg>

external/uefi/boot_services.h

@@ -14,8 +14,10 @@
 namespace uefi {
 namespace bs_impl {
 	using allocate_pages = status (*)(allocate_type, memory_type, size_t, void**);
+	using free_pages = status (*)(void*, size_t);
 	using get_memory_map = status (*)(size_t*, memory_descriptor*, size_t*, size_t*, uint32_t*);
 	using allocate_pool = status (*)(memory_type, uint64_t, void**);
+	using free_pool = status (*)(void*);
 	using handle_protocol = status (*)(handle, const guid*, void**);
 	using create_event = status (*)(evt, tpl, event_notify, void*, event*);
 	using exit_boot_services = status (*)(handle, size_t);
@@ -35,10 +37,10 @@ struct boot_services {
 
 	// Memory Services
 	bs_impl::allocate_pages allocate_pages;
-	void *free_pages;
+	bs_impl::free_pages free_pages;
 	bs_impl::get_memory_map get_memory_map;
 	bs_impl::allocate_pool allocate_pool;
-	void *free_pool;
+	bs_impl::free_pool free_pool;
 
 	// Event & Timer Services
 	bs_impl::create_event create_event;

external/uefi/networking.h

@@ -0,0 +1,390 @@
+#pragma once
+#ifndef _uefi_networking_h_
+#define _uefi_networking_h_
+
+// This Source Code Form is part of the j6-uefi-headers project and is subject
+// to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was
+// not distributed with this file, You can obtain one at
+// http://mozilla.org/MPL/2.0/.
+
+#include <stdint.h>
+#include <uefi/types.h>
+
+namespace uefi {
+
+//
+// IPv4 definitions
+//
+struct ipv4_address
+{
+	uint8_t addr[4];
+};
+
+//
+// IPv6 definitions
+//
+struct ipv6_address
+{
+	uint8_t addr[16];
+};
+
+struct ip6_address_info
+{
+    ipv6_address address;
+    uint8_t prefix_length;
+};
+
+struct ip6_route_table
+{
+    ipv6_address gateway;
+    ipv6_address destination;
+    uint8_t prefix_length;
+};
+
+enum class ip6_neighbor_state : int {
+    incomplete,
+    reachable,
+    stale,
+    delay,
+    probe,
+}
+
+struct ip6_neighbor_cache
+{
+    ipv6_address neighbor;
+    mac_address link_address;
+    ip6_neighbor_state state;
+};
+
+enum class icmpv6_type : uint8_t
+{
+    dest_unreachable = 0x1,
+    packet_too_big = 0x2,
+    time_exceeded = 0x3,
+    parameter_problem = 0x4,
+    echo_request = 0x80,
+    echo_reply = 0x81,
+    listener_query = 0x82,
+    listener_report = 0x83,
+    listener_done = 0x84,
+    router_solicit = 0x85,
+    router_advertise = 0x86,
+    neighbor_solicit = 0x87,
+    neighbor_advertise = 0x88,
+    redirect = 0x89,
+    listener_report_2 = 0x8f,
+};
+
+enum class icmpv6_code : uint8_t
+{
+    // codes for icmpv6_type::dest_unreachable
+    no_route_to_dest = 0x0,
+    comm_prohibited = 0x1,
+    beyond_scope = 0x2,
+    addr_unreachable = 0x3,
+    port_unreachable = 0x4,
+    source_addr_failed = 0x5,
+    route_rejected = 0x6,
+
+    // codes for icmpv6_type::time_exceeded
+    timeout_hop_limit = 0x0,
+    timeout_reassemble = 0x1,
+
+    // codes for icmpv6_type::parameter_problem
+    erroneous_header = 0x0,
+    unrecognize_next_hdr = 0x1,
+    unrecognize_option = 0x2,
+};
+
+struct ip6_icmp_type
+{
+    icmpv6_type type;
+    icmpv6_code code;
+};
+
+struct ip6_config_data
+{
+    uint8_t default_protocol;
+    bool accept_any_protocol;
+    bool accept_icmp_errors;
+    bool accept_promiscuous;
+    ipv6_address destination_address;
+    ipv6_address station_address;
+    uint8_t traffic_class;
+    uint8_t hop_limit;
+    uint32_t flow_label;
+    uint32_t receive_timeout;
+    uint32_t transmit_timeout;
+};
+
+struct ip6_mode_data
+{
+    bool is_started;
+    uint32_t max_packet_size;
+    ip6_config_data config_data;
+    bool is_configured;
+    uint32_t address_count;
+    ip6_address_info * address_list;
+    uint32_t group_count;
+    ipv6_address * group_table;
+    uint32_t route_count;
+    ip6_route_table * route_table;
+    uint32_t neighbor_count;
+    ip6_neighbor_cache * neighbor_cache;
+    uint32_t prefix_count;
+    ip6_address_info * prefix_table;
+    uint32_t icmp_type_count;
+     * icmp_type_list;
+};
+
+struct ip6_header
+{
+    uint8_t traffic_class_h : 4;
+    uint8_t version : 4;
+    uint8_t flow_label_h : 4;
+    uint8_t traffic_class_l : 4;
+    uint16_t flow_label_l;
+    uint16_t payload_length;
+    uint8_t next_header;
+    uint8_t hop_limit;
+    ipv6_address source_address;
+    ipv6_address destination_address;
+} __attribute__ ((packed));
+
+struct ip6_fragment_data
+{
+    uint32_t fragment_length;
+    void *fragment_buffer;
+};
+
+struct ip6_override_data
+{
+    uint8_t protocol;
+    uint8_t hop_limit;
+    uint32_t flow_label;
+};
+
+struct ip6_receive_data
+{
+    time time_stamp;
+    event recycle_signal;
+    uint32_t header_length;
+    ip6_header *header;
+    uint32_t data_length;
+    uint32_t fragment_count;
+    ip6_fragment_data fragment_table[1];
+};
+
+struct ip6_transmit_data
+{
+    ipv6_address destination_address;
+    ip6_override_data *override_data;
+
+    uint32_t ext_hdrs_length;
+    void *ext_hdrs;
+    uint8_t next_header;
+    uint32_t data_length;
+    uint32_t fragment_count;
+    ip6_fragment_data fragment_table[1];
+};
+
+struct ip6_completion_token
+{
+    event event;
+    status status;
+    union {
+        ip6_receive_data *rx_data;
+        ip6_transmit_data *tx_data;
+    } packet;
+};
+
+enum class ip6_config_data_type : int
+{
+    interface_info,
+    alt_interface_id,
+    policy,
+    dup_addr_detect_transmits,
+    manual_address,
+    gateway,
+    dns_server,
+    maximum
+};
+
+struct ip6_config_interface_info
+{
+    wchar_t name[32];
+    uint8_t if_type;
+    uint32_t hw_address_size;
+    mac_address hw_address;
+    uint32_t address_info_count;
+    ip6_address_info *address_info;
+    uint32_t route_count;
+    ip6_route_table *route_table;
+};
+
+struct ip6_config_interface_id
+{
+    uint8_t id[8];
+};
+
+enum class ip6_config_policy : int
+{
+    manual,
+    automatic
+};
+
+struct ip6_config_dup_addr_detect_transmits
+{
+    uint32_t dup_addr_detect_transmits;
+};
+
+struct ip6_config_manual_address
+{
+    ipv6_address address;
+    bool is_anycast;
+    uint8_t prefix_length;
+};
+
+//
+// IP definitions
+//
+union ip_address
+{
+	uint8_t addr[4];
+	ipv4_address v4;
+	ipv6_address v6;
+};
+
+//
+// HTTP definitions
+//
+struct httpv4_access_point
+{
+	bool use_default_address;
+	ipv4_address local_address;
+	ipv4_address local_subnet;
+	uint16_t local_port;
+};
+
+struct httpv6_access_point
+{
+	ipv6_address local_address;
+	uint16_t local_port;
+};
+
+enum class http_version : int {
+	v10,
+	v11,
+	unsupported,
+};
+
+struct http_config_data
+{
+	http_version http_version;
+	uint32_t time_out_millisec;
+	bool local_address_is_ipv6;
+	union {
+		httpv4_access_point *ipv4_node;
+		httpv6_access_point *ipv6_node;
+	} access_point;
+};
+
+enum class http_method : int {
+	get,
+	post,
+	patch,
+	options,
+	connect,
+	head,
+	put,
+	delete_,
+	trace,
+};
+
+struct http_request_data
+{
+	http_method method;
+	wchar_t *url;
+};
+
+enum class http_status_code : int {
+	unsupported,
+	continue_,
+	switching_protocols,
+	ok,
+	created,
+	accepted,
+	non_authoritative_information,
+	no_content,
+	reset_content,
+	partial_content,
+	multiple_choices,
+	moved_permanently,
+	found,
+	see_other,
+	not_modified,
+	use_proxy,
+	temporary_redirect,
+	bad_request,
+	unauthorized,
+	payment_required,
+	forbidden,
+	not_found,
+	method_not_allowed,
+	not_acceptable,
+	proxy_authentication_required,
+	request_time_out,
+	conflict,
+	gone,
+	length_required,
+	precondition_failed,
+	request_entity_too_large,
+	request_uri_too_large,
+	unsupported_media_type,
+	requested_range_not_satisfied,
+	expectation_failed,
+	internal_server_error,
+	not_implemented,
+	bad_gateway,
+	service_unavailable,
+	gateway_timeout,
+	http_version_not_supported,
+	permanent_redirect,  // I hate your decisions, uefi.
+};
+
+struct http_response_data
+{
+	http_status_code status_code;
+};
+
+struct http_header
+{
+	char *field_name;
+	char *field_value;
+};
+
+struct http_message
+{
+	union {
+		http_request_data *request;
+		http_response_data *response;
+	} data;
+
+	size_t header_count;
+	http_header *headers;
+
+	size_t body_length;
+	void *body;
+};
+
+struct http_token
+{
+	event event;
+	status status;
+	http_message *message;
+};
+
+
+} // namespace uefi
+
+#endif

external/uefi/protos/device_path.h

@@ -16,6 +16,7 @@ struct device_path
 {
     static constexpr uefi::guid guid{ 0x09576e91,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
+
     uint8_t type;
     uint8_t sub_type;
     uint16_t length;
@@ -27,4 +28,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_device_path_h_
+#endif // _uefi_protos_device_path_h_

external/uefi/protos/file.h

@@ -14,7 +14,7 @@ struct file;
 
 struct file
 {
-    
+
 
     inline uefi::status open(file ** new_handle, const wchar_t * file_name, file_mode open_mode, file_attr attributes) {
         return _open(this, new_handle, file_name, open_mode, attributes);
@@ -123,4 +123,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_file_h_
+#endif // _uefi_protos_file_h_

external/uefi/protos/file_info.h

@@ -16,6 +16,7 @@ struct file_info
 {
     static constexpr uefi::guid guid{ 0x09576e92,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
+
     uint64_t size;
     uint64_t file_size;
     uint64_t physical_size;
@@ -32,4 +33,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_file_info_h_
+#endif // _uefi_protos_file_info_h_

external/uefi/protos/graphics_output.h

@@ -17,6 +17,7 @@ struct graphics_output
 {
     static constexpr uefi::guid guid{ 0x9042a9de,0x23dc,0x4a38,{0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a} };
 
+
     inline uefi::status query_mode(uint32_t mode_number, uint64_t * size_of_info, uefi::graphics_output_mode_info ** info) {
         return _query_mode(this, mode_number, size_of_info, info);
     }
@@ -47,4 +48,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_graphics_output_h_
+#endif // _uefi_protos_graphics_output_h_

external/uefi/protos/http.h

@@ -0,0 +1,72 @@
+#pragma once
+#ifndef _uefi_protos_http_h_
+#define _uefi_protos_http_h_
+
+// This file was auto generated by the j6-uefi-headers project. Please see
+// https://github.com/justinian/j6-uefi-headers for more information.
+
+#include <uefi/guid.h>
+#include <uefi/types.h>
+#include <uefi/networking.h>
+
+namespace uefi {
+namespace protos {
+struct http;
+
+struct http
+{
+    static constexpr uefi::guid guid{ 0x7a59b29b,0x910b,0x4171,{0x82,0x42,0xa8,0x5a,0x0d,0xf2,0x5b,0x5b} };
+    static constexpr uefi::guid service_binding{ 0xbdc8e6af,0xd9bc,0x4379,{0xa7,0x2a,0xe0,0xc4,0xe7,0x5d,0xae,0x1c} };
+
+
+    inline uefi::status get_mode_data(uefi::http_config_data * http_config_data) {
+        return _get_mode_data(this, http_config_data);
+    }
+
+    inline uefi::status configure(uefi::http_config_data * http_config_data) {
+        return _configure(this, http_config_data);
+    }
+
+    inline uefi::status request(uefi::http_token * token) {
+        return _request(this, token);
+    }
+
+    inline uefi::status cancel(uefi::http_token * token) {
+        return _cancel(this, token);
+    }
+
+    inline uefi::status response(uefi::http_token * token) {
+        return _response(this, token);
+    }
+
+    inline uefi::status poll() {
+        return _poll(this);
+    }
+
+
+protected:
+    using _get_mode_data_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
+    _get_mode_data_def _get_mode_data;
+
+    using _configure_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
+    _configure_def _configure;
+
+    using _request_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
+    _request_def _request;
+
+    using _cancel_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
+    _cancel_def _cancel;
+
+    using _response_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
+    _response_def _response;
+
+    using _poll_def = uefi::status (*)(uefi::protos::http *);
+    _poll_def _poll;
+
+public:
+};
+
+} // namespace protos
+} // namespace uefi
+
+#endif // _uefi_protos_http_h_

external/uefi/protos/ip6.h

@@ -0,0 +1,93 @@
+#pragma once
+#ifndef _uefi_protos_ip6_h_
+#define _uefi_protos_ip6_h_
+
+// This file was auto generated by the j6-uefi-headers project. Please see
+// https://github.com/justinian/j6-uefi-headers for more information.
+
+#include <uefi/guid.h>
+#include <uefi/types.h>
+#include <uefi/networking.h>
+
+namespace uefi {
+namespace protos {
+struct ip6;
+
+struct ip6
+{
+    static constexpr uefi::guid guid{ 0x2c8759d5,0x5c2d,0x66ef,{0x92,0x5f,0xb6,0x6c,0x10,0x19,0x57,0xe2} };
+    static constexpr uefi::guid service_binding{ 0xec835dd3,0xfe0f,0x617b,{0xa6,0x21,0xb3,0x50,0xc3,0xe1,0x33,0x88} };
+
+
+    inline uefi::status get_mode_data(uefi::ip6_mode_data * ip6_mode_data, uefi::managed_network_config_data * mnp_config_data, uefi::simple_network_mode * snp_config_data) {
+        return _get_mode_data(this, ip6_mode_data, mnp_config_data, snp_config_data);
+    }
+
+    inline uefi::status configure(uefi::ip6_config_data * ip6_config_data) {
+        return _configure(this, ip6_config_data);
+    }
+
+    inline uefi::status groups(bool join_flag, uefi::ipv6_address * group_address) {
+        return _groups(this, join_flag, group_address);
+    }
+
+    inline uefi::status routes(bool delete_route, uefi::ipv6_address * destination, uint8_t prefix_length, uefi::ipv6_address * gateway_address) {
+        return _routes(this, delete_route, destination, prefix_length, gateway_address);
+    }
+
+    inline uefi::status neighbors(bool delete_flag, uefi::ipv6_address * target_ip6_address, uefi::mac_address * target_link_address, uint32_t timeout, bool override) {
+        return _neighbors(this, delete_flag, target_ip6_address, target_link_address, timeout, override);
+    }
+
+    inline uefi::status transmit(uefi::ip6_completion_token * token) {
+        return _transmit(this, token);
+    }
+
+    inline uefi::status receive(uefi::ip6_completion_token * token) {
+        return _receive(this, token);
+    }
+
+    inline uefi::status cancel(uefi::ip6_completion_token * token) {
+        return _cancel(this, token);
+    }
+
+    inline uefi::status poll() {
+        return _poll(this);
+    }
+
+
+protected:
+    using _get_mode_data_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_mode_data *, uefi::managed_network_config_data *, uefi::simple_network_mode *);
+    _get_mode_data_def _get_mode_data;
+
+    using _configure_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_config_data *);
+    _configure_def _configure;
+
+    using _groups_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *);
+    _groups_def _groups;
+
+    using _routes_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uint8_t, uefi::ipv6_address *);
+    _routes_def _routes;
+
+    using _neighbors_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uefi::mac_address *, uint32_t, bool);
+    _neighbors_def _neighbors;
+
+    using _transmit_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
+    _transmit_def _transmit;
+
+    using _receive_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
+    _receive_def _receive;
+
+    using _cancel_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
+    _cancel_def _cancel;
+
+    using _poll_def = uefi::status (*)(uefi::protos::ip6 *);
+    _poll_def _poll;
+
+public:
+};
+
+} // namespace protos
+} // namespace uefi
+
+#endif // _uefi_protos_ip6_h_

external/uefi/protos/ip6_config.h

@@ -0,0 +1,57 @@
+#pragma once
+#ifndef _uefi_protos_ip6_config_h_
+#define _uefi_protos_ip6_config_h_
+
+// This file was auto generated by the j6-uefi-headers project. Please see
+// https://github.com/justinian/j6-uefi-headers for more information.
+
+#include <uefi/guid.h>
+#include <uefi/types.h>
+#include <uefi/networking.h>
+
+namespace uefi {
+namespace protos {
+struct ip6_config;
+
+struct ip6_config
+{
+    static constexpr uefi::guid guid{ 0x937fe521,0x95ae,0x4d1a,{0x89,0x29,0x48,0xbc,0xd9,0x0a,0xd3,0x1a} };
+
+
+    inline uefi::status set_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
+        return _set_data(this, data_type, data_size, data);
+    }
+
+    inline uefi::status get_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
+        return _get_data(this, data_type, data_size, data);
+    }
+
+    inline uefi::status register_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
+        return _register_data_notify(this, data_type, event);
+    }
+
+    inline uefi::status unregister_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
+        return _unregister_data_notify(this, data_type, event);
+    }
+
+
+protected:
+    using _set_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
+    _set_data_def _set_data;
+
+    using _get_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
+    _get_data_def _get_data;
+
+    using _register_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
+    _register_data_notify_def _register_data_notify;
+
+    using _unregister_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
+    _unregister_data_notify_def _unregister_data_notify;
+
+public:
+};
+
+} // namespace protos
+} // namespace uefi
+
+#endif // _uefi_protos_ip6_config_h_

external/uefi/protos/load_file.h

@@ -0,0 +1,36 @@
+#pragma once
+#ifndef _uefi_protos_load_file_h_
+#define _uefi_protos_load_file_h_
+
+// This file was auto generated by the j6-uefi-headers project. Please see
+// https://github.com/justinian/j6-uefi-headers for more information.
+
+#include <uefi/guid.h>
+#include <uefi/types.h>
+#include <uefi/protos/device_path.h>
+
+namespace uefi {
+namespace protos {
+struct load_file;
+
+struct load_file
+{
+    static constexpr uefi::guid guid{ {0x56ec3091,0x954c,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
+
+
+    inline uefi::status load_file(uefi::protos::device_path * file_path, bool boot_policy, size_t * buffer_size, void * buffer) {
+        return _load_file(this, file_path, boot_policy, buffer_size, buffer);
+    }
+
+
+protected:
+    using _load_file_def = uefi::status (*)(uefi::protos::load_file *, uefi::protos::device_path *, bool, size_t *, void *);
+    _load_file_def _load_file;
+
+public:
+};
+
+} // namespace protos
+} // namespace uefi
+
+#endif // _uefi_protos_load_file_h_

external/uefi/protos/loaded_image.h

@@ -18,6 +18,7 @@ struct loaded_image
 {
     static constexpr uefi::guid guid{ 0x5b1b31a1,0x9562,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
+
     inline uefi::status unload(uefi::handle image_handle) {
         return _unload(image_handle);
     }
@@ -45,4 +46,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_loaded_image_h_
+#endif // _uefi_protos_loaded_image_h_

external/uefi/protos/service_binding.h

@@ -0,0 +1,41 @@
+#pragma once
+#ifndef _uefi_protos_service_binding_h_
+#define _uefi_protos_service_binding_h_
+
+// This file was auto generated by the j6-uefi-headers project. Please see
+// https://github.com/justinian/j6-uefi-headers for more information.
+
+#include <uefi/guid.h>
+#include <uefi/types.h>
+
+namespace uefi {
+namespace protos {
+struct service_binding;
+
+struct service_binding
+{
+
+
+    inline uefi::status create_child(uefi::handle * child_handle) {
+        return _create_child(this, child_handle);
+    }
+
+    inline uefi::status destroy_child(uefi::handle child_handle) {
+        return _destroy_child(this, child_handle);
+    }
+
+
+protected:
+    using _create_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle *);
+    _create_child_def _create_child;
+
+    using _destroy_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle);
+    _destroy_child_def _destroy_child;
+
+public:
+};
+
+} // namespace protos
+} // namespace uefi
+
+#endif // _uefi_protos_service_binding_h_

external/uefi/protos/simple_file_system.h

@@ -17,6 +17,7 @@ struct simple_file_system
 {
     static constexpr uefi::guid guid{ 0x0964e5b22,0x6459,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
+
     inline uefi::status open_volume(uefi::protos::file ** root) {
         return _open_volume(this, root);
     }
@@ -33,4 +34,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_simple_file_system_h_
+#endif // _uefi_protos_simple_file_system_h_

external/uefi/protos/simple_text_output.h

@@ -17,6 +17,7 @@ struct simple_text_output
 {
     static constexpr uefi::guid guid{ 0x387477c2,0x69c7,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
+
     inline uefi::status reset(bool extended_verification) {
         return _reset(this, extended_verification);
     }
@@ -89,4 +90,4 @@ public:
 } // namespace protos
 } // namespace uefi
 
-#endif // _uefi_protos_simple_text_output_h_
+#endif // _uefi_protos_simple_text_output_h_

modules.yaml

@@ -6,11 +6,13 @@ modules:
         output: jsix.elf
         target: host
         deps:
+            - cpu
             - kutil
         includes:
             - src/kernel
         source:
             - src/kernel/apic.cpp
+            - src/kernel/ap_startup.s
             - src/kernel/assert.cpp
             - src/kernel/boot.s
             - src/kernel/clock.cpp
@@ -20,16 +22,15 @@ modules:
             - src/kernel/debug.cpp
             - src/kernel/debug.s
             - src/kernel/device_manager.cpp
-            - src/kernel/font.cpp
             - src/kernel/frame_allocator.cpp
             - src/kernel/fs/gpt.cpp
             - src/kernel/gdt.cpp
-            - src/kernel/gdt.s
+            - src/kernel/gdtidt.s
             - src/kernel/hpet.cpp
+            - src/kernel/idt.cpp
             - src/kernel/interrupts.cpp
             - src/kernel/interrupts.s
             - src/kernel/io.cpp
-            - src/kernel/loader.s
             - src/kernel/log.cpp
             - src/kernel/main.cpp
             - src/kernel/memory_bootstrap.cpp
@@ -42,9 +43,9 @@ modules:
             - src/kernel/objects/system.cpp
             - src/kernel/objects/vm_area.cpp
             - src/kernel/page_table.cpp
+            - src/kernel/page_tree.cpp
             - src/kernel/pci.cpp
             - src/kernel/scheduler.cpp
-            - src/kernel/screen.cpp
             - src/kernel/serial.cpp
             - src/kernel/symbol_table.cpp
             - src/kernel/syscall.cpp
@@ -57,13 +58,15 @@ modules:
             - src/kernel/syscalls/thread.cpp
             - src/kernel/syscalls/vm_area.cpp
             - src/kernel/task.s
-            - src/kernel/vm_mapper.cpp
+            - src/kernel/tss.cpp
             - src/kernel/vm_space.cpp
 
     boot:
         kind: exe
         target: boot
         output: boot.efi
+        deps:
+            - cpu
         source:
             - src/boot/main.cpp
             - src/boot/console.cpp
@@ -73,6 +76,7 @@ modules:
             - src/boot/loader.cpp
             - src/boot/memory.cpp
             - src/boot/paging.cpp
+            - src/boot/status.cpp
             - src/boot/support.cpp
 
     nulldrv:
@@ -84,9 +88,20 @@ modules:
         source:
             - src/drivers/nulldrv/io.cpp
             - src/drivers/nulldrv/main.cpp
-            - src/drivers/nulldrv/main.s
             - src/drivers/nulldrv/serial.cpp
 
+    fb:
+        kind: exe
+        target: user
+        output: fb.elf
+        deps:
+          - libc
+        source:
+            - src/drivers/fb/font.cpp
+            - src/drivers/fb/main.cpp
+            - src/drivers/fb/screen.cpp
+            - src/drivers/fb/scrollback.cpp
+
     kutil:
         kind: lib
         output: libkutil.a
@@ -99,13 +114,32 @@ modules:
             - src/libraries/kutil/logger.cpp
             - src/libraries/kutil/memory.cpp
             - src/libraries/kutil/printf.c
+            - src/libraries/kutil/spinlock.cpp
 
+    cpu:
+        kind: lib
+        output: libcpu.a
+        includes:
+            - src/libraries/cpu/include
+        source:
+            - src/libraries/cpu/cpu_id.cpp
+
+    j6:
+      kind: lib
+      output: libj6.a
+      includes:
+        - src/libraries/j6/include
+      target: user
+      source:
+        - src/libraries/j6/syscalls.s
 
     libc:
         kind: lib
         output: libc.a
         includes:
           - src/libraries/libc/include
+        deps:
+            - j6
         target: user
         defines:
             - DISABLE_SSE
@@ -122,7 +156,8 @@ modules:
               #- LACKS_TIME_H
         source:
             - src/libraries/libc/arch/x86_64/_Exit.s
-            - src/libraries/libc/arch/x86_64/syscalls.s
+            - src/libraries/libc/arch/x86_64/crt0.s
+            - src/libraries/libc/arch/x86_64/init_libc.c
             - src/libraries/libc/ctype/isalnum.c
             - src/libraries/libc/ctype/isalpha.c
             - src/libraries/libc/ctype/isblank.c
@@ -291,6 +326,7 @@ modules:
             - src/tests/main.cpp
             - src/tests/map.cpp
             - src/tests/vector.cpp
+
 overlays:
     - url: https://f000.backblazeb2.com/file/jsix-os/sysroot-llvm8-20190706.tar.bz2
       path: sysroot

qemu.sh

@@ -6,6 +6,7 @@ debug=""
 debugtarget="${build}/jsix.elf"
 flash_name="ovmf_vars"
 gfx="-nographic"
+vga="-vga none"
 kvm=""
 cpu="Broadwell,+pdpe1gb"
 
@@ -22,6 +23,10 @@ for arg in $@; do
 			;;
 		--gfx)
 			gfx=""
+			vga=""
+			;;
+		--vga)
+			vga=""
 			;;
 		--kvm)
 			kvm="-enable-kvm"
@@ -72,4 +77,4 @@ exec qemu-system-x86_64 \
 	-cpu "${cpu}" \
 	-M q35 \
 	-no-reboot \
-	$gfx $kvm $debug
+	$gfx $vga $kvm $debug

scripts/build_symbol_table.py

@@ -19,15 +19,19 @@ def parse_syms(infile):
     representing the symbols in the text segment of the binary. Returns
     a list of (address, symbol_name)."""
 
-    from cxxfilt import demangle
+    from cxxfilt import demangle, InvalidName
 
     syms = []
     for line in sys.stdin:
         addr, t, mangled = line.split()
         if t not in "tTvVwW": continue
 
+        try:
+            name = demangle(mangled)
+        except InvalidName:
+            continue
+
         addr = int(addr, base=16)
-        name = demangle(mangled)
         syms.append((addr, name))
 
     return sorted(syms)

scripts/fontpsf.py

@@ -0,0 +1,75 @@
+_MAGIC = (0x72, 0xb5, 0x4a, 0x86)
+_VERSION = 0
+
+class PSF2:
+    from collections import namedtuple
+    Header = namedtuple("PSF2Header",
+        ["version", "offset", "flags", "count", "charsize", "height", "width"])
+
+    def __init__(self, filename, header, data):
+        self.__filename = filename
+        self.__header = header
+        self.__data = data
+
+    data = property(lambda self: self.__data)
+    header = property(lambda self: self.__header)
+    count = property(lambda self: self.__header.count)
+    charsize = property(lambda self: self.__header.charsize)
+    dimension = property(lambda self: (self.__header.width, self.__header.height))
+    filename = property(lambda self: self.__filename)
+
+    @classmethod
+    def load(cls, filename):
+        from os.path import basename
+        from struct import unpack_from
+
+        data = open(filename, 'rb').read()
+
+        fmt = "BBBBIIIIIII"
+        values = unpack_from(fmt, data)
+
+        if values[:len(_MAGIC)] != _MAGIC:
+            raise Exception("Bad magic number in header")
+
+        header = PSF2.Header(*values[len(_MAGIC):])
+        if header.version != _VERSION:
+            raise Exception(f"Bad version {header.version} in header")
+
+        return cls(basename(filename), header, data)
+
+    class Glyph:
+        __slots__ = ['index', 'data']
+        def __init__(self, i, data):
+            self.index = i
+            self.data = data
+        def __index__(self):
+            return self.index
+        def empty(self):
+            return not bool([b for b in self.data if b != 0])
+        def description(self):
+            c = chr(self.index)
+            if c.isprintable():
+                return "Glyph {:02x}: '{}'".format(self.index, c)
+            else:
+                return "Glyph {:02x}".format(self.index)
+
+    def __getitem__(self, i):
+        c = self.__header.charsize
+        n = i * c + self.__header.offset
+        return PSF2.Glyph(i, self.__data[n:n+c])
+
+    class __iter:
+        __slots__ = ['font', 'n']
+        def __init__(self, font):
+            self.font = font
+            self.n = 0
+        def __next__(self):
+            if self.n < self.font.count:
+                glyph = self.font[self.n]
+                self.n += 1
+                return glyph
+            else:
+                raise StopIteration
+
+    def __iter__(self):
+        return PSF2.__iter(self)

scripts/parse_font.py

@@ -1,21 +1,6 @@
 #!/usr/bin/env python3
 
-MAGIC = (0x72, 0xb5, 0x4a, 0x86)
-
-from collections import namedtuple
-PSF2 = namedtuple("PSF2", ["version", "offset", "flags", "count", "charsize", "height", "width"])
-
-def read_header(data):
-    from struct import unpack_from, calcsize
-
-    fmt = "BBBBIIIIIII"
-
-    values = unpack_from(fmt, data)
-    if values[:len(MAGIC)] != MAGIC:
-        raise Exception("Bad magic number in header")
-
-    return PSF2(*values[len(MAGIC):])
-
+from fontpsf import PSF2
 
 def print_glyph(header, data):
     bw = (header.width + 7) // 8
@@ -28,16 +13,15 @@ def print_glyph(header, data):
 
 
 def display_font(filename):
-    data = open(filename, 'rb').read()
+    font = PSF2.load(filename)
+    print(font.header)
 
-    header = read_header(data)
-    print(header)
-
-    c = header.charsize
-    for i in range(0, header.count):
-        n = i * c + header.offset
-        print("Glyph {}:".format(i))
-        print_glyph(header, data[n:n+c])
+    for glyph in font:
+        if glyph.empty():
+            print("{}: BLANK".format(glyph.description()))
+        else:
+            print("{}:".format(glyph.description()))
+            print_glyph(font.header, glyph.data)
 
 
 if __name__ == "__main__":

scripts/psf_to_cpp.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+
+from fontpsf import PSF2
+
+def print_header(filename):
+    font = PSF2.load(filename)
+
+    print("#pragma once")
+    print(f"// This file was autogenerated by psf_to_cpp.py from {font.filename}\n")
+
+    print(f"const uint8_t font_glyph_size = {font.charsize};")
+    print(f"const uint8_t font_glyph_width = {font.dimension[0]};")
+    print(f"const uint8_t font_glyph_height = {font.dimension[1]};")
+    print(f"const uint16_t font_glyph_count = {font.count};\n")
+
+    print('const uint8_t font_glyph_data[] = {')
+
+    for glyph in font:
+        print("    ", "".join([f"0x{b:02x}," for b in glyph.data]), end="")
+        print(" // {}".format(glyph.description()))
+
+    print("};")
+
+if __name__ == "__main__":
+    import sys
+    for filename in sys.argv[1:]:
+        print_header(filename)

scripts/templates/build.ninja.j2

@@ -190,15 +190,15 @@ build $builddir/fatroot/efi/boot/bootx64.efi : cp $builddir/boot/boot.efi
 build $builddir/fatroot/nulldrv.elf : cp $builddir/user/nulldrv.elf
     name = null driver to FAT image
 
-build $builddir/fatroot/terminal.elf : cp $builddir/user/nulldrv.elf
-    name = terminal driver to FAT image
+build $builddir/fatroot/fb.elf : cp $builddir/user/fb.elf
+    name = fb driver to FAT image
 
 build ${builddir}/fatroot/symbol_table.dat : makest ${builddir}/jsix.elf
 
 build $builddir/jsix.img : makefat | $
     $builddir/fatroot/symbol_table.dat $
     $builddir/fatroot/nulldrv.elf $
-    $builddir/fatroot/terminal.elf $
+    $builddir/fatroot/fb.elf $
     $builddir/fatroot/jsix.elf $
     $builddir/fatroot/efi/boot/bootx64.efi
     name = jsix.img

src/arch/x86_64/kernel.ld

@@ -1,4 +1,4 @@
-ENTRY(_start)
+ENTRY(_kernel_start)
 SECTIONS
 {
 	. = 0xFFFF800000000000;

src/boot/console.cpp

@@ -17,23 +17,7 @@ namespace boot {
 size_t ROWS = 0;
 size_t COLS = 0;
 
-static constexpr int level_ok = 0;
-static constexpr int level_warn = 1;
-static constexpr int level_fail = 2;
-
-static const wchar_t *level_tags[] = {
-	L"  ok  ",
-	L" warn ",
-	L"failed"
-};
-static const uefi::attribute level_colors[] = {
-	uefi::attribute::green,
-	uefi::attribute::brown,
-	uefi::attribute::light_red
-};
-
 console *console::s_console = nullptr;
-status_line *status_line::s_current = nullptr;
 
 static const wchar_t digits[] = {u'0', u'1', u'2', u'3', u'4', u'5',
 	u'6', u'7', u'8', u'9', u'a', u'b', u'c', u'd', u'e', u'f'};
@@ -49,9 +33,10 @@ wstrlen(const wchar_t *s)
 
 
 console::console(uefi::boot_services *bs, uefi::protos::simple_text_output *out) :
-	m_rows(0),
-	m_cols(0),
-	m_out(out)
+	m_rows {0},
+	m_cols {0},
+	m_out {out},
+	m_fb {0, 0}
 {
 	pick_mode(bs);
 
@@ -70,7 +55,26 @@ console::console(uefi::boot_services *bs, uefi::protos::simple_text_output *out)
 	m_out->output_string(GIT_VERSION_WIDE);
 
 	m_out->set_attribute(uefi::attribute::light_gray);
-	m_out->output_string(L" booting...\r\n\n");
+	m_out->output_string(L" booting...\r\n");
+
+	if (m_fb.type != kernel::args::fb_type::none) {
+		wchar_t const * type = nullptr;
+		switch (m_fb.type) {
+		case kernel::args::fb_type::rgb8:
+			type = L"rgb8";
+			break;
+		case kernel::args::fb_type::bgr8:
+			type = L"bgr8";
+			break;
+		default:
+			type = L"unknown";
+		}
+
+		printf(L"Found framebuffer: %dx%d[%d] type %s @0x%x\r\n",
+			m_fb.horizontal, m_fb.vertical, m_fb.scanline, type, m_fb.phys_addr);
+	} else {
+		printf(L"No framebuffer found.\r\n");
+	}
 
 	s_console = this;
 }
@@ -82,9 +86,14 @@ console::pick_mode(uefi::boot_services *bs)
 	uefi::protos::graphics_output *gfx_out_proto;
 	uefi::guid guid = uefi::protos::graphics_output::guid;
 
-	try_or_raise(
-		bs->locate_protocol(&guid, nullptr, (void **)&gfx_out_proto),
-		L"Failed to find a Graphics Output Protocol handle");
+	m_fb.type = kernel::args::fb_type::none;
+
+	uefi::status has_gop = bs->locate_protocol(&guid, nullptr,
+		(void **)&gfx_out_proto);
+
+	if (has_gop != uefi::status::success)
+		// No video output found, skip it
+		return;
 
 	const uint32_t modes = gfx_out_proto->mode->max_mode;
 	uint32_t best = gfx_out_proto->mode->mode;
@@ -93,7 +102,7 @@ console::pick_mode(uefi::boot_services *bs)
 		(uefi::graphics_output_mode_info *)gfx_out_proto->mode;
 
 	uint32_t res = info->horizontal_resolution * info->vertical_resolution;
-	int is_fb = info->pixel_format != uefi::pixel_format::blt_only;
+	int pixmode = static_cast<int>(info->pixel_format);
 
 	for (uint32_t i = 0; i < modes; ++i) {
 		size_t size = 0;
@@ -107,17 +116,37 @@ console::pick_mode(uefi::boot_services *bs)
 #endif
 
 		const uint32_t new_res = info->horizontal_resolution * info->vertical_resolution;
-		const int new_is_fb = info->pixel_format != uefi::pixel_format::blt_only;
+		int new_pixmode = static_cast<int>(info->pixel_format);
 
-		if (new_is_fb > is_fb && new_res >= res) {
+		if (new_pixmode <= pixmode && new_res >= res) {
 			best = i;
 			res = new_res;
+			pixmode = new_pixmode;
 		}
 	}
 
 	try_or_raise(
 		gfx_out_proto->set_mode(best),
 		L"Failed to set graphics mode");
+
+	if (pixmode <= static_cast<int>(uefi::pixel_format::bgr8)) {
+		m_fb.phys_addr = gfx_out_proto->mode->frame_buffer_base;
+		m_fb.size = gfx_out_proto->mode->frame_buffer_size;
+		m_fb.vertical = gfx_out_proto->mode->info->vertical_resolution;
+		m_fb.horizontal = gfx_out_proto->mode->info->horizontal_resolution;
+		m_fb.scanline = gfx_out_proto->mode->info->pixels_per_scanline;
+
+		switch (gfx_out_proto->mode->info->pixel_format) {
+		case uefi::pixel_format::rgb8:
+			m_fb.type = kernel::args::fb_type::rgb8;
+			break;
+		case uefi::pixel_format::bgr8:
+			m_fb.type = kernel::args::fb_type::bgr8;
+			break;
+		default:
+			m_fb.type = kernel::args::fb_type::none;
+		}
+	}
 }
 
 size_t
@@ -272,176 +301,4 @@ console::print(const wchar_t *fmt, ...)
 	return result;
 }
 
-status_line::status_line(const wchar_t *message, const wchar_t *context) :
-	m_level(level_ok)
-{
-	auto out = console::get().m_out;
-	m_line = out->mode->cursor_row;
-	m_depth = (s_current ? 1 + s_current->m_depth : 0);
-
-	int indent = 2 * m_depth;
-	out->set_cursor_position(indent, m_line);
-	out->set_attribute(uefi::attribute::light_gray);
-	out->output_string(message);
-
-	if (context) {
-		out->output_string(L": ");
-		out->output_string(context);
-	}
-
-	out->output_string(L"\r\n");
-
-	m_next = s_current;
-	s_current = this;
-}
-
-status_line::~status_line()
-{
-	if (s_current != this)
-		error::raise(uefi::status::unsupported, L"Destroying non-current status_line");
-
-	finish();
-	if (m_next && m_level > m_next->m_level) {
-		m_next->m_level = m_level;
-		m_next->print_status_tag();
-	}
-	s_current = m_next;
-}
-
-void
-status_line::print_status_tag()
-{
-	auto out = console::get().m_out;
-	int row = out->mode->cursor_row;
-	int col = out->mode->cursor_column;
-
-	uefi::attribute color = level_colors[m_level];
-	const wchar_t *tag = level_tags[m_level];
-
-	out->set_cursor_position(50, m_line);
-
-	out->set_attribute(uefi::attribute::light_gray);
-	out->output_string(L"[");
-	out->set_attribute(color);
-	out->output_string(tag);
-	out->set_attribute(uefi::attribute::light_gray);
-	out->output_string(L"]\r\n");
-
-	out->set_cursor_position(col, row);
-}
-
-void
-status_line::do_warn(const wchar_t *message, const wchar_t *error)
-{
-	auto out = console::get().m_out;
-	int row = out->mode->cursor_row;
-
-	if (m_level < level_warn) {
-		m_level = level_warn;
-		print_status_tag();
-	}
-
-	int indent = 2 + 2 * m_depth;
-	out->set_cursor_position(indent, row);
-	out->set_attribute(uefi::attribute::yellow);
-	out->output_string(message);
-
-	if (error) {
-		out->output_string(L": ");
-		out->output_string(error);
-	}
-
-	out->set_attribute(uefi::attribute::light_gray);
-	out->output_string(L"\r\n");
-}
-
-void
-status_line::do_fail(const wchar_t *message, const wchar_t *error)
-{
-	auto out = console::get().m_out;
-	int row = out->mode->cursor_row;
-
-	if (s_current->m_level < level_fail) {
-		m_level = level_fail;
-		print_status_tag();
-	}
-
-	int indent = 2 + 2 * m_depth;
-	out->set_cursor_position(indent, row);
-	out->set_attribute(uefi::attribute::red);
-	out->output_string(message);
-
-	if (error) {
-		out->output_string(L": ");
-		out->output_string(error);
-	}
-
-	out->set_attribute(uefi::attribute::light_gray);
-	out->output_string(L"\r\n");
-}
-
-void
-status_line::finish()
-{
-	if (m_level <= level_ok)
-		print_status_tag();
-}
-
-/*
-uefi::status
-con_get_framebuffer(
-	EFI_BOOT_SERVICES *bootsvc,
-	void **buffer,
-	size_t *buffer_size,
-	uint32_t *hres,
-	uint32_t *vres,
-	uint32_t *rmask,
-	uint32_t *gmask,
-	uint32_t *bmask)
-{
-	uefi::status status;
-
-	uefi::protos::graphics_output *gop;
-	status = bootsvc->LocateProtocol(&guid_gfx_out, NULL, (void **)&gop);
-	if (status != EFI_NOT_FOUND) {
-		CHECK_EFI_STATUS_OR_RETURN(status, "LocateProtocol gfx");
-
-		*buffer = (void *)gop->Mode->FrameBufferBase;
-		*buffer_size = gop->Mode->FrameBufferSize;
-		*hres = gop->Mode->Info->horizontal_resolution;
-		*vres = gop->Mode->Info->vertical_resolution;
-
-		switch (gop->Mode->Info->PixelFormat) {
-			case PixelRedGreenBlueReserved8BitPerColor:
-				*rmask = 0x0000ff;
-				*gmask = 0x00ff00;
-				*bmask = 0xff0000;
-				return EFI_SUCCESS;
-
-			case PixelBlueGreenRedReserved8BitPerColor:
-				*bmask = 0x0000ff;
-				*gmask = 0x00ff00;
-				*rmask = 0xff0000;
-				return EFI_SUCCESS;
-
-			case PixelBitMask:
-				*rmask = gop->Mode->Info->PixelInformation.RedMask;
-				*gmask = gop->Mode->Info->PixelInformation.GreenMask;
-				*bmask = gop->Mode->Info->PixelInformation.BlueMask;
-				return EFI_SUCCESS;
-
-			default:
-				// Not a framebuffer, fall through to zeroing out
-				// values below.
-				break;
-		}
-	}
-
-	*buffer = NULL;
-	*buffer_size = *hres = *vres = 0;
-	*rmask = *gmask = *bmask = 0;
-	return EFI_SUCCESS;
-}
-*/
-
 } // namespace boot

src/boot/console.h

@@ -1,10 +1,12 @@
 /// \file console.h
-/// Text output and status message handling
+/// Text output handler
 #pragma once
 #include <stdarg.h>
 #include <stddef.h>
 #include <uefi/boot_services.h>
 #include <uefi/protos/simple_text_output.h>
+#include "kernel_args.h"
+#include "types.h"
 
 namespace boot {
 
@@ -12,6 +14,8 @@ namespace boot {
 class console
 {
 public:
+	using framebuffer = kernel::args::framebuffer;
+
 	console(uefi::boot_services *bs, uefi::protos::simple_text_output *out);
 
 	size_t print_hex(uint32_t n) const;
@@ -20,6 +24,8 @@ public:
 	size_t print_long_dec(uint64_t n) const;
 	size_t printf(const wchar_t *fmt, ...) const;
 
+	const framebuffer & fb() const { return m_fb; };
+
 	static console & get() { return *s_console; }
 	static size_t print(const wchar_t *fmt, ...);
 
@@ -31,50 +37,9 @@ private:
 
 	size_t m_rows, m_cols;
 	uefi::protos::simple_text_output *m_out;
+	framebuffer m_fb;
 
 	static console *s_console;
 };
 
-/// Scoped status line reporter. Prints a message and an "OK" if no errors
-/// or warnings were reported before destruction, otherwise reports the
-/// error or warning.
-class status_line
-{
-public:
-	/// Constructor.
-	/// \arg message  Description of the operation in progress
-	/// \arg context  If non-null, printed after `message` and a colon
-	status_line(const wchar_t *message, const wchar_t *context = nullptr);
-	~status_line();
-
-	/// Set the state to warning, and print a message. If the state is already at
-	/// warning or error, the state is unchanged but the message is still printed.
-	/// \arg message  The warning message to print
-	/// \arg error    If non-null, printed after `message`
-	inline static void warn(const wchar_t *message, const wchar_t *error = nullptr) {
-		if (s_current) s_current->do_warn(message, error);
-	}
-
-	/// Set the state to error, and print a message. If the state is already at
-	/// error, the state is unchanged but the message is still printed.
-	/// \arg message  The error message to print
-	/// \arg error    If non-null, printed after `message`
-	inline static void fail(const wchar_t *message, const wchar_t *error = nullptr) {
-		if (s_current) s_current->do_fail(message, error);
-	}
-
-private:
-	void print_status_tag();
-	void do_warn(const wchar_t *message, const wchar_t *error);
-	void do_fail(const wchar_t *message, const wchar_t *error);
-	void finish();
-
-	size_t m_line;
-	int m_level;
-	int m_depth;
-
-	status_line *m_next;
-	static status_line *s_current;
-};
-
 } // namespace boot

src/boot/error.cpp

@@ -1,11 +1,11 @@
 #include "error.h"
 #include "console.h"
+#include "kernel_args.h"
+#include "status.h"
 
 namespace boot {
 namespace error {
 
-handler *handler::s_current = nullptr;
-
 struct error_code_desc {
 	uefi::status code;
 	const wchar_t *name;
@@ -20,8 +20,8 @@ struct error_code_desc error_table[] = {
 	{ uefi::status::success, nullptr }
 };
 
-static const wchar_t *
-error_message(uefi::status status)
+const wchar_t *
+message(uefi::status status)
 {
 	int32_t i = -1;
 	while (error_table[++i].name != nullptr) {
@@ -34,56 +34,32 @@ error_message(uefi::status status)
 		return L"Unknown Warning";
 }
 
-[[ noreturn ]] void
-raise(uefi::status status, const wchar_t *message)
-{
-	if (handler::s_current) {
-		handler::s_current->handle(status, message);
-	}
-	while (1) asm("hlt");
-}
-
-handler::handler() :
-	m_next(s_current)
-{
-	s_current = this;
-}
-
-handler::~handler()
-{
-	if (s_current != this)
-		raise(uefi::status::warn_stale_data,
-				L"Non-current error handler destructing");
-	s_current = m_next;
-}
-
-uefi_handler::uefi_handler(console &con) :
-	handler(),
-	m_con(con)
-{
-}
-
-void
-uefi_handler::handle(uefi::status s, const wchar_t *message)
-{
-	status_line::fail(message, error_message(s));
-}
-
-cpu_assert_handler::cpu_assert_handler() : handler() {}
-
-void
-cpu_assert_handler::handle(uefi::status s, const wchar_t *message)
+[[ noreturn ]] static void
+cpu_assert(uefi::status s, const wchar_t *message, size_t line)
 {
 	asm volatile (
 		"movq $0xeeeeeeebadbadbad, %%r8;"
 		"movq %0, %%r9;"
+		"movq %1, %%r10;"
+		"movq %2, %%r11;"
 		"movq $0, %%rdx;"
 		"divq %%rdx;"
 		:
-		: "r"((uint64_t)s)
-		: "rax", "rdx", "r8", "r9");
+		: "r"((uint64_t)s), "r"(message), "r"(line)
+		: "rax", "rdx", "r8", "r9", "r10");
+	while (1) asm("hlt");
 }
 
+[[ noreturn ]] void
+raise(uefi::status status, const wchar_t *message, size_t line)
+{
+	if(status_line::fail(message, status))
+		while (1) asm("hlt");
+	else
+		cpu_assert(status, message, line);
+}
+
+
 } // namespace error
 } // namespace boot
 

src/boot/error.h

@@ -12,50 +12,9 @@ class console;
 namespace error {
 
 /// Halt or exit the program with the given error status/message
-[[ noreturn ]] void raise(uefi::status status, const wchar_t *message);
+[[ noreturn ]] void raise(uefi::status status, const wchar_t *message, size_t line = 0);
 
-/// Interface for error-handling functors
-class handler
-{
-public:
-	/// Constructor must be called by implementing classes.
-	handler();
-	virtual ~handler();
-
-	/// Interface for implementations of error handling.
-	virtual void handle(uefi::status, const wchar_t*) = 0;
-
-private:
-	friend void raise(uefi::status, const wchar_t *);
-
-	handler *m_next;
-	static handler *s_current;
-};
-
-/// Error handler using UEFI boot services. Integrates with `status_line`
-/// to print formatted error messages to the screen.
-class uefi_handler :
-	public handler
-{
-public:
-	uefi_handler(console &con);
-	virtual ~uefi_handler() {}
-	void handle(uefi::status, const wchar_t*) override;
-
-private:
-	console &m_con;
-};
-
-/// Error handler that doesn't rely on UEFI. Sets status into CPU
-/// registers and then causes a CPU #DE exception.
-class cpu_assert_handler :
-	public handler
-{
-public:
-	cpu_assert_handler();
-	virtual ~cpu_assert_handler() {}
-	void handle(uefi::status, const wchar_t*) override;
-};
+const wchar_t * message(uefi::status status);
 
 } // namespace error
 } // namespace boot
@@ -69,6 +28,6 @@ void debug_break();
 #define try_or_raise(s, m) \
 	do { \
 		uefi::status _s = (s); \
-		if (uefi::is_error(_s)) ::boot::error::raise(_s, (m)); \
+		if (uefi::is_error(_s)) ::boot::error::raise(_s, (m), __LINE__); \
 	} while(0)
 

src/boot/fs.cpp

@@ -8,6 +8,7 @@
 #include "console.h"
 #include "error.h"
 #include "memory.h"
+#include "status.h"
 
 namespace boot {
 namespace fs {
@@ -77,7 +78,7 @@ file::load(uefi::memory_type mem_type)
 		m_file->read(&size, data),
 		L"Could not read from file");
 
-	return { .data = data, .size = size };
+	return { .size = size,  .data = data };
 }
 
 file

src/boot/guids.cpp

@@ -1,13 +0,0 @@
-#include "guids.h"
-
-#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) \
-	EFI_GUID name __attribute__((section(".rodata"))) = {dw, w1, w2, {b1, b2, b3, b4, b5, b6, b7, b8}}
-#include "guids.inc"
-#undef GUID
-
-int is_guid(EFI_GUID *a, EFI_GUID *b)
-{
-	uint64_t *ai = (uint64_t *)a;
-	uint64_t *bi = (uint64_t *)b;
-	return ai[0] == bi[0] && ai[1] == bi[1];
-}

src/boot/guids.h

@@ -1,8 +0,0 @@
-#pragma once
-#include <efi/efi.h>
-
-int is_guid(EFI_GUID *a, EFI_GUID *b);
-
-#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) extern EFI_GUID name
-#include "guids.inc"
-#undef GUID

src/boot/guids.inc

@@ -1,11 +0,0 @@
-GUID(0xeb9d2d30,0x2d88,0x11d3,0x9a,0x16,0x00,0x90,0x27,0x3f,0xc1,0x4d, guid_acpi1);
-GUID(0x8868e871,0xe4f1,0x11d3,0xbc,0x22,0x00,0x80,0xc7,0x3c,0x88,0x81, guid_acpi2);
-GUID(0x09576e92,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_file_info);
-GUID(0x9042a9de,0x23dc,0x4a38,0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a, guid_gfx_out);
-GUID(0x964e5b22,0x6459,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_simple_filesystem);
-GUID(0x09576e91,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_device_path);
-GUID(0x8b843e20,0x8132,0x4852,0x90,0xcc,0x55,0x1a,0x4e,0x4a,0x7f,0x1c, guid_device_path_to_text);
-
-GUID(0x10d0669c,0x9ec6,0x4268,0xbc,0x48,0xff,0x74,0x75,0x21,0xfe,0x07, guid_jsix_vendor);
-
-// vim: ft=c

src/boot/hardware.cpp

@@ -1,6 +1,7 @@
 #include "hardware.h"
 #include "console.h"
 #include "error.h"
+#include "status.h"
 
 namespace boot {
 namespace hw {
@@ -36,8 +37,25 @@ find_acpi_table(uefi::system_table *st)
 	return reinterpret_cast<void*>(acpi1_table);
 }
 
+static uint64_t
+rdmsr(uint32_t addr)
+{
+	uint32_t low, high;
+	__asm__ __volatile__ ("rdmsr" : "=a"(low), "=d"(high) : "c"(addr));
+	return (static_cast<uint64_t>(high) << 32) | low;
+}
+
+static void
+wrmsr(uint32_t addr, uint64_t value)
+{
+	uint32_t low = value & 0xffffffff;
+	uint32_t high = value >> 32;
+	__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
+}
+
+
 void
-setup_cr4()
+setup_control_regs()
 {
 	uint64_t cr4 = 0;
 	asm volatile ( "mov %%cr4, %0" : "=r" (cr4) );
@@ -48,6 +66,15 @@ setup_cr4()
 		0x020000 | // Enable PCIDs
 		0;
 	asm volatile ( "mov %0, %%cr4" :: "r" (cr4) );
+
+	// Set up IA32_EFER
+	constexpr uint32_t IA32_EFER = 0xC0000080;
+	uint64_t efer = rdmsr(IA32_EFER);
+	efer |=
+		0x0001 | // Enable SYSCALL
+		0x0800 | // Enable NX bit
+		0;
+	wrmsr(IA32_EFER, efer);
 }
 
 } // namespace hw

src/boot/hardware.h

@@ -13,8 +13,8 @@ namespace hw {
 /// significant bit set to 1.
 void * find_acpi_table(uefi::system_table *st);
 
-/// Enable CPU options in CR4 for the kernel starting state.
-void setup_cr4();
+/// Enable CPU options in CR4 etc for the kernel starting state.
+void setup_control_regs();
 
 } // namespace hw
 } // namespace boot

src/boot/loader.cpp

@@ -8,6 +8,7 @@
 #include "fs.h"
 #include "memory.h"
 #include "paging.h"
+#include "status.h"
 
 namespace args = kernel::args;
 
@@ -26,7 +27,7 @@ load_file(
 	fs::file file = disk.open(path);
 	buffer b = file.load(type);
 
-	console::print(L"    Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
+	//console::print(L"    Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
 	return b;
 }
 
@@ -81,11 +82,14 @@ load_program(
 
 	try_or_raise(
 		bs->allocate_pages(uefi::allocate_type::any_pages,
-			memory::program_type, num_pages, &pages),
+			uefi::memory_type::loader_data, num_pages, &pages),
 		L"Failed allocating space for program");
 
 	bs->set_mem(pages, total_size, 0);
 
+	program.base = prog_base;
+	program.total_size = total_size;
+	program.num_sections = 0;
 	for (int i = 0; i < header->ph_num; ++i) {
 		ptrdiff_t offset = header->ph_offset + i * header->ph_entsize;
 		const elf::program_header *pheader =
@@ -94,19 +98,18 @@ load_program(
 		if (pheader->type != elf::PT_LOAD)
 			continue;
 
+		args::program_section &section = program.sections[program.num_sections++];
+
 		void *src_start = offset_ptr<void>(data.data, pheader->offset);
 		void *dest_start = offset_ptr<void>(pages, pheader->vaddr - prog_base);
-		bs->copy_mem(dest_start, src_start, pheader->file_size);
 
-		console::print(L"    section %d phys: 0x%lx\r\n", i, pages);
-		console::print(L"    section %d virt: 0x%lx\r\n", i, pheader->vaddr);
+		bs->copy_mem(dest_start, src_start, pheader->file_size);
+		section.phys_addr = reinterpret_cast<uintptr_t>(dest_start);
+		section.virt_addr = pheader->vaddr;
+		section.size = pheader->mem_size;
+		section.type = static_cast<args::section_flags>(pheader->flags);
 	}
 
-	console::print(L"    entrypoint:     0x%lx\r\n", header->entrypoint);
-
-	program.phys_addr = reinterpret_cast<uintptr_t>(pages);
-	program.size = total_size;
-	program.virt_addr = prog_base;
 	program.entrypoint = header->entrypoint;
 }
 

src/boot/main.cpp

@@ -8,12 +8,14 @@
 #include <stdint.h>
 
 #include "console.h"
+#include "cpu/cpu_id.h"
 #include "error.h"
 #include "fs.h"
 #include "hardware.h"
 #include "loader.h"
 #include "memory.h"
 #include "paging.h"
+#include "status.h"
 
 #include "kernel_args.h"
 
@@ -37,7 +39,7 @@ struct program_desc
 const program_desc program_list[] = {
 	{L"kernel", L"jsix.elf"},
 	{L"null driver", L"nulldrv.elf"},
-	//{L"terminal driver", L"terminal.elf"},
+	{L"fb driver", L"fb.elf"},
 };
 
 /// Change a pointer to point to the higher-half linear-offset version
@@ -56,7 +58,7 @@ allocate_args_structure(
 	size_t max_modules,
 	size_t max_programs)
 {
-	status_line status(L"Setting up kernel args memory");
+	status_line status {L"Setting up kernel args memory"};
 
 	args::header *args = nullptr;
 
@@ -66,7 +68,7 @@ allocate_args_structure(
 		max_programs * sizeof(args::program); // The program structures
 
 	try_or_raise(
-		bs->allocate_pool(memory::args_type, args_size,
+		bs->allocate_pool(uefi::memory_type::loader_data, args_size,
 			reinterpret_cast<void**>(&args)),
 		L"Could not allocate argument memory");
 
@@ -91,19 +93,39 @@ add_module(args::header *args, args::mod_type type, buffer &data)
 	m.type = type;
 	m.location = data.data;
 	m.size = data.size;
+
+	change_pointer(m.location);
+}
+
+/// Check that all required cpu features are supported
+void
+check_cpu_supported()
+{
+	status_line status {L"Checking CPU features"};
+
+	cpu::cpu_id cpu;
+	uint64_t missing = cpu.missing();
+	if (missing) {
+#define CPU_FEATURE_OPT(...)
+#define CPU_FEATURE_REQ(name, ...) \
+		if (!cpu.has_feature(cpu::feature::name)) { \
+			status::fail(L"CPU required feature " L ## #name, uefi::status::unsupported); \
+		}
+#include "cpu/features.inc"
+#undef CPU_FEATURE_REQ
+#undef CPU_FEATURE_OPT
+
+		error::raise(uefi::status::unsupported, L"CPU not supported");
+	}
 }
 
 /// The main procedure for the portion of the loader that runs while
 /// UEFI is still in control of the machine. (ie, while the loader still
 /// has access to boot services.
 args::header *
-bootloader_main_uefi(
-	uefi::handle image,
-	uefi::system_table *st,
-	console &con)
+uefi_preboot(uefi::handle image, uefi::system_table *st)
 {
-	error::uefi_handler handler(con);
-	status_line status(L"Performing UEFI pre-boot");
+	status_line status {L"Performing UEFI pre-boot"};
 
 	uefi::boot_services *bs = st->boot_services;
 	uefi::runtime_services *rs = st->runtime_services;
@@ -122,9 +144,8 @@ bootloader_main_uefi(
 
 	fs::file disk = fs::get_boot_volume(image, bs);
 
-	const uefi::memory_type mod_type = memory::module_type;
 	buffer symbols = loader::load_file(disk, L"symbol table", L"symbol_table.dat",
-				memory::module_type);
+				uefi::memory_type::loader_data);
 	add_module(args, args::mod_type::symbol_table, symbols);
 
 	for (auto &desc : program_list) {
@@ -133,43 +154,62 @@ bootloader_main_uefi(
 		loader::load_program(program, desc.name, buf, bs);
 	}
 
-	for (unsigned i = 0; i < args->num_modules; ++i) {
-		args::module &mod = args->modules[i];
-		change_pointer(mod.location);
-	}
-
 	return args;
 }
 
+memory::efi_mem_map
+uefi_exit(args::header *args, uefi::handle image, uefi::boot_services *bs)
+{
+	status_line status {L"Exiting UEFI", nullptr, false};
+
+	memory::efi_mem_map map =
+		memory::build_kernel_mem_map(args, bs);
+
+	try_or_raise(
+		bs->exit_boot_services(image, map.key),
+		L"Failed to exit boot services");
+
+	return map;
+}
+
 } // namespace boot
 
 /// The UEFI entrypoint for the loader.
 extern "C" uefi::status
-efi_main(uefi::handle image_handle, uefi::system_table *st)
+efi_main(uefi::handle image, uefi::system_table *st)
 {
 	using namespace boot;
-
-	error::cpu_assert_handler handler;
 	console con(st->boot_services, st->con_out);
+	check_cpu_supported();
 
-	args::header *args =
-		bootloader_main_uefi(image_handle, st, con);
+	args::header *args = uefi_preboot(image, st);
+	memory::efi_mem_map map = uefi_exit(args, image, st->boot_services);
 
+	args->video = con.fb();
+	status_bar status {con.fb()}; // Switch to fb status display
+
+	// Map the kernel to the appropriate address
 	args::program &kernel = args->programs[0];
-	paging::map_pages(args, kernel.phys_addr, kernel.virt_addr, kernel.size);
+	for (auto &section : kernel.sections)
+		if (section.size)
+			paging::map_section(args, section);
+
+	memory::fix_frame_blocks(args);
+
 	kernel::entrypoint kentry =
 		reinterpret_cast<kernel::entrypoint>(kernel.entrypoint);
+	status.next();
 
-	memory::efi_mem_map map =
-		memory::build_kernel_mem_map(args, st->boot_services);
-
-	try_or_raise(
-		st->boot_services->exit_boot_services(image_handle, map.key),
-		L"Failed to exit boot services");
-
+	hw::setup_control_regs();
 	memory::virtualize(args->pml4, map, st->runtime_services);
+	status.next();
+
+	change_pointer(args);
 	change_pointer(args->pml4);
-	hw::setup_cr4();
+	change_pointer(args->modules);
+	change_pointer(args->programs);
+
+	status.next();
 
 	kentry(args);
 	debug_break();

src/boot/memory.cpp

@@ -7,12 +7,15 @@
 #include "error.h"
 #include "memory.h"
 #include "paging.h"
+#include "status.h"
 
 namespace boot {
 namespace memory {
 
 using mem_entry = kernel::args::mem_entry;
 using mem_type = kernel::args::mem_type;
+using frame_block = kernel::args::frame_block;
+using kernel::args::frames_per_block;
 
 size_t fixup_pointer_index = 0;
 void **fixup_pointers[64];
@@ -35,20 +38,28 @@ static const wchar_t *memory_type_names[] = {
 	L"persistent memory"
 };
 
+static const wchar_t *kernel_memory_type_names[] = {
+	L"free",
+	L"pending",
+	L"acpi",
+	L"uefi_runtime",
+	L"mmio",
+	L"persistent"
+};
+
 static const wchar_t *
 memory_type_name(uefi::memory_type t)
 {
-	if (t < uefi::memory_type::max_memory_type) {
+	if (t < uefi::memory_type::max_memory_type)
 		return memory_type_names[static_cast<uint32_t>(t)];
-	}
 
-	switch(t) {
-		case args_type:		return L"jsix kernel args";
-		case module_type:	return L"jsix bootloader module";
-		case program_type:	return L"jsix kernel or program code";
-		case table_type:	return L"jsix page tables";
-		default: return L"Bad Type Value";
-	}
+	return L"Bad Type Value";
+}
+
+static const wchar_t *
+kernel_memory_type_name(kernel::args::mem_type t)
+{
+	return kernel_memory_type_names[static_cast<uint32_t>(t)];
 }
 
 void
@@ -98,58 +109,174 @@ can_merge(mem_entry &prev, mem_type type, uefi::memory_descriptor *next)
 }
 
 void
-get_uefi_mappings(efi_mem_map *map, bool allocate, uefi::boot_services *bs)
+get_uefi_mappings(efi_mem_map &map, uefi::boot_services *bs)
 {
-	status_line(L"Getting UEFI memory map");
-
+	size_t length = map.total;
 	uefi::status status = bs->get_memory_map(
-		&map->length, nullptr, &map->key, &map->size, &map->version);
+		&length, map.entries, &map.key, &map.size, &map.version);
+	map.length = length;
+
+	if (status == uefi::status::success)
+		return;
 
 	if (status != uefi::status::buffer_too_small)
 		error::raise(status, L"Error getting memory map size");
 
-	if (allocate) {
-		map->length += 10*map->size;
-
+	if (map.entries) {
 		try_or_raise(
-			bs->allocate_pool(
-				uefi::memory_type::loader_data, map->length,
-				reinterpret_cast<void**>(&map->entries)),
-			L"Allocating space for memory map");
+			bs->free_pool(reinterpret_cast<void*>(map.entries)),
+			L"Freeing previous memory map space");
+	}
 
-		try_or_raise(
-			bs->get_memory_map(&map->length, map->entries, &map->key, &map->size, &map->version),
-			L"Getting UEFI memory map");
+	map.total = length + 10*map.size;
+
+	try_or_raise(
+		bs->allocate_pool(
+			uefi::memory_type::loader_data, map.total,
+			reinterpret_cast<void**>(&map.entries)),
+		L"Allocating space for memory map");
+
+	map.length = map.total;
+	try_or_raise(
+		bs->get_memory_map(&map.length, map.entries, &map.key, &map.size, &map.version),
+		L"Getting UEFI memory map");
+}
+
+inline size_t bitmap_size(size_t frames) { return (frames + 63) / 64; }
+inline size_t num_blocks(size_t frames) { return (frames + (frames_per_block-1)) / frames_per_block; }
+
+void
+build_kernel_frame_blocks(const mem_entry *map, size_t nent, kernel::args::header *args, uefi::boot_services *bs)
+{
+	status_line status {L"Creating kernel frame accounting map"};
+
+	size_t block_count = 0;
+	size_t total_bitmap_size = 0;
+	for (size_t i = 0; i < nent; ++i) {
+		const mem_entry &ent = map[i];
+		if (ent.type != mem_type::free)
+			continue;
+
+		block_count += num_blocks(ent.pages);
+		total_bitmap_size += bitmap_size(ent.pages) * sizeof(uint64_t);
+	}
+
+	size_t total_size = block_count * sizeof(frame_block) + total_bitmap_size;
+
+	frame_block *blocks = nullptr;
+	try_or_raise(
+		bs->allocate_pages(
+			uefi::allocate_type::any_pages,
+			uefi::memory_type::loader_data,
+			bytes_to_pages(total_size),
+			reinterpret_cast<void**>(&blocks)),
+		L"Error allocating kernel frame block space");
+
+	frame_block *next_block = blocks;
+	for (size_t i = 0; i < nent; ++i) {
+		const mem_entry &ent = map[i];
+		if (ent.type != mem_type::free)
+			continue;
+
+		size_t page_count = ent.pages;
+		uintptr_t base_addr = ent.start;
+		while (page_count) {
+			frame_block *blk = next_block++;
+			bs->set_mem(blk, sizeof(frame_block), 0);
+
+			blk->attrs = ent.attr;
+			blk->base = base_addr;
+			base_addr += frames_per_block * page_size;
+
+			if (page_count >= frames_per_block) {
+				page_count -= frames_per_block;
+				blk->count = frames_per_block;
+				blk->map1 = ~0ull;
+				bs->set_mem(blk->map2, sizeof(blk->map2), 0xff);
+			} else {
+				blk->count = page_count;
+				unsigned i = 0;
+
+				uint64_t b1 = (page_count + 4095) / 4096;
+				blk->map1 = (1 << b1) - 1;
+
+				uint64_t b2 = (page_count + 63) / 64;
+				uint64_t b2q = b2 / 64;
+				uint64_t b2r = b2 % 64;
+				bs->set_mem(blk->map2, b2q, 0xff);
+				blk->map2[b2q] = (1 << b2r) - 1;
+				break;
+			}
+		}
+	}
+
+	uint64_t *bitmap = reinterpret_cast<uint64_t*>(next_block);
+	bs->set_mem(bitmap, total_bitmap_size, 0);
+	for (unsigned i = 0; i < block_count; ++i) {
+		frame_block &blk = blocks[i];
+		blk.bitmap = bitmap;
+
+		size_t b = blk.count / 64;
+		size_t r = blk.count % 64;
+		bs->set_mem(blk.bitmap, b*8, 0xff);
+		blk.bitmap[b] = (1 << r) - 1;
+
+		bitmap += bitmap_size(blk.count);
+	}
+
+	args->frame_block_count = block_count;
+	args->frame_block_pages = bytes_to_pages(total_size);
+	args->frame_blocks = blocks;
+}
+
+void
+fix_frame_blocks(kernel::args::header *args)
+{
+	// Map the frame blocks to the appropriate address
+	paging::map_pages(args,
+		reinterpret_cast<uintptr_t>(args->frame_blocks),
+		::memory::bitmap_start,
+		args->frame_block_pages,
+		true, false);
+
+	uintptr_t offset = ::memory::bitmap_start -
+		reinterpret_cast<uintptr_t>(args->frame_blocks);
+
+	for (unsigned i = 0; i < args->frame_block_count; ++i) {
+		frame_block &blk = args->frame_blocks[i];
+		blk.bitmap = reinterpret_cast<uint64_t*>(
+			reinterpret_cast<uintptr_t>(blk.bitmap) + offset);
 	}
 }
 
 efi_mem_map
 build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 {
-	status_line(L"Creating kernel memory map");
+	status_line status {L"Creating kernel memory map"};
 
-	efi_mem_map efi_map;
-	get_uefi_mappings(&efi_map, false, bs);
+	efi_mem_map map;
+	get_uefi_mappings(map, bs);
 
-	size_t map_size = efi_map.num_entries() * sizeof(mem_entry);
+	size_t map_size = map.num_entries() * sizeof(mem_entry);
 
 	kernel::args::mem_entry *kernel_map = nullptr;
 	try_or_raise(
 		bs->allocate_pages(
 			uefi::allocate_type::any_pages,
-			module_type,
+			uefi::memory_type::loader_data,
 			bytes_to_pages(map_size),
 			reinterpret_cast<void**>(&kernel_map)),
 		L"Error allocating kernel memory map module space");
 
 	bs->set_mem(kernel_map, map_size, 0);
-	get_uefi_mappings(&efi_map, true, bs);
+	get_uefi_mappings(map, bs);
 
-	size_t i = 0;
+	size_t nent = 0;
 	bool first = true;
-	for (auto desc : efi_map) {
+	for (auto desc : map) {
 		/*
-		console::print(L"   Range %lx (%lx) %x(%s) [%lu]\r\n",
+		// EFI map dump
+		console::print(L"   eRange %lx (%lx) %x(%s) [%lu]\r\n",
 			desc->physical_start, desc->attribute, desc->type, memory_type_name(desc->type), desc->number_of_pages);
 		*/
 
@@ -162,10 +289,10 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 				continue;
 
 			case uefi::memory_type::loader_code:
-			case uefi::memory_type::loader_data:
 			case uefi::memory_type::boot_services_code:
 			case uefi::memory_type::boot_services_data:
 			case uefi::memory_type::conventional_memory:
+			case uefi::memory_type::loader_data:
 				type = mem_type::free;
 				break;
 
@@ -187,22 +314,6 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 				type = mem_type::persistent;
 				break;
 
-			case args_type:
-				type = mem_type::args;
-				break;
-
-			case module_type:
-				type = mem_type::module;
-				break;
-
-			case program_type:
-				type = mem_type::program;
-				break;
-
-			case table_type:
-				type = mem_type::table;
-				break;
-
 			default:
 				error::raise(
 					uefi::status::invalid_parameter,
@@ -212,18 +323,19 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 		// TODO: validate uefi's map is sorted
 		if (first) {
 			first = false;
-			kernel_map[i].start = desc->physical_start;
-			kernel_map[i].pages = desc->number_of_pages;
-			kernel_map[i].type = type;
-			kernel_map[i].attr = (desc->attribute & 0xffffffff);
+			mem_entry &ent = kernel_map[nent++];
+			ent.start = desc->physical_start;
+			ent.pages = desc->number_of_pages;
+			ent.type = type;
+			ent.attr = (desc->attribute & 0xffffffff);
 			continue;
 		}
 
-		mem_entry &prev = kernel_map[i];
+		mem_entry &prev = kernel_map[nent - 1];
 		if (can_merge(prev, type, desc)) {
 			prev.pages += desc->number_of_pages;
 		} else {
-			mem_entry &next = kernel_map[++i];
+			mem_entry &next = kernel_map[nent++];
 			next.start = desc->physical_start;
 			next.pages = desc->number_of_pages;
 			next.type = type;
@@ -233,9 +345,20 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 
 	// Give just the actually-set entries in the header
 	args->mem_map = kernel_map;
-	args->map_count = i;
+	args->map_count = nent;
 
-	return efi_map;
+	/*
+	// kernel map dump
+	for (unsigned i = 0; i < nent; ++i) {
+		const kernel::args::mem_entry &e = kernel_map[i];
+		console::print(L"   kRange %lx (%lx) %x(%s) [%lu]\r\n",
+			e.start, e.attr, e.type, kernel_memory_type_name(e.type), e.pages);
+	}
+	*/
+
+	build_kernel_frame_blocks(kernel_map, nent, args, bs);
+	get_uefi_mappings(map, bs);
+	return map;
 }
 
 void

src/boot/memory.h

@@ -18,28 +18,6 @@ inline constexpr size_t bytes_to_pages(size_t bytes) {
 	return ((bytes - 1) / page_size) + 1;
 }
 
-/// \defgroup memory_types
-/// Custom UEFI memory type values used for data being passed to the kernel
-/// @{
-
-/// Memory containing the kernel args structure
-constexpr uefi::memory_type args_type =
-	static_cast<uefi::memory_type>(0x80000000);
-
-/// Memory containing any loaded modules to be passed to the kernel
-constexpr uefi::memory_type module_type =
-	static_cast<uefi::memory_type>(0x80000001);
-
-/// Memory containing loaded kernel or program code and data sections
-constexpr uefi::memory_type program_type =
-	static_cast<uefi::memory_type>(0x80000002);
-
-/// Memory containing page tables set up by the loader
-constexpr uefi::memory_type table_type =
-	static_cast<uefi::memory_type>(0x80000003);
-
-/// @}
-
 /// \defgroup pointer_fixup
 /// Memory virtualization pointer fixup functions. Handles changing affected pointers
 /// when calling UEFI's `set_virtual_address_map` function to change the location of
@@ -63,12 +41,13 @@ struct efi_mem_map
 	using iterator = offset_iterator<desc>;
 
 	size_t length;     ///< Total length of the map data
+	size_t total;      ///< Total allocated space for map data
 	size_t size;       ///< Size of an entry in the array
 	size_t key;        ///< Key for detecting changes
 	uint32_t version;  ///< Version of the `memory_descriptor` struct
 	desc *entries;     ///< The array of UEFI descriptors
 
-	efi_mem_map() : length(0), size(0), key(0), version(0), entries(nullptr) {}
+	efi_mem_map() : length(0), total(0), size(0), key(0), version(0), entries(nullptr) {}
 
 	/// Get the count of entries in the array
 	inline size_t num_entries() const { return length / size; }
@@ -84,6 +63,14 @@ struct efi_mem_map
 /// \returns  The uefi memory map used to build the kernel map
 efi_mem_map build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs);
 
+/// Create the kernel frame allocation maps
+void build_kernel_frame_blocks(
+		const kernel::args::mem_entry *map, size_t nent,
+		kernel::args::header *args, uefi::boot_services *bs);
+
+/// Map the frame allocation maps to the right spot and fix up pointers
+void fix_frame_blocks(kernel::args::header *args);
+
 /// Activate the given memory mappings. Sets the given page tables live as well
 /// as informs UEFI runtime services of the new mappings.
 /// \arg pml4  The root page table for the new mappings

src/boot/paging.cpp

@@ -6,6 +6,7 @@
 #include "memory.h"
 #include "paging.h"
 #include "pointer_manipulation.h"
+#include "status.h"
 
 namespace boot {
 namespace paging {
@@ -14,7 +15,7 @@ using memory::page_size;
 using ::memory::pml4e_kernel;
 using ::memory::table_entries;
 
-// Flags: 0 0 0 1  0 0 0 0  0 0 1 1 = 0x0103
+// Flags: 0 0 0 1  0 0 0 0  0 0 0 1 = 0x0101
 //         IGN  |  | | | |  | | | +- Present
 //              |  | | | |  | | +--- Writeable
 //              |  | | | |  | +----- Usermode access (supervisor only)
@@ -25,9 +26,9 @@ using ::memory::table_entries;
 //              |  +---------------- PAT (determining memory type for page)
 //              +------------------- Global
 /// Page table entry flags for entries pointing at a page
-constexpr uint16_t page_flags = 0x103;
+constexpr uint64_t page_flags = 0x101;
 
-// Flags: 0  0 0 0 1  1 0 0 0  0 0 1 1 = 0x0183
+// Flags: 0  0 0 0 1  1 0 0 0  1 0 1 1 = 0x018b
 //        |   IGN  |  | | | |  | | | +- Present
 //        |        |  | | | |  | | +--- Writeable
 //        |        |  | | | |  | +----- Supervisor only
@@ -39,7 +40,7 @@ constexpr uint16_t page_flags = 0x103;
 //        |        +------------------- Global
 //        +---------------------------- PAT (determining memory type for page)
 /// Page table entry flags for entries pointing at a huge page
-constexpr uint16_t huge_page_flags = 0x183;
+constexpr uint64_t huge_page_flags = 0x18b;
 
 // Flags: 0 0 0 0  0 0 0 0  0 0 1 1 = 0x0003
 //        IGNORED  | | | |  | | | +- Present
@@ -51,7 +52,7 @@ constexpr uint16_t huge_page_flags = 0x183;
 //                 | +-------------- Ignored
 //                 +---------------- Reserved 0 (Table pointer, not page)
 /// Page table entry flags for entries pointing at another table
-constexpr uint16_t table_flags = 0x003;
+constexpr uint64_t table_flags = 0x003;
 
 /// Iterator over page table entries.
 template <unsigned D = 4>
@@ -117,7 +118,7 @@ private:
 
 		if (!(parent_ent & 1)) {
 			if (!m_cache_count--)
-				error::raise(uefi::status::out_of_resources, L"Page table cache empty");
+				error::raise(uefi::status::out_of_resources, L"Page table cache empty", 0x7ab1e5);
 
 			page_table *table = reinterpret_cast<page_table*>(m_page_cache);
 			m_page_cache = offset_ptr<void>(m_page_cache, page_size);
@@ -190,7 +191,7 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
 	status_line status(L"Allocating initial page tables");
 
 	static constexpr size_t pd_tables = 256;   // number of pages for kernelspace PDs
-	static constexpr size_t extra_tables = 49; // number of extra pages
+	static constexpr size_t extra_tables = 64; // number of extra pages
 
 	// number of pages for kernelspace PDs + PML4
 	static constexpr size_t kernel_tables = pd_tables + 1;
@@ -201,42 +202,61 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
 	try_or_raise(
 		bs->allocate_pages(
 			uefi::allocate_type::any_pages,
-			memory::table_type,
+			uefi::memory_type::loader_data,
 			tables_needed,
 			&addr),
 		L"Error allocating page table pages.");
 
 	bs->set_mem(addr, tables_needed*page_size, 0);
 
-	args->pml4 = addr;
+	page_table *pml4 = reinterpret_cast<page_table*>(addr);
+
+	args->pml4 = pml4;
+	args->table_pages = tables_needed;
 	args->table_count = tables_needed - 1;
 	args->page_tables = offset_ptr<void>(addr, page_size);
 
-	page_table *pml4 = reinterpret_cast<page_table*>(addr);
+	console::print(L"    First page (pml4) at: 0x%lx\r\n", pml4);
+
 	add_kernel_pds(pml4, args->page_tables, args->table_count);
 	add_offset_mappings(pml4, args->page_tables, args->table_count);
 
-	console::print(L"    Set up initial mappings, %d spare tables.\r\n", args->table_count);
+	//console::print(L"    Set up initial mappings, %d spare tables.\r\n", args->table_count);
+}
+
+template <typename E>
+constexpr bool has_flag(E set, E flag) {
+	return
+		(static_cast<uint64_t>(set) & static_cast<uint64_t>(flag)) ==
+		 static_cast<uint64_t>(flag);
 }
 
 void
 map_pages(
 	kernel::args::header *args,
 	uintptr_t phys, uintptr_t virt,
-	size_t size)
+	size_t count, bool write_flag, bool exe_flag)
 {
+	if (!count)
+		return;
+
 	paging::page_table *pml4 =
 		reinterpret_cast<paging::page_table*>(args->pml4);
 
-	size_t pages = memory::bytes_to_pages(size);
 	page_entry_iterator<4> iterator{
 		virt, pml4,
 		args->page_tables,
 		args->table_count};
 
+	uint64_t flags = page_flags;
+	if (!exe_flag)
+		flags |= (1ull << 63); // set NX bit
+	if (write_flag)
+		flags |= 2;
+
 	while (true) {
-		*iterator = phys | page_flags;
-		if (--pages == 0)
+		*iterator = phys | flags;
+		if (--count == 0)
 			break;
 
 		iterator.increment();
@@ -244,6 +264,24 @@ map_pages(
 	}
 }
 
+void
+map_section(
+	kernel::args::header *args,
+	const kernel::args::program_section &section)
+{
+	using kernel::args::section_flags;
+
+	size_t pages = memory::bytes_to_pages(section.size);
+
+	map_pages(
+		args,
+		section.phys_addr,
+		section.virt_addr,
+		pages,
+		has_flag(section.type, section_flags::write),
+		has_flag(section.type, section_flags::execute));
+}
+
 
 } // namespace paging
 } // namespace boot

src/boot/paging.h

@@ -38,15 +38,22 @@ void allocate_tables(
 /// tables in the current PML4.
 void add_current_mappings(page_table *new_pml4);
 
-/// Map a physical address to a virtual address in the given page tables.
-/// \arg args  The kernel args header, used for the page table cache and pml4
-/// \arg phys  The phyiscal address to map in
-/// \arg virt  The virtual address to map in
-/// \arg size  The size in bytes of the mapping 
+/// Map physical memory pages to virtual addresses in the given page tables.
+/// \arg args    The kernel args header, used for the page table cache and pml4
+/// \arg section The program section to load
 void map_pages(
 	kernel::args::header *args,
 	uintptr_t phys, uintptr_t virt,
-	size_t bytes);
+	size_t count, bool write_flag, bool exe_flag);
+
+/// Map a program section in physical memory to its virtual address in the
+/// given page tables.
+/// \arg args    The kernel args header, used for the page table cache and pml4
+/// \arg section The program section to load
+void map_section(
+	kernel::args::header *args,
+	const kernel::args::program_section &section);
+
 
 } // namespace paging
 } // namespace boot

src/boot/status.cpp

@@ -0,0 +1,265 @@
+#include <uefi/types.h>
+#include <uefi/graphics.h>
+
+#include "console.h"
+#include "error.h"
+#include "kernel_args.h"
+#include "status.h"
+
+constexpr int num_boxes = 30;
+
+namespace boot {
+
+static constexpr int level_ok = 0;
+static constexpr int level_warn = 1;
+static constexpr int level_fail = 2;
+
+static const wchar_t *level_tags[] = {
+	L"  ok  ",
+	L" warn ",
+	L"failed"
+};
+
+static const uefi::attribute level_colors[] = {
+	uefi::attribute::green,
+	uefi::attribute::brown,
+	uefi::attribute::light_red
+};
+
+status *status::s_current = nullptr;
+unsigned status::s_current_type = 0;
+unsigned status_bar::s_count = 0;
+
+status_line::status_line(const wchar_t *message, const wchar_t *context, bool fails_clean) :
+	status(fails_clean),
+	m_level(level_ok),
+	m_depth(0),
+	m_outer(nullptr)
+{
+	if (status::s_current_type == status_line::type) {
+		m_outer = static_cast<status_line*>(s_current);
+		m_depth = (m_outer ? 1 + m_outer->m_depth : 0);
+	}
+	s_current = this;
+	s_current_type = status_line::type;
+
+	auto out = console::get().m_out;
+	m_line = out->mode->cursor_row;
+
+	int indent = 2 * m_depth;
+	out->set_cursor_position(indent, m_line);
+	out->set_attribute(uefi::attribute::light_gray);
+	out->output_string(message);
+
+	if (context) {
+		out->output_string(L": ");
+		out->output_string(context);
+	}
+
+	out->output_string(L"\r\n");
+	print_status_tag();
+}
+
+status_line::~status_line()
+{
+	if (s_current != this)
+		error::raise(uefi::status::unsupported, L"Destroying non-current status_line");
+
+	if (m_outer && m_level > m_outer->m_level) {
+		m_outer->m_level = m_level;
+		m_outer->print_status_tag();
+	}
+	s_current = m_outer;
+}
+
+void
+status_line::print_status_tag()
+{
+	auto out = console::get().m_out;
+	int row = out->mode->cursor_row;
+	int col = out->mode->cursor_column;
+
+	uefi::attribute color = level_colors[m_level];
+	const wchar_t *tag = level_tags[m_level];
+
+	out->set_cursor_position(50, m_line);
+
+	out->set_attribute(uefi::attribute::light_gray);
+	out->output_string(L"[");
+	out->set_attribute(color);
+	out->output_string(tag);
+	out->set_attribute(uefi::attribute::light_gray);
+	out->output_string(L"]\r\n");
+
+	out->set_cursor_position(col, row);
+}
+
+void
+status_line::do_warn(const wchar_t *message, uefi::status status)
+{
+	auto out = console::get().m_out;
+	int row = out->mode->cursor_row;
+
+	if (m_level < level_warn) {
+		m_level = level_warn;
+		print_status_tag();
+	}
+
+	int indent = 2 + 2 * m_depth;
+	out->set_cursor_position(indent, row);
+	out->set_attribute(uefi::attribute::yellow);
+	out->output_string(message);
+
+	const wchar_t *error = error::message(status);
+	if (error) {
+		out->output_string(L": ");
+		out->output_string(error);
+	}
+
+	out->set_attribute(uefi::attribute::light_gray);
+	out->output_string(L"\r\n");
+}
+
+void
+status_line::do_fail(const wchar_t *message, uefi::status status)
+{
+	auto out = console::get().m_out;
+	int row = out->mode->cursor_row;
+
+	if (m_level < level_fail) {
+		m_level = level_fail;
+		print_status_tag();
+	}
+
+	int indent = 2 + 2 * m_depth;
+	out->set_cursor_position(indent, row);
+	out->set_attribute(uefi::attribute::red);
+	out->output_string(message);
+
+	const wchar_t *error = error::message(status);
+	if (error) {
+		out->output_string(L": ");
+		out->output_string(error);
+	}
+
+	out->set_attribute(uefi::attribute::light_gray);
+	out->output_string(L"\r\n");
+}
+
+
+
+status_bar::status_bar(kernel::args::framebuffer const &fb) :
+	status(fb.size),
+	m_outer(nullptr)
+{
+	m_size = (fb.vertical / num_boxes) - 1;
+	m_top = fb.vertical - m_size;
+	m_horiz = fb.horizontal;
+	m_fb = reinterpret_cast<uint32_t*>(fb.phys_addr);
+	m_type = static_cast<uint16_t>(fb.type);
+	next();
+
+	if (status::s_current_type == status_bar::type)
+		m_outer = static_cast<status_bar*>(s_current);
+	s_current = this;
+	s_current_type = status_bar::type;
+}
+
+status_bar::~status_bar()
+{
+	if (s_current != this)
+		error::raise(uefi::status::unsupported, L"Destroying non-current status_bar");
+	draw_box();
+	s_current = m_outer;
+}
+
+void
+status_bar::do_warn(const wchar_t *message, uefi::status status)
+{
+	m_status = status;
+	if (m_level < level_warn) {
+		m_level = level_warn;
+		draw_box();
+	}
+}
+
+void
+status_bar::do_fail(const wchar_t *message, uefi::status status)
+{
+	m_status = status;
+	if (m_level < level_fail) {
+		m_level = level_fail;
+		draw_box();
+	}
+}
+
+static uint32_t
+make_color(uint8_t r, uint8_t g, uint8_t b, uint16_t type)
+{
+	switch (static_cast<kernel::args::fb_type>(type)) {
+	case kernel::args::fb_type::bgr8:
+		return
+			(static_cast<uint32_t>(b) <<  0) |
+			(static_cast<uint32_t>(g) <<  8) |
+			(static_cast<uint32_t>(r) << 16);
+
+	case kernel::args::fb_type::rgb8:
+		return
+			(static_cast<uint32_t>(r) <<  0) |
+			(static_cast<uint32_t>(g) <<  8) |
+			(static_cast<uint32_t>(b) << 16);
+	
+	default:
+		return 0;
+	}
+}
+
+void
+status_bar::draw_box()
+{
+	static const uint32_t colors[] = {0x909090, 0xf0f0f0};
+	constexpr unsigned ncolors = sizeof(colors) / sizeof(uint32_t);
+
+	if (m_fb == nullptr)
+		return;
+
+	unsigned x0 = m_current * m_size;
+	unsigned x1 = x0 + m_size - 3;
+	unsigned y0 = m_top;
+	unsigned y1 = m_top + m_size - 3;
+
+	uint32_t color = 0;
+	switch (m_level) {
+	case level_ok:
+		color = colors[m_current % ncolors];
+		break;
+	case level_warn:
+		color = make_color(0xff, 0xb2, 0x34, m_type);
+		break;
+	case level_fail:
+		color = make_color(0xfb, 0x0a, 0x1e, m_type);
+		break;
+	default:
+		color = 0;
+	}
+
+	for (unsigned y = y0; y < y1; ++y)
+		for (unsigned x = x0; x < x1; ++x)
+			m_fb[y * m_horiz + x] = color;
+
+	if (m_level > level_ok) {
+		unsigned nbars = static_cast<uint64_t>(m_status) & 0xffff;
+		constexpr unsigned bar_height = 4;
+
+		for (unsigned i = 1; i <= nbars; ++i) {
+			y0 = m_top - 2 * i * bar_height;
+			y1 = y0 + bar_height;
+
+			for (unsigned y = y0; y < y1; ++y)
+				for (unsigned x = x0; x < x1; ++x)
+					m_fb[y * m_horiz + x] = color;
+		}
+	}
+}
+
+} // namespace boot

src/boot/status.h

@@ -0,0 +1,131 @@
+/// \file status.h
+/// Status message and indicator handling
+#pragma once
+
+#include <stdint.h>
+#include <uefi/types.h>
+
+namespace kernel {
+namespace args {
+	class framebuffer;
+}
+}
+
+namespace boot {
+
+// Abstract base class for status reporters.
+class status
+{
+public:
+	status(bool fails_clean = true) : m_fails_clean(fails_clean) {}
+
+	virtual void do_warn(const wchar_t *message, uefi::status status) = 0;
+	virtual void do_fail(const wchar_t *message, uefi::status status) = 0;
+
+	/// Set the state to warning, and print a message. If the state is already at
+	/// warning or error, the state is unchanged but the message is still printed.
+	/// \arg message  The warning message to print, if text is supported
+	/// \arg status   If set, the error or warning code that should be represented
+	/// \returns      True if there was a status handler to display the warning
+	inline static bool warn(const wchar_t *message, uefi::status status = uefi::status::success) {
+		if (!s_current) return false;
+		s_current->do_warn(message, status);
+		return s_current->m_fails_clean;
+	}
+
+	/// Set the state to error, and print a message. If the state is already at
+	/// error, the state is unchanged but the message is still printed.
+	/// \arg message  The error message to print, if text is supported
+	/// \arg status   The error or warning code that should be represented
+	/// \returns      True if there was a status handler to display the failure
+	inline static bool fail(const wchar_t *message, uefi::status status) {
+		if (!s_current) return false;
+		s_current->do_fail(message, status);
+		return s_current->m_fails_clean;
+	}
+
+protected:
+	static status *s_current;
+	static unsigned s_current_type;
+
+private:
+	bool m_fails_clean;
+};
+
+/// Scoped status line reporter. Prints a message and an "OK" if no errors
+/// or warnings were reported before destruction, otherwise reports the
+/// error or warning.
+class status_line :
+	public status
+{
+public:
+	constexpr static unsigned type = 1;
+
+	/// Constructor.
+	/// \arg message  Description of the operation in progress
+	/// \arg context  If non-null, printed after `message` and a colon
+	/// \arg fails_clean If true, this object can handle printing failure
+	status_line(
+		const wchar_t *message,
+		const wchar_t *context = nullptr,
+		bool fails_clean = true);
+	~status_line();
+
+	virtual void do_warn(const wchar_t *message, uefi::status status) override;
+	virtual void do_fail(const wchar_t *message, uefi::status status) override;
+
+private:
+	void print_status_tag();
+
+	size_t m_line;
+	int m_level;
+	int m_depth;
+
+	status_line *m_outer;
+};
+
+/// Scoped status bar reporter. Draws a row of boxes along the bottom of
+/// the screen, turning one red if there's an error in that step.
+class status_bar :
+	public status
+{
+public:
+	constexpr static unsigned type = 2;
+
+	using framebuffer = kernel::args::framebuffer;
+
+	/// Constructor.
+	/// \arg fb  The framebuffer descriptor to draw to
+	status_bar(kernel::args::framebuffer const &fb);
+	~status_bar();
+
+	virtual void do_warn(const wchar_t *message, uefi::status status) override;
+	virtual void do_fail(const wchar_t *message, uefi::status status) override;
+
+	inline void next() {
+		m_current = s_count++;
+		m_level = 0;
+		m_status = uefi::status::success;
+		draw_box();
+	}
+
+private:
+	void draw_box();
+
+	uint32_t *m_fb;
+	uint32_t m_size;
+	uint32_t m_top;
+	uint32_t m_horiz;
+
+	int m_level;
+	uefi::status m_status;
+
+	uint16_t m_type;
+	uint16_t m_current;
+
+	status_bar *m_outer;
+
+	static unsigned s_count;
+};
+
+} // namespace boot

src/drivers/fb/default_font.inc

@@ -0,0 +1,390 @@
+  0x72, 0xb5, 0x4a, 0x86, 0x00, 0x00, 0x00, 0x00, 0x20, 0x00, 0x00, 0x00,
+  0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
+  0x10, 0x00, 0x00, 0x00, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x7e, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x7e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa,
+  0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x55, 0xaa, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x38, 0x44, 0x44, 0x44, 0x38, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0xf8,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0xf8, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x0f,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0xff, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x0f, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0xf8, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0xff,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0xff, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1c,
+  0x22, 0x20, 0x20, 0xf8, 0x20, 0x20, 0x72, 0x8c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10,
+  0x10, 0x10, 0x10, 0x10, 0x00, 0x00, 0x10, 0x10, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x24, 0x24, 0x24, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0x24, 0x7e, 0x24, 0x24,
+  0x24, 0x7e, 0x24, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x08,
+  0x1e, 0x20, 0x20, 0x1c, 0x02, 0x02, 0x3c, 0x08, 0x08, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x30, 0x49, 0x4a, 0x34, 0x08, 0x16, 0x29, 0x49, 0x06,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x30, 0x48, 0x48, 0x48, 0x30, 0x31,
+  0x49, 0x46, 0x46, 0x39, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10, 0x10,
+  0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x04, 0x08, 0x08, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x08,
+  0x08, 0x04, 0x00, 0x00, 0x00, 0x00, 0x20, 0x10, 0x10, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x10, 0x10, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x24, 0x18, 0x7e, 0x18, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x08, 0x08, 0x08, 0x7f, 0x08, 0x08, 0x08, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x18, 0x18, 0x08, 0x08, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x04, 0x04, 0x08, 0x08,
+  0x10, 0x10, 0x20, 0x20, 0x40, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c,
+  0x42, 0x46, 0x4a, 0x52, 0x62, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x08, 0x18, 0x28, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x42, 0x02, 0x02, 0x04,
+  0x08, 0x10, 0x20, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e,
+  0x04, 0x08, 0x1c, 0x02, 0x02, 0x02, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x04, 0x0c, 0x14, 0x24, 0x44, 0x7e, 0x04, 0x04, 0x04,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x40, 0x40, 0x7c, 0x02,
+  0x02, 0x02, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1c,
+  0x20, 0x40, 0x40, 0x7c, 0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x7e, 0x02, 0x04, 0x04, 0x08, 0x08, 0x10, 0x10, 0x10,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x42, 0x42, 0x42, 0x3c,
+  0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c,
+  0x42, 0x42, 0x42, 0x3e, 0x02, 0x02, 0x04, 0x38, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x00, 0x00, 0x00, 0x18, 0x18,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x18, 0x00,
+  0x00, 0x00, 0x18, 0x18, 0x08, 0x08, 0x10, 0x00, 0x00, 0x00, 0x00, 0x04,
+  0x08, 0x10, 0x20, 0x40, 0x20, 0x10, 0x08, 0x04, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x00, 0x00, 0x7e, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x10, 0x08, 0x04, 0x02,
+  0x04, 0x08, 0x10, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x42,
+  0x02, 0x04, 0x08, 0x10, 0x00, 0x00, 0x10, 0x10, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x1c, 0x22, 0x41, 0x4f, 0x51, 0x51, 0x51, 0x53, 0x4d, 0x40,
+  0x20, 0x1f, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x42, 0x42, 0x42,
+  0x7e, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c,
+  0x42, 0x42, 0x42, 0x7c, 0x42, 0x42, 0x42, 0x7c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x1e, 0x20, 0x40, 0x40, 0x40, 0x40, 0x40, 0x20, 0x1e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x78, 0x44, 0x42, 0x42, 0x42,
+  0x42, 0x42, 0x44, 0x78, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e,
+  0x40, 0x40, 0x40, 0x7c, 0x40, 0x40, 0x40, 0x7e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x7e, 0x40, 0x40, 0x40, 0x7c, 0x40, 0x40, 0x40, 0x40,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x1e, 0x20, 0x40, 0x40, 0x46,
+  0x42, 0x42, 0x22, 0x1e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42,
+  0x42, 0x42, 0x42, 0x7e, 0x42, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x3e, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x02, 0x02, 0x02,
+  0x02, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42,
+  0x44, 0x48, 0x50, 0x60, 0x50, 0x48, 0x44, 0x42, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x40, 0x7e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x41, 0x63, 0x55, 0x49, 0x49,
+  0x41, 0x41, 0x41, 0x41, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42,
+  0x62, 0x52, 0x4a, 0x46, 0x42, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x3c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x42, 0x42, 0x42, 0x7c,
+  0x40, 0x40, 0x40, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c,
+  0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c, 0x04, 0x02, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x7c, 0x42, 0x42, 0x42, 0x7c, 0x48, 0x44, 0x42, 0x42,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3e, 0x40, 0x40, 0x20, 0x18,
+  0x04, 0x02, 0x02, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7f,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42,
+  0x24, 0x24, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x41,
+  0x41, 0x41, 0x41, 0x49, 0x49, 0x49, 0x55, 0x63, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x41, 0x41, 0x22, 0x14, 0x08, 0x14, 0x22, 0x41, 0x41,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x41, 0x41, 0x22, 0x14, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e,
+  0x04, 0x08, 0x08, 0x10, 0x10, 0x20, 0x20, 0x7e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x1e, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10,
+  0x10, 0x1e, 0x00, 0x00, 0x00, 0x00, 0x40, 0x40, 0x20, 0x20, 0x10, 0x10,
+  0x08, 0x08, 0x04, 0x04, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x78, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x78, 0x00, 0x00,
+  0x00, 0x00, 0x10, 0x28, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0x00, 0x00, 0x00, 0x20, 0x10, 0x08,
+  0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x40, 0x40, 0x7c, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x3c, 0x42, 0x40, 0x40, 0x40, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x02, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3c, 0x42, 0x42,
+  0x7e, 0x40, 0x40, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0x10,
+  0x10, 0x7e, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x3e, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e,
+  0x02, 0x02, 0x3c, 0x00, 0x00, 0x00, 0x40, 0x40, 0x40, 0x7c, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x08,
+  0x00, 0x38, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x04, 0x04, 0x00, 0x1c, 0x04, 0x04, 0x04, 0x04, 0x04, 0x04,
+  0x04, 0x04, 0x38, 0x00, 0x00, 0x00, 0x40, 0x40, 0x40, 0x44, 0x48, 0x50,
+  0x70, 0x48, 0x44, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x77, 0x49, 0x49, 0x49, 0x49, 0x49, 0x49,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x3c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x7c,
+  0x40, 0x40, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3e, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x3e, 0x02, 0x02, 0x02, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x2e, 0x30, 0x20, 0x20, 0x20, 0x20, 0x20, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x3e, 0x40, 0x20, 0x18, 0x04, 0x02, 0x7c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10, 0x7e, 0x10, 0x10,
+  0x10, 0x10, 0x10, 0x0e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0x42, 0x42, 0x24, 0x24, 0x18, 0x18,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x41, 0x41, 0x41,
+  0x49, 0x49, 0x55, 0x63, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x41, 0x22, 0x14, 0x08, 0x14, 0x22, 0x41, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e,
+  0x02, 0x02, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e, 0x04, 0x08,
+  0x10, 0x20, 0x40, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0e, 0x10, 0x10,
+  0x10, 0x10, 0x10, 0xe0, 0x10, 0x10, 0x10, 0x10, 0x10, 0x0e, 0x00, 0x00,
+  0x00, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x00, 0x00, 0x00, 0x70, 0x08, 0x08, 0x08, 0x08, 0x08, 0x07,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x70, 0x00, 0x00, 0x00, 0x00, 0x31, 0x49,
+  0x46, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x10, 0x00,
+  0x00, 0x10, 0x10, 0x10, 0x10, 0x10, 0x10, 0x00, 0x00, 0x00, 0x08, 0x08,
+  0x1c, 0x22, 0x40, 0x40, 0x40, 0x22, 0x1c, 0x08, 0x08, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x1c, 0x22, 0x20, 0x20, 0xf8, 0x20, 0x20, 0x72, 0x8c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0x3c, 0x24, 0x24,
+  0x24, 0x3c, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x41,
+  0x22, 0x14, 0x08, 0x3e, 0x08, 0x3e, 0x08, 0x08, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x08, 0x08, 0x08, 0x08, 0x00, 0x00, 0x00, 0x08, 0x08,
+  0x08, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x22, 0x22, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x1c, 0x22, 0x41, 0x4d, 0x51, 0x51, 0x4d, 0x41, 0x22,
+  0x1c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x09, 0x12, 0x24, 0x48, 0x24, 0x12, 0x09, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x38, 0x44, 0x44, 0x44, 0x38,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x08, 0x08, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x48, 0x24, 0x12, 0x09, 0x12, 0x24, 0x48,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x08, 0x08, 0x00,
+  0x00, 0x08, 0x10, 0x20, 0x40, 0x42, 0x3c, 0x00, 0x20, 0x10, 0x00, 0x18,
+  0x18, 0x24, 0x24, 0x24, 0x7e, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00,
+  0x04, 0x08, 0x00, 0x18, 0x18, 0x24, 0x24, 0x24, 0x7e, 0x42, 0x42, 0x42,
+  0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00, 0x18, 0x18, 0x24, 0x24, 0x24,
+  0x7e, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x32, 0x4c, 0x00, 0x18,
+  0x18, 0x24, 0x24, 0x24, 0x7e, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00,
+  0x24, 0x24, 0x00, 0x18, 0x18, 0x24, 0x24, 0x24, 0x7e, 0x42, 0x42, 0x42,
+  0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x24, 0x18, 0x18, 0x24, 0x24, 0x24,
+  0x7e, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f,
+  0x14, 0x14, 0x24, 0x27, 0x3c, 0x44, 0x44, 0x47, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x1e, 0x20, 0x40, 0x40, 0x40, 0x40, 0x40, 0x20, 0x1e,
+  0x08, 0x08, 0x30, 0x00, 0x20, 0x10, 0x00, 0x7e, 0x40, 0x40, 0x40, 0x7c,
+  0x40, 0x40, 0x40, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x04, 0x08, 0x00, 0x7e,
+  0x40, 0x40, 0x40, 0x7c, 0x40, 0x40, 0x40, 0x7e, 0x00, 0x00, 0x00, 0x00,
+  0x18, 0x24, 0x00, 0x7e, 0x40, 0x40, 0x40, 0x7c, 0x40, 0x40, 0x40, 0x7e,
+  0x00, 0x00, 0x00, 0x00, 0x24, 0x24, 0x00, 0x7e, 0x40, 0x40, 0x40, 0x7c,
+  0x40, 0x40, 0x40, 0x7e, 0x00, 0x00, 0x00, 0x00, 0x10, 0x08, 0x00, 0x3e,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x04, 0x08, 0x00, 0x3e, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00, 0x3e, 0x08, 0x08, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x22, 0x22, 0x00, 0x3e,
+  0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x3c, 0x22, 0x21, 0x21, 0x79, 0x21, 0x21, 0x22, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x32, 0x4c, 0x00, 0x42, 0x62, 0x52, 0x4a, 0x46,
+  0x42, 0x42, 0x42, 0x42, 0x00, 0x00, 0x00, 0x00, 0x10, 0x08, 0x00, 0x1c,
+  0x22, 0x41, 0x41, 0x41, 0x41, 0x41, 0x22, 0x1c, 0x00, 0x00, 0x00, 0x00,
+  0x04, 0x08, 0x00, 0x1c, 0x22, 0x41, 0x41, 0x41, 0x41, 0x41, 0x22, 0x1c,
+  0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00, 0x1c, 0x22, 0x41, 0x41, 0x41,
+  0x41, 0x41, 0x22, 0x1c, 0x00, 0x00, 0x00, 0x00, 0x32, 0x4c, 0x00, 0x1c,
+  0x22, 0x41, 0x41, 0x41, 0x41, 0x41, 0x22, 0x1c, 0x00, 0x00, 0x00, 0x00,
+  0x24, 0x24, 0x00, 0x1c, 0x22, 0x41, 0x41, 0x41, 0x41, 0x41, 0x22, 0x1c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x22, 0x14,
+  0x08, 0x14, 0x22, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x3c,
+  0x42, 0x46, 0x4a, 0x52, 0x62, 0x42, 0x42, 0x3c, 0x40, 0x00, 0x00, 0x00,
+  0x20, 0x10, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x04, 0x08, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00, 0x42,
+  0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x24, 0x24, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x04, 0x08, 0x00, 0x41, 0x41, 0x22, 0x14, 0x08,
+  0x08, 0x08, 0x08, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40,
+  0x40, 0x7c, 0x42, 0x42, 0x42, 0x7c, 0x40, 0x40, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x3c, 0x42, 0x44, 0x4c, 0x42, 0x42, 0x42, 0x44, 0x58,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x10, 0x00, 0x3c, 0x02, 0x02,
+  0x3e, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x08,
+  0x00, 0x3c, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x18, 0x24, 0x00, 0x00, 0x3c, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x32, 0x4c, 0x00, 0x00, 0x3c, 0x02, 0x02,
+  0x3e, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0x24,
+  0x00, 0x3c, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x18, 0x24, 0x24, 0x18, 0x00, 0x3c, 0x02, 0x02, 0x3e, 0x42, 0x42, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x36, 0x09, 0x39,
+  0x4f, 0x48, 0x48, 0x37, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x3c, 0x42, 0x40, 0x40, 0x40, 0x42, 0x3c, 0x08, 0x08, 0x30, 0x00,
+  0x00, 0x00, 0x20, 0x10, 0x00, 0x3c, 0x42, 0x42, 0x7e, 0x40, 0x40, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x08, 0x00, 0x3c, 0x42, 0x42,
+  0x7e, 0x40, 0x40, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00,
+  0x00, 0x3c, 0x42, 0x42, 0x7e, 0x40, 0x40, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x24, 0x24, 0x00, 0x3c, 0x42, 0x42, 0x7e, 0x40, 0x40, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x08, 0x00, 0x38, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x08,
+  0x00, 0x38, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x18, 0x24, 0x00, 0x00, 0x38, 0x08, 0x08, 0x08, 0x08, 0x08, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0x24, 0x00, 0x38, 0x08, 0x08,
+  0x08, 0x08, 0x08, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x09, 0x06, 0x1a,
+  0x01, 0x1d, 0x23, 0x41, 0x41, 0x41, 0x22, 0x1c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x32, 0x4c, 0x00, 0x00, 0x7c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x08, 0x00, 0x3c, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x08,
+  0x00, 0x3c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x18, 0x24, 0x00, 0x00, 0x3c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x32, 0x4c, 0x00, 0x00, 0x3c, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0x24,
+  0x00, 0x3c, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3c, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x00, 0x18, 0x18, 0x00, 0x00, 0x7e, 0x00, 0x00, 0x18, 0x18,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x3c, 0x46, 0x4a,
+  0x52, 0x62, 0x42, 0x3c, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x20, 0x10,
+  0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x04, 0x08, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x18, 0x24, 0x00, 0x00, 0x42, 0x42, 0x42,
+  0x42, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x24, 0x24,
+  0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e, 0x00, 0x00, 0x00, 0x00,
+  0x00, 0x00, 0x04, 0x08, 0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e,
+  0x02, 0x02, 0x3c, 0x00, 0x00, 0x00, 0x40, 0x40, 0x40, 0x5c, 0x62, 0x41,
+  0x41, 0x41, 0x62, 0x5c, 0x40, 0x40, 0x40, 0x00, 0x00, 0x00, 0x24, 0x24,
+  0x00, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x3e, 0x02, 0x02, 0x3c, 0x00,
+  0xff, 0xff, 0xe2, 0x96, 0x92, 0xff, 0xff, 0xff, 0xff, 0xff, 0xc2, 0xb0,
+  0xff, 0xff, 0xff, 0xff, 0xe2, 0x94, 0x98, 0xff, 0xe2, 0x94, 0x90, 0xff,
+  0xe2, 0x94, 0x8c, 0xff, 0xe2, 0x94, 0x94, 0xff, 0xe2, 0x94, 0xbc, 0xff,
+  0xff, 0xff, 0xe2, 0x94, 0x80, 0xff, 0xff, 0xff, 0xe2, 0x94, 0x9c, 0xff,
+  0xe2, 0x94, 0xa4, 0xff, 0xe2, 0x94, 0xb4, 0xff, 0xe2, 0x94, 0xac, 0xff,
+  0xe2, 0x94, 0x82, 0xff, 0xff, 0xff, 0xff, 0xff, 0xc2, 0xa3, 0xff, 0xff,
+  0x20, 0xff, 0x21, 0xff, 0x22, 0xff, 0x23, 0xff, 0x24, 0xff, 0x25, 0xff,
+  0x26, 0xff, 0x27, 0xff, 0x28, 0xff, 0x29, 0xff, 0x2a, 0xff, 0x2b, 0xff,
+  0x2c, 0xff, 0x2d, 0xff, 0x2e, 0xff, 0x2f, 0xff, 0x30, 0xff, 0x31, 0xff,
+  0x32, 0xff, 0x33, 0xff, 0x34, 0xff, 0x35, 0xff, 0x36, 0xff, 0x37, 0xff,
+  0x38, 0xff, 0x39, 0xff, 0x3a, 0xff, 0x3b, 0xff, 0x3c, 0xff, 0x3d, 0xff,
+  0x3e, 0xff, 0x3f, 0xff, 0x40, 0xff, 0x41, 0xff, 0x42, 0xff, 0x43, 0xff,
+  0x44, 0xff, 0x45, 0xff, 0x46, 0xff, 0x47, 0xff, 0x48, 0xff, 0x49, 0xff,
+  0x4a, 0xff, 0x4b, 0xff, 0x4c, 0xff, 0x4d, 0xff, 0x4e, 0xff, 0x4f, 0xff,
+  0x50, 0xff, 0x51, 0xff, 0x52, 0xff, 0x53, 0xff, 0x54, 0xff, 0x55, 0xff,
+  0x56, 0xff, 0x57, 0xff, 0x58, 0xff, 0x59, 0xff, 0x5a, 0xff, 0x5b, 0xff,
+  0x5c, 0xff, 0x5d, 0xff, 0x5e, 0xff, 0x5f, 0xff, 0x60, 0xff, 0x61, 0xff,
+  0x62, 0xff, 0x63, 0xff, 0x64, 0xff, 0x65, 0xff, 0x66, 0xff, 0x67, 0xff,
+  0x68, 0xff, 0x69, 0xff, 0x6a, 0xff, 0x6b, 0xff, 0x6c, 0xff, 0x6d, 0xff,
+  0x6e, 0xff, 0x6f, 0xff, 0x70, 0xff, 0x71, 0xff, 0x72, 0xff, 0x73, 0xff,
+  0x74, 0xff, 0x75, 0xff, 0x76, 0xff, 0x77, 0xff, 0x78, 0xff, 0x79, 0xff,
+  0x7a, 0xff, 0x7b, 0xff, 0x7c, 0xff, 0x7d, 0xff, 0x7e, 0xff, 0xff, 0xff,
+  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xc2, 0xa0, 0xff, 0xc2, 0xa1,
+  0xff, 0xc2, 0xa2, 0xff, 0xc2, 0xa3, 0xff, 0xc2, 0xa4, 0xff, 0xc2, 0xa5,
+  0xff, 0xc2, 0xa6, 0xff, 0xff, 0xc2, 0xa8, 0xff, 0xc2, 0xa9, 0xff, 0xff,
+  0xc2, 0xab, 0xff, 0xff, 0xc2, 0xad, 0xff, 0xff, 0xff, 0xc2, 0xb0, 0xff,
+  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xc2, 0xb8, 0xff, 0xff, 0xff,
+  0xc2, 0xbb, 0xff, 0xff, 0xff, 0xff, 0xc2, 0xbf, 0xff, 0xc3, 0x80, 0xff,
+  0xc3, 0x81, 0xff, 0xc3, 0x82, 0xff, 0xc3, 0x83, 0xff, 0xc3, 0x84, 0xff,
+  0xc3, 0x85, 0xff, 0xc3, 0x86, 0xff, 0xc3, 0x87, 0xff, 0xc3, 0x88, 0xff,
+  0xc3, 0x89, 0xff, 0xc3, 0x8a, 0xff, 0xc3, 0x8b, 0xff, 0xc3, 0x8c, 0xff,
+  0xc3, 0x8d, 0xff, 0xc3, 0x8e, 0xff, 0xc3, 0x8f, 0xff, 0xc3, 0x90, 0xff,
+  0xc3, 0x91, 0xff, 0xc3, 0x92, 0xff, 0xc3, 0x93, 0xff, 0xc3, 0x94, 0xff,
+  0xc3, 0x95, 0xff, 0xc3, 0x96, 0xff, 0xc3, 0x97, 0xff, 0xc3, 0x98, 0xff,
+  0xc3, 0x99, 0xff, 0xc3, 0x9a, 0xff, 0xc3, 0x9b, 0xff, 0xc3, 0x9c, 0xff,
+  0xc3, 0x9d, 0xff, 0xc3, 0x9e, 0xff, 0xc3, 0x9f, 0xff, 0xc3, 0xa0, 0xff,
+  0xc3, 0xa1, 0xff, 0xc3, 0xa2, 0xff, 0xc3, 0xa3, 0xff, 0xc3, 0xa4, 0xff,
+  0xc3, 0xa5, 0xff, 0xc3, 0xa6, 0xff, 0xc3, 0xa7, 0xff, 0xc3, 0xa8, 0xff,
+  0xc3, 0xa9, 0xff, 0xc3, 0xaa, 0xff, 0xc3, 0xab, 0xff, 0xc3, 0xac, 0xff,
+  0xc3, 0xad, 0xff, 0xc3, 0xae, 0xff, 0xc3, 0xaf, 0xff, 0xc3, 0xb0, 0xff,
+  0xc3, 0xb1, 0xff, 0xc3, 0xb2, 0xff, 0xc3, 0xb3, 0xff, 0xc3, 0xb4, 0xff,
+  0xc3, 0xb5, 0xff, 0xc3, 0xb6, 0xff, 0xc3, 0xb7, 0xff, 0xc3, 0xb8, 0xff,
+  0xc3, 0xb9, 0xff, 0xc3, 0xba, 0xff, 0xc3, 0xbb, 0xff, 0xc3, 0xbc, 0xff,
+  0xc3, 0xbd, 0xff, 0xc3, 0xbe, 0xff, 0xc3, 0xbf, 0xff

src/drivers/fb/font.cpp

@@ -1,6 +1,7 @@
-#include "kutil/assert.h"
+#include <assert.h>
 #include "font.h"
 
+
 /* PSF2 header format
  * Taken from the Linux KBD documentation
  * http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html
@@ -27,43 +28,51 @@ struct psf2_header {
 	uint32_t height, width; // max dimensions of glyphs
 };
 
+const uint8_t default_font[] = {
+// xxd -i < font_file.psf > default_font.inc
+#include "default_font.inc"
+};
 
 font::font(void const *data) :
-	m_size(0, 0),
-	m_count(0),
-	m_data(nullptr)
+	m_sizex {0},
+	m_sizey {0},
+	m_count {0},
+	m_data {nullptr}
 {
+	if (!data)
+		data = default_font;
+
 	psf2_header const *psf2 = static_cast<psf2_header const *>(data);
 	for (int i = 0; i < sizeof(magic); ++i) {
-		kassert(psf2->magic[i] == magic[i], "Bad font magic number.");
+		assert(psf2->magic[i] == magic[i] && "Bad font magic number.");
 	}
 
 	m_data = static_cast<uint8_t const *>(data) + psf2->header_size;
-	m_size.x = psf2->width;
-	m_size.y = psf2->height;
+	m_sizex = psf2->width;
+	m_sizey = psf2->height;
 	m_count = psf2->length;
 }
 
 void
 font::draw_glyph(
-		screen *s,
+		screen &s,
 		uint32_t glyph,
 		screen::pixel_t fg,
 		screen::pixel_t bg,
 		unsigned x,
 		unsigned y) const
 {
-	unsigned bwidth = (m_size.x+7)/8;
+	unsigned bwidth = (m_sizex+7)/8;
 	uint8_t const *data = m_data + (glyph * glyph_bytes());
 
-	for (int dy = 0; dy < m_size.y; ++dy) {
+	for (int dy = 0; dy < m_sizey; ++dy) {
 		for (int dx = 0; dx < bwidth; ++dx) {
 			uint8_t byte = data[dy * bwidth + dx];
 			for (int i = 0; i < 8; ++i) {
-				if (dx*8 + i >= m_size.x) continue;
+				if (dx*8 + i >= m_sizex) break;
 				const uint8_t mask = 1 << (7-i);
 				uint32_t c = (byte & mask) ? fg : bg;
-				s->draw_pixel(x + dx*8 + i, y + dy, c);
+				s.draw_pixel(x + dx*8 + i, y + dy, c);
 			}
 		}
 	}

src/drivers/fb/font.h

@@ -0,0 +1,33 @@
+#pragma once
+#include <stdint.h>
+
+#include "screen.h"
+
+class font
+{
+public:
+	/// Constructor.
+	/// \arg data  The font data to load. If null, will load the default
+	///            built-in font.
+	font(void const *data = nullptr);
+
+	unsigned glyph_bytes() const { return m_sizey * ((m_sizex + 7) / 8); }
+	unsigned count() const { return m_count; }
+	unsigned width() const { return m_sizex; }
+	unsigned height() const { return m_sizey; }
+	bool valid() const { return m_count > 0; }
+
+	void draw_glyph(
+			screen &s,
+			uint32_t glyph,
+			screen::pixel_t fg,
+			screen::pixel_t bg,
+			unsigned x,
+			unsigned y) const;
+
+private:
+	unsigned m_sizex, m_sizey;
+	unsigned m_count;
+	uint8_t const *m_data;
+};
+

src/drivers/fb/main.cpp

@@ -0,0 +1,139 @@
+#include <stdint.h>
+#include <stdlib.h>
+#include <stdio.h>
+
+#include "j6/init.h"
+#include "j6/errors.h"
+#include "j6/flags.h"
+#include "j6/signals.h"
+#include "j6/syscalls.h"
+#include "j6/types.h"
+
+#include "font.h"
+#include "screen.h"
+#include "scrollback.h"
+
+extern "C" {
+	int main(int, const char **);
+	void _get_init(size_t *initc, struct j6_init_value **initv);
+}
+
+extern j6_handle_t __handle_sys;
+extern j6_handle_t __handle_self;
+
+struct entry
+{
+	uint8_t bytes;
+	uint8_t area;
+	uint8_t severity;
+	uint8_t sequence;
+	char message[0];
+};
+
+int
+main(int argc, const char **argv)
+{
+	j6_system_log("fb driver starting");
+
+	size_t initc = 0;
+	j6_init_value *initv = nullptr;
+	_get_init(&initc, &initv);
+
+	j6_init_framebuffer *fb = nullptr;
+	for (unsigned i = 0; i < initc; ++i) {
+		if (initv[i].type == j6_init_desc_framebuffer) {
+			fb = reinterpret_cast<j6_init_framebuffer*>(initv[i].data);
+			break;
+		}
+	}
+
+	if (!fb || fb->addr == 0) {
+		j6_system_log("fb driver didn't find a framebuffer, exiting");
+		return 1;
+	}
+
+	j6_handle_t fb_handle = j6_handle_invalid;
+	uint32_t flags =
+		j6_vm_flag_write |
+		j6_vm_flag_write_combine;
+	j6_status_t s = j6_system_map_mmio(__handle_sys, &fb_handle, fb->addr, fb->size, flags);
+	if (s != j6_status_ok) {
+		return s;
+	}
+
+	s = j6_vma_map(fb_handle, __handle_self, fb->addr);
+	if (s != j6_status_ok) {
+		return s;
+	}
+
+	const screen::pixel_order order = (fb->flags & 1) ?
+		screen::pixel_order::bgr8 : screen::pixel_order::rgb8;
+
+	screen scr(
+		reinterpret_cast<void*>(fb->addr),
+		fb->horizontal,
+		fb->vertical,
+		fb->scanline,
+		order);
+
+	font fnt;
+
+	screen::pixel_t fg = scr.color(0xb0, 0xb0, 0xb0);
+	screen::pixel_t bg = scr.color(49, 79, 128);
+	scr.fill(bg);
+	scr.update();
+
+	constexpr int margin = 2;
+	const unsigned xstride = (margin + fnt.width());
+	const unsigned ystride = (margin + fnt.height());
+	const unsigned rows = (scr.height() - margin) / ystride;
+	const unsigned cols = (scr.width() - margin) / xstride;
+
+	scrollback scroll(rows, cols);
+
+	int pending = 0;
+	constexpr int pending_threshold = 5;
+
+	j6_handle_t sys = __handle_sys;
+	size_t buffer_size = 0;
+	void *message_buffer = nullptr;
+
+	while (true) {
+		size_t size = buffer_size;
+		j6_status_t s = j6_system_get_log(sys, message_buffer, &size);
+
+		if (s == j6_err_insufficient) {
+			free(message_buffer);
+			message_buffer = malloc(size * 2);
+			buffer_size = size;
+			continue;
+		} else if (s != j6_status_ok) {
+			j6_system_log("fb driver got error from get_log, quitting");
+			return s;
+		}
+
+		if (size > 0) {
+			entry *e = reinterpret_cast<entry*>(message_buffer);
+
+			size_t eom = e->bytes - sizeof(entry);
+			e->message[eom] = 0;
+
+			scroll.add_line(e->message, eom);
+			if (++pending > pending_threshold) {
+				scroll.render(scr, fnt);
+				scr.update();
+				pending = 0;
+			}
+		} else {
+			if (pending) {
+				scroll.render(scr, fnt);
+				scr.update();
+				pending = 0;
+			}
+		}
+	}
+
+	j6_system_log("fb driver done, exiting");
+	return 0;
+}
+

src/drivers/fb/screen.cpp

@@ -0,0 +1,48 @@
+#include <stdlib.h>
+#include <string.h>
+#include "screen.h"
+
+screen::screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order) :
+	m_fb(static_cast<volatile pixel_t *>(addr)),
+	m_order(order),
+	m_scanline(scanline),
+	m_resx(hres),
+	m_resy(vres)
+{
+	const size_t size = scanline * vres;
+	m_back = reinterpret_cast<pixel_t*>(malloc(size * sizeof(pixel_t)));
+}
+
+screen::pixel_t
+screen::color(uint8_t r, uint8_t g, uint8_t b) const
+{
+	switch (m_order) {
+	case pixel_order::bgr8:
+		return
+			(static_cast<uint32_t>(b) <<  0) |
+			(static_cast<uint32_t>(g) <<  8) |
+			(static_cast<uint32_t>(r) << 16);
+
+	case pixel_order::rgb8:
+		return
+			(static_cast<uint32_t>(r) <<  0) |
+			(static_cast<uint32_t>(g) <<  8) |
+			(static_cast<uint32_t>(b) << 16);
+	}
+}
+
+void
+screen::fill(pixel_t color)
+{
+	const size_t len = m_scanline * m_resy;
+	asm volatile ( "rep stosl" : :
+		"a"(color), "c"(len), "D"(m_back) );
+}
+
+void
+screen::update()
+{
+	const size_t len = m_scanline * m_resy * sizeof(pixel_t);
+	asm volatile ( "rep movsb" : :
+		"c"(len), "S"(m_back), "D"(m_fb) );
+}

src/drivers/fb/screen.h

@@ -0,0 +1,36 @@
+#pragma once
+#include <stddef.h>
+#include <stdint.h>
+
+class screen
+{
+public:
+	using pixel_t = uint32_t;
+
+	enum class pixel_order : uint8_t { bgr8, rgb8, };
+
+	screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order);
+
+	unsigned width() const { return m_resx; }
+	unsigned height() const { return m_resy; }
+
+	pixel_t color(uint8_t r, uint8_t g, uint8_t b) const;
+
+	void fill(pixel_t color);
+
+	inline void draw_pixel(unsigned x, unsigned y, pixel_t color) {
+		const size_t index = x + y * m_scanline;
+		m_back[index] = color;
+	}
+
+	void update();
+
+private:
+	volatile pixel_t *m_fb;
+	pixel_t *m_back;
+	pixel_order m_order;
+	unsigned m_scanline;
+	unsigned m_resx, m_resy;
+
+	screen() = delete;
+};

src/drivers/fb/scrollback.cpp

@@ -0,0 +1,58 @@
+#include <stdlib.h>
+#include <string.h>
+
+#include "font.h"
+#include "screen.h"
+#include "scrollback.h"
+
+scrollback::scrollback(unsigned lines, unsigned cols, unsigned margin) :
+	m_rows {lines},
+	m_cols {cols},
+	m_count {0},
+	m_margin {margin}
+{
+	m_data = reinterpret_cast<char*>(malloc(lines*cols));
+	memset(m_data, ' ', lines*cols);
+}
+
+void
+scrollback::add_line(const char *line, size_t len)
+{
+	unsigned i = m_count++ % m_rows;
+
+	if (len > m_cols)
+		len = m_cols;
+
+	char *start = m_data + (i * m_cols);
+	memcpy(start, line, len);
+	if (len < m_cols)
+		memset(start + len, ' ', m_cols - len);
+}
+
+char *
+scrollback::get_line(unsigned i)
+{
+	unsigned line = (i + m_count) % m_rows;
+	return &m_data[line*m_cols];
+}
+
+void
+scrollback::render(screen &scr, font &fnt)
+{
+	screen::pixel_t fg = scr.color(0xb0, 0xb0, 0xb0);
+	screen::pixel_t bg = scr.color(49, 79, 128);
+
+	const unsigned xstride = (m_margin + fnt.width());
+	const unsigned ystride = (m_margin + fnt.height());
+
+	unsigned start = m_count <= m_rows ? 0 :
+		m_count  % m_rows;
+
+	for (unsigned y = 0; y < m_rows; ++y) {
+		unsigned i = (start + y) % m_rows;
+		char *line = &m_data[i*m_cols];
+		for (unsigned x = 0; x < m_cols; ++x) {
+			fnt.draw_glyph(scr, line[x], fg, bg, m_margin+x*xstride, m_margin+y*ystride);
+		}
+	}
+}

src/drivers/fb/scrollback.h

@@ -0,0 +1,25 @@
+#pragma once
+/// \file scrollback.h
+
+class screen;
+class font;
+
+class scrollback
+{
+public:
+	scrollback(unsigned lines, unsigned cols, unsigned margin = 2);
+
+	void add_line(const char *line, size_t len);
+
+	char * get_line(unsigned i);
+
+	void render(screen &scr, font &fnt);
+
+private:
+	char *m_data;
+	unsigned m_rows, m_cols;
+	unsigned m_start;
+	unsigned m_count;
+	unsigned m_margin;
+};
+

src/drivers/nulldrv/main.cpp

@@ -4,57 +4,49 @@
 #include "j6/types.h"
 #include "j6/errors.h"
 #include "j6/signals.h"
-
-#include <j6libc/syscalls.h>
+#include "j6/syscalls.h"
 
 #include "io.h"
 #include "serial.h"
 
 char inbuf[1024];
-j6_handle_t sys = j6_handle_invalid;
+extern j6_handle_t __handle_sys;
 j6_handle_t endp = j6_handle_invalid;
 
 extern "C" {
-	void _init_libc(j6_process_init *);
 	int main(int, const char **);
 }
 
 void
 thread_proc()
 {
-	_syscall_system_log("sub thread starting");
+	j6_system_log("sub thread starting");
 
 	char buffer[512];
 	size_t len = sizeof(buffer);
 	j6_tag_t tag = 0;
-	j6_status_t result = _syscall_endpoint_receive(endp, &tag, &len, (void*)buffer);
+	j6_status_t result = j6_endpoint_receive(endp, &tag, &len, (void*)buffer);
 	if (result != j6_status_ok)
-		_syscall_thread_exit(result);
+		j6_thread_exit(result);
 
-	_syscall_system_log("sub thread received message");
+	j6_system_log("sub thread received message");
 
 	for (int i = 0; i < len; ++i)
 		if (buffer[i] >= 'A' && buffer[i] <= 'Z')
 			buffer[i] += 0x20;
 
 	tag++;
-	result = _syscall_endpoint_send(endp, tag, len, (void*)buffer);
+	result = j6_endpoint_send(endp, tag, len, (void*)buffer);
 	if (result != j6_status_ok)
-		_syscall_thread_exit(result);
+		j6_thread_exit(result);
 
-	_syscall_system_log("sub thread sent message");
+	j6_system_log("sub thread sent message");
 
 	for (int i = 1; i < 5; ++i)
-		_syscall_thread_sleep(i*10);
+		j6_thread_sleep(i*10);
 
-	_syscall_system_log("sub thread exiting");
-	_syscall_thread_exit(0);
-}
-
-void
-_init_libc(j6_process_init *init)
-{
-	sys = init->handles[0];
+	j6_system_log("sub thread exiting");
+	j6_thread_exit(0);
 }
 
 int
@@ -63,7 +55,10 @@ main(int argc, const char **argv)
 	j6_handle_t child = j6_handle_invalid;
 	j6_signal_t out = 0;
 
-	_syscall_system_log("main thread starting");
+	j6_system_log("main thread starting");
+
+	for (int i = 0; i < argc; ++i)
+		j6_system_log(argv[i]);
 
 	void *base = malloc(0x1000);
 	if (!base)
@@ -73,43 +68,48 @@ main(int argc, const char **argv)
 	for (int i = 0; i < 3; ++i)
 		vma_ptr[i*100] = uint64_t(i);
 
-	_syscall_system_log("main thread wrote to memory area");
+	j6_system_log("main thread wrote to memory area");
 
-	j6_status_t result = _syscall_endpoint_create(&endp);
+	j6_status_t result = j6_endpoint_create(&endp);
 	if (result != j6_status_ok)
 		return result;
 
-	_syscall_system_log("main thread created endpoint");
+	j6_system_log("main thread created endpoint");
 
-	result = _syscall_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
+	result = j6_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
 	if (result != j6_status_ok)
 		return result;
 
-	_syscall_system_log("main thread created sub thread");
+	j6_system_log("main thread created sub thread");
 
 	char message[] = "MAIN THREAD SUCCESSFULLY CALLED SENDRECV IF THIS IS LOWERCASE";
 	size_t size = sizeof(message);
 	j6_tag_t tag = 16;
-	result = _syscall_endpoint_sendrecv(endp, &tag, &size, (void*)message);
+	result = j6_endpoint_sendrecv(endp, &tag, &size, (void*)message);
 	if (result != j6_status_ok)
 		return result;
 
 	if (tag != 17)
-		_syscall_system_log("GOT WRONG TAG FROM SENDRECV");
+		j6_system_log("GOT WRONG TAG FROM SENDRECV");
 
-	result = _syscall_system_bind_irq(sys, endp, 3);
+	result = j6_system_bind_irq(__handle_sys, endp, 3);
 	if (result != j6_status_ok)
 		return result;
 
-	_syscall_system_log(message);
+	j6_system_log(message);
 
-	_syscall_system_log("main thread waiting on child");
-
-	result = _syscall_object_wait(child, -1ull, &out);
+	j6_system_log("main thread waiting on child");
+	result = j6_object_wait(child, -1ull, &out);
 	if (result != j6_status_ok)
 		return result;
 
-	_syscall_system_log("main testing irqs");
+	j6_system_log("main thread creating a new process");
+	j6_handle_t child_proc = j6_handle_invalid;
+	result = j6_process_create(&child_proc);
+	if (result != j6_status_ok)
+		return result;
+
+	j6_system_log("main testing irqs");
 
 
 	serial_port com2(COM2);
@@ -123,21 +123,21 @@ main(int argc, const char **argv)
 
 	size_t len = 0;
 	while (true) {
-		result = _syscall_endpoint_receive(endp, &tag, &len, nullptr);
+		result = j6_endpoint_receive(endp, &tag, &len, nullptr);
 		if (result != j6_status_ok)
 			return result;
 
 		if (j6_tag_is_irq(tag))
-			_syscall_system_log("main thread got irq!");
+			j6_system_log("main thread got irq!");
 	}
 
-	_syscall_system_log("main thread closing endpoint");
+	j6_system_log("main thread closing endpoint");
 
-	result = _syscall_object_close(endp);
+	result = j6_object_close(endp);
 	if (result != j6_status_ok)
 		return result;
 
-	_syscall_system_log("main thread done, exiting");
+	j6_system_log("main thread done, exiting");
 	return 0;
 }
 

src/include/j6/flags.h

@@ -0,0 +1,10 @@
+#pragma once
+/// \file flags.h
+/// Enums used as flags for syscalls
+
+enum j6_vm_flags {
+#define VM_FLAG(name, v) j6_vm_flag_ ## name = v,
+#include "j6/tables/vm_flags.inc"
+#undef VM_FLAG
+	j6_vm_flags_MAX
+};

src/include/j6/init.h

@@ -0,0 +1,37 @@
+#pragma once
+/// \file init.h
+/// Types used in process and thread initialization
+
+#include <stdint.h>
+#include "j6/types.h"
+
+enum j6_init_type {					// `value` is a:
+	j6_init_handle_self,			// Handle to the system
+	j6_init_handle_other,			// Handle to this process
+	j6_init_desc_framebuffer		// Pointer to a j6_init_framebuffer descriptor
+};
+
+struct j6_typed_handle {
+	enum j6_object_type type;
+	j6_handle_t handle;
+};
+
+struct j6_init_value {
+	enum j6_init_type type;
+	union {
+		struct j6_typed_handle handle;
+		void *data;
+	};
+};
+
+/// Structure defining a framebuffer.
+/// `flags` has the following bits:
+///   0-3:   Pixel layout. 0000: rgb8, 0001: bgr8
+struct j6_init_framebuffer {
+	uintptr_t addr;
+	size_t size;
+	uint32_t vertical;
+	uint32_t horizontal;
+	uint32_t scanline;
+	uint32_t flags;
+};

src/include/j6/signals.h

@@ -8,9 +8,42 @@
 
 // Signals 16-47 are defined per-object-type
 
-// Process signals
+// Event signals
+#define j7_signal_event00           (1ull << 16)
+#define j7_signal_event01           (1ull << 17)
+#define j7_signal_event02           (1ull << 18)
+#define j7_signal_event03           (1ull << 19)
+#define j7_signal_event04           (1ull << 20)
+#define j7_signal_event05           (1ull << 21)
+#define j7_signal_event06           (1ull << 22)
+#define j7_signal_event07           (1ull << 23)
+#define j7_signal_event08           (1ull << 24)
+#define j7_signal_event09           (1ull << 25)
+#define j7_signal_event10           (1ull << 26)
+#define j7_signal_event11           (1ull << 27)
+#define j7_signal_event12           (1ull << 28)
+#define j7_signal_event13           (1ull << 29)
+#define j7_signal_event14           (1ull << 30)
+#define j7_signal_event15           (1ull << 31)
+#define j7_signal_event16           (1ull << 32)
+#define j7_signal_event17           (1ull << 33)
+#define j7_signal_event18           (1ull << 34)
+#define j7_signal_event19           (1ull << 35)
+#define j7_signal_event20           (1ull << 36)
+#define j7_signal_event21           (1ull << 37)
+#define j7_signal_event22           (1ull << 38)
+#define j7_signal_event23           (1ull << 39)
+#define j7_signal_event24           (1ull << 40)
+#define j7_signal_event25           (1ull << 41)
+#define j7_signal_event26           (1ull << 42)
+#define j7_signal_event27           (1ull << 43)
+#define j7_signal_event28           (1ull << 44)
+#define j7_signal_event29           (1ull << 45)
+#define j7_signal_event30           (1ull << 46)
+#define j7_signal_event31           (1ull << 47)
 
-// Thread signals
+// System signals
+#define j6_signal_system_has_log	(1ull << 16)
 
 // Channel signals
 #define j6_signal_channel_can_send	(1ull << 16)

src/include/j6/tables/log_areas.inc

@@ -1,13 +1,14 @@
 LOG(apic,   info);
 LOG(device, debug);
-LOG(paging, warn);
+LOG(paging, info);
 LOG(driver, info);
-LOG(memory, info);
+LOG(memory, debug);
 LOG(fs,     info);
-LOG(task,   debug);
-LOG(loader, info);
+LOG(task,   info);
+LOG(sched,  info);
+LOG(loader, debug);
 LOG(boot,   debug);
-LOG(syscall,debug);
+LOG(syscall,info);
 LOG(vmem,   debug);
 LOG(objs,   debug);
 LOG(timer,  debug);

src/include/j6/tables/object_types.inc

@@ -0,0 +1,12 @@
+OBJECT_TYPE( none,     0x00 )
+
+OBJECT_TYPE( system,   0x01 )
+
+OBJECT_TYPE( event,    0x02 )
+OBJECT_TYPE( channel,  0x03 )
+OBJECT_TYPE( endpoint, 0x04 )
+
+OBJECT_TYPE( vma,      0x05 )
+
+OBJECT_TYPE( process,  0x06 )
+OBJECT_TYPE( thread,   0x07 )

src/include/j6/tables/syscalls.inc

@@ -1,17 +1,21 @@
 SYSCALL(0x00, system_log,        const char *)
 SYSCALL(0x01, system_noop,       void)
-SYSCALL(0x02, system_get_log,    j6_handle_t, j6_handle_t *)
+SYSCALL(0x02, system_get_log,    j6_handle_t, void *, size_t *)
 SYSCALL(0x03, system_bind_irq,   j6_handle_t, j6_handle_t, unsigned)
+SYSCALL(0x04, system_map_mmio,   j6_handle_t, j6_handle_t *, uintptr_t, size_t, uint32_t)
 
 SYSCALL(0x08, object_koid,       j6_handle_t, j6_koid_t *)
 SYSCALL(0x09, object_wait,       j6_handle_t, j6_signal_t, j6_signal_t *)
 SYSCALL(0x0a, object_signal,     j6_handle_t, j6_signal_t)
 SYSCALL(0x0b, object_close,      j6_handle_t)
 
-SYSCALL(0x10, process_exit,      int64_t)
+SYSCALL(0x10, process_create,    j6_handle_t *)
+SYSCALL(0x11, process_start,     j6_handle_t, uintptr_t, j6_handle_t *, size_t)
+SYSCALL(0x11, process_kill,      j6_handle_t)
+SYSCALL(0x17, process_exit,      int32_t)
 
 SYSCALL(0x18, thread_create,     void *, j6_handle_t *)
-SYSCALL(0x19, thread_exit,       int64_t)
+SYSCALL(0x19, thread_exit,       int32_t)
 SYSCALL(0x1a, thread_pause,      void)
 SYSCALL(0x1b, thread_sleep,      uint64_t)
 
@@ -26,6 +30,6 @@ SYSCALL(0x2b, endpoint_sendrecv, j6_handle_t, j6_tag_t *, size_t *, void *)
 
 SYSCALL(0x30, vma_create,        j6_handle_t *, size_t, uint32_t)
 SYSCALL(0x31, vma_create_map,    j6_handle_t *, size_t, uintptr_t, uint32_t)
-SYSCALL(0x32, vma_map,           j6_handle_t, uintptr_t)
-SYSCALL(0x33, vma_unmap,         j6_handle_t)
+SYSCALL(0x32, vma_map,           j6_handle_t, j6_handle_t, uintptr_t)
+SYSCALL(0x33, vma_unmap,         j6_handle_t, j6_handle_t)
 SYSCALL(0x34, vma_resize,        j6_handle_t, size_t *)

src/include/j6/tables/vm_flags.inc

@@ -0,0 +1,9 @@
+VM_FLAG( none,            0x00000000)
+VM_FLAG( write,           0x00000001)
+VM_FLAG( exec,            0x00000002)
+VM_FLAG( zero,            0x00000010)
+VM_FLAG( contiguous,      0x00000020)
+VM_FLAG( large_pages,     0x00000100)
+VM_FLAG( huge_pages,      0x00000200)
+VM_FLAG( write_combine,   0x00001000)
+VM_FLAG( mmio,            0x00010000)

src/include/j6/types.h

@@ -18,6 +18,7 @@ typedef uint64_t j6_signal_t;
 typedef uint64_t j6_tag_t;
 
 #define j6_tag_system_flag    0x8000000000000000
+#define j6_tag_invalid        0x0000000000000000
 
 /// If all high bits except the last 16 are set, then the tag represents
 /// an IRQ.
@@ -35,15 +36,10 @@ typedef uint64_t j6_handle_t;
 #define j6_handle_id_mask 0xffffffffull
 #define j6_handle_invalid 0xffffffffull
 
-/// A process' initial data structure for communicating with the system
-struct j6_process_init
-{
-	j6_handle_t process;
-	j6_handle_t handles[3];
-};
+enum j6_object_type {
+#define OBJECT_TYPE( name, val ) j6_object_type_ ## name = val,
+#include "j6/tables/object_types.inc"
+#undef OBJECT_TYPE
 
-/// A thread's initial data structure
-struct j6_thread_init
-{
-	j6_handle_t thread;
+	j6_object_type_max
 };

src/include/kernel_args.h

@@ -11,8 +11,7 @@ constexpr uint32_t magic = 0x600dda7a;
 constexpr uint16_t version = 1;
 
 enum class mod_type : uint32_t {
-	symbol_table,
-	framebuffer
+	symbol_table
 };
 
 struct module {
@@ -21,19 +20,46 @@ struct module {
 	mod_type type;
 };
 
-struct program {
+enum class fb_type : uint16_t {
+	none,
+	rgb8,
+	bgr8
+};
+
+struct framebuffer {
+	uintptr_t phys_addr;
+	size_t size;
+	uint32_t vertical;
+	uint32_t horizontal;
+	uint16_t scanline;
+	fb_type type;
+};
+
+enum class section_flags : uint32_t {
+	none    = 0,
+	execute = 1,
+	write   = 2,
+	read    = 4,
+};
+
+struct program_section {
 	uintptr_t phys_addr;
 	uintptr_t virt_addr;
+	uint32_t size;
+	section_flags type;
+};
+
+struct program {
 	uintptr_t entrypoint;
-	size_t size;
+	uintptr_t base;
+	size_t total_size;
+	size_t num_sections;
+	program_section sections[3];
 };
 
 enum class mem_type : uint32_t {
 	free,
-	args,
-	program,
-	module,
-	table,
+	pending,
 	acpi,
 	uefi_runtime,
 	mmio,
@@ -49,6 +75,18 @@ struct mem_entry
 	uint32_t attr;
 };
 
+constexpr size_t frames_per_block = 64 * 64 * 64;
+
+struct frame_block
+{
+	uintptr_t base;
+	uint32_t count;
+	uint32_t attrs;
+	uint64_t map1;
+	uint64_t map2[64];
+	uint64_t *bitmap;
+};
+
 enum class boot_flags : uint16_t {
 	none  = 0x0000,
 	debug = 0x0001
@@ -62,6 +100,7 @@ struct header {
 	void *pml4;
 	void *page_tables;
 	size_t table_count;
+	size_t table_pages;
 
 	program *programs;
 	size_t num_programs;
@@ -72,8 +111,14 @@ struct header {
 	mem_entry *mem_map;
 	size_t map_count;
 
+	frame_block *frame_blocks;
+	size_t frame_block_count;
+	size_t frame_block_pages;
+
 	void *runtime_services;
 	void *acpi_table;
+
+	framebuffer video;
 }
 __attribute__((aligned(alignof(max_align_t))));
 

src/include/kernel_memory.h

@@ -17,7 +17,7 @@ namespace memory {
 	constexpr uintptr_t page_offset = 0xffffc00000000000ull;
 
 	/// Max number of pages for a kernel stack
-	constexpr unsigned kernel_stack_pages = 4;
+	constexpr unsigned kernel_stack_pages = 2;
 
 	/// Max number of pages for a kernel buffer
 	constexpr unsigned kernel_buffer_pages = 16;
@@ -35,11 +35,17 @@ namespace memory {
 	constexpr uintptr_t stacks_start = heap_start - kernel_max_stacks;
 
 	/// Max size of kernel buffers area
-	constexpr size_t kernel_max_buffers = 0x10000000000ull; // 1TiB
+	constexpr size_t kernel_max_buffers = 0x8000000000ull; // 512GiB
 
 	/// Start of kernel buffers
 	constexpr uintptr_t buffers_start = stacks_start - kernel_max_buffers;
 
+	/// Max size of kernel bitmap area
+	constexpr size_t kernel_max_bitmap = 0x8000000000ull; // 512GiB
+
+	/// Start of kernel bitmap
+	constexpr uintptr_t bitmap_start = buffers_start - kernel_max_bitmap;
+
 	/// First kernel space PML4 entry
 	constexpr unsigned pml4e_kernel = 256;
 
@@ -58,6 +64,11 @@ namespace memory {
 		return reinterpret_cast<T*>(a|page_offset);
 	}
 
+	/// Convert a physical address to a virtual one (in the offset-mapped area)
+	template <typename T> T * to_virtual(T *p) {
+		return to_virtual<T>(reinterpret_cast<uintptr_t>(p));
+	}
+
 	/// Get the number of pages needed for a given number of bytes.
 	/// \arg bytes  The number of bytes desired
 	/// \returns    The number of pages needed to contain the desired bytes

src/kernel/acpi_tables.h

@@ -23,7 +23,7 @@ struct acpi_table_header
 } __attribute__ ((packed));
 
 #define TABLE_HEADER(signature) \
-	static const uint32_t type_id = kutil::byteswap(signature); \
+	static constexpr uint32_t type_id = kutil::byteswap(signature); \
 	acpi_table_header header;
 
 
@@ -198,3 +198,14 @@ struct acpi_hpet
 	uint8_t attributes;
 } __attribute__ ((packed));
 
+struct acpi_bgrt
+{
+	TABLE_HEADER('BGRT');
+	uint16_t version;
+	uint8_t status;
+	uint8_t type;
+	uintptr_t address;
+	uint32_t offset_x;
+	uint32_t offset_y;
+} __attribute__ ((packed));
+

src/kernel/ap_startup.s

@@ -0,0 +1,149 @@
+%include "tasking.inc"
+
+section .ap_startup
+
+BASE equ 0x8000 ; Where the kernel will map this at runtime
+
+CR0_PE  equ (1 << 0)
+CR0_MP  equ (1 << 1)
+CR0_ET  equ (1 << 4)
+CR0_NE  equ (1 << 5)
+CR0_WP  equ (1 << 16)
+CR0_PG  equ (1 << 31)
+CR0_VAL equ CR0_PE|CR0_MP|CR0_ET|CR0_NE|CR0_WP|CR0_PG
+
+CR4_DE         equ (1 << 3)
+CR4_PAE        equ (1 << 5)
+CR4_MCE        equ (1 << 6)
+CR4_PGE        equ (1 << 7)
+CR4_OSFXSR     equ (1 << 9)
+CR4_OSCMMEXCPT equ (1 << 10)
+CR4_FSGSBASE   equ (1 << 16)
+CR4_PCIDE      equ (1 << 17)
+CR4_INIT equ CR4_PAE|CR4_PGE
+CR4_VAL equ CR4_DE|CR4_PAE|CR4_MCE|CR4_PGE|CR4_OSFXSR|CR4_OSCMMEXCPT|CR4_FSGSBASE|CR4_PCIDE
+
+EFER_MSR  equ 0xC0000080
+EFER_SCE  equ (1 << 0)
+EFER_LME  equ (1 << 8)
+EFER_NXE  equ (1 << 11)
+EFER_VAL equ EFER_SCE|EFER_LME|EFER_NXE
+
+bits 16
+default rel
+align 8
+
+global ap_startup
+ap_startup:
+	jmp .start_real
+
+align 8
+	.pml4:  dq 0
+	.cpu:   dq 0
+	.ret:   dq 0
+
+align 16
+.gdt:
+	dq 0x0        ; Null GDT entry
+
+	dq 0x00209A0000000000 ; Code
+	dq 0x0000920000000000 ; Data
+
+align 4
+.gdtd:
+	dw ($ - .gdt)
+	dd BASE + (.gdt - ap_startup)
+
+align 4
+.idtd:
+	dw 0 ; zero-length IDT descriptor
+	dd 0
+
+.start_real:
+	cli
+	cld
+
+	xor ax, ax
+	mov ds, ax
+
+	; set the temporary null IDT
+	lidt [BASE + (.idtd - ap_startup)]
+
+	; Enter long mode
+	mov eax, cr4
+	or eax, CR4_INIT
+	mov cr4, eax
+
+	mov eax, [BASE + (.pml4 - ap_startup)]
+	mov cr3, eax
+
+	mov ecx, EFER_MSR
+	rdmsr
+	or eax, EFER_VAL
+	wrmsr
+
+	mov eax, CR0_VAL
+	mov cr0, eax
+
+	; Set the temporary minimal GDT
+	lgdt [BASE + (.gdtd - ap_startup)]
+
+	jmp (1 << 3):(BASE + (.start_long - ap_startup))
+
+bits 64
+default abs
+align 8
+.start_long:
+	; set data segments
+	mov ax, (2 << 3)
+	mov ds, ax
+	mov es, ax
+	mov fs, ax
+	mov gs, ax
+	mov ss, ax
+
+	mov eax, CR4_VAL
+
+	mov rdi, [BASE + (.cpu - ap_startup)]
+	mov rax, [rdi + CPU_DATA.rsp0]
+	mov rsp, rax
+
+	mov rax, [BASE + (.ret - ap_startup)]
+	jmp rax
+
+
+global ap_startup_code_size
+ap_startup_code_size:
+	dq ($ - ap_startup)
+
+
+section .text
+global init_ap_trampoline
+init_ap_trampoline:
+	push rbp
+	mov rbp, rsp
+
+	; rdi is the kernel pml4
+	mov [BASE + (ap_startup.pml4 - ap_startup)], rdi
+
+	; rsi is the cpu data for this AP
+	mov [BASE + (ap_startup.cpu - ap_startup)], rsi
+
+	; rdx is the address to jump to
+	mov [BASE + (ap_startup.ret - ap_startup)], rdx
+
+	; rcx is the processor id
+	mov rdi, rdx
+
+	pop rbp
+	ret
+
+extern long_ap_startup
+global ap_idle
+ap_idle:
+	call long_ap_startup
+	sti
+.hang:
+	hlt
+	jmp .hang
+

src/kernel/apic.cpp

@@ -6,11 +6,18 @@
 #include "kernel_memory.h"
 #include "log.h"
 
+uint64_t lapic::s_ticks_per_us = 0;
+
+static constexpr uint16_t lapic_id         = 0x0020;
 static constexpr uint16_t lapic_spurious   = 0x00f0;
 
+static constexpr uint16_t lapic_icr_low    = 0x0300;
+static constexpr uint16_t lapic_icr_high   = 0x0310;
+
 static constexpr uint16_t lapic_lvt_timer  = 0x0320;
 static constexpr uint16_t lapic_lvt_lint0  = 0x0350;
 static constexpr uint16_t lapic_lvt_lint1  = 0x0360;
+static constexpr uint16_t lapic_lvt_error  = 0x0370;
 
 static constexpr uint16_t lapic_timer_init = 0x0380;
 static constexpr uint16_t lapic_timer_cur  = 0x0390;
@@ -25,6 +32,7 @@ apic_read(uint32_t volatile *apic, uint16_t offset)
 static void
 apic_write(uint32_t volatile *apic, uint16_t offset, uint32_t value)
 {
+	log::debug(logs::apic, "LAPIC write: %x = %08lx", offset, value);
 	*(apic + offset/sizeof(uint32_t)) = value;
 }
 
@@ -48,14 +56,58 @@ apic::apic(uintptr_t base) :
 }
 
 
-lapic::lapic(uintptr_t base, isr spurious) :
+lapic::lapic(uintptr_t base) :
 	apic(base),
 	m_divisor(0)
 {
-	apic_write(m_base, lapic_spurious, static_cast<uint32_t>(spurious));
+	apic_write(m_base, lapic_lvt_error, static_cast<uint32_t>(isr::isrAPICError));
+	apic_write(m_base, lapic_spurious, static_cast<uint32_t>(isr::isrSpurious));
 	log::info(logs::apic, "LAPIC created, base %lx", m_base);
 }
 
+uint8_t
+lapic::get_id()
+{
+	return static_cast<uint8_t>(apic_read(m_base, lapic_id) >> 24);
+}
+
+void
+lapic::send_ipi(ipi mode, uint8_t vector, uint8_t dest)
+{
+	// Wait until the APIC is ready to send
+	ipi_wait();
+
+	uint32_t command =
+		static_cast<uint32_t>(vector) |
+		static_cast<uint32_t>(mode);
+
+	apic_write(m_base, lapic_icr_high, static_cast<uint32_t>(dest) << 24);
+	apic_write(m_base, lapic_icr_low, command);
+}
+
+void
+lapic::send_ipi_broadcast(ipi mode, bool self, uint8_t vector)
+{
+	// Wait until the APIC is ready to send
+	ipi_wait();
+
+	uint32_t command =
+		static_cast<uint32_t>(vector) |
+		static_cast<uint32_t>(mode) |
+		(self ? 0 : (1 << 18)) |
+		(1 << 19);
+
+	apic_write(m_base, lapic_icr_high, 0);
+	apic_write(m_base, lapic_icr_low, command);
+}
+
+void
+lapic::ipi_wait()
+{
+	while (apic_read(m_base, lapic_icr_low) & (1<<12))
+		asm volatile ("pause" : : : "memory");
+}
+
 void
 lapic::calibrate_timer()
 {
@@ -68,14 +120,14 @@ lapic::calibrate_timer()
 	set_divisor(1);
 	apic_write(m_base, lapic_timer_init, initial);
 
-	uint64_t us = 200000;
+	uint64_t us = 20000;
 	clock::get().spinwait(us);
 
 	uint32_t remaining = apic_read(m_base, lapic_timer_cur);
-	uint32_t ticks_total = initial - remaining;
-	m_ticks_per_us = ticks_total / us;
+	uint64_t ticks_total = initial - remaining;
+	s_ticks_per_us = ticks_total / us;
 
-	log::info(logs::apic, "APIC timer ticks %d times per microsecond.", m_ticks_per_us);
+	log::info(logs::apic, "APIC timer ticks %d times per microsecond.", s_ticks_per_us);
 
 	interrupts_enable();
 }
@@ -95,7 +147,7 @@ lapic::set_divisor(uint8_t divisor)
 	case  64: divbits = 0x9; break;
 	case 128: divbits = 0xa; break;
 	default:
-		kassert(0, "Invalid divisor passed to lapic::enable_timer");
+		kassert(0, "Invalid divisor passed to lapic::set_divisor");
 	}
 
 	apic_write(m_base, lapic_timer_div, divbits);

src/kernel/apic.h

@@ -3,6 +3,7 @@
 /// Classes to control both local and I/O APICs.
 
 #include <stdint.h>
+#include "kutil/enum_bitfields.h"
 
 enum class isr : uint8_t;
 
@@ -18,6 +19,22 @@ protected:
 	uint32_t *m_base;
 };
 
+enum class ipi : uint32_t
+{
+	// Delivery modes
+	fixed    = 0x0000,
+	smi      = 0x0200,
+	nmi      = 0x0400,
+	init     = 0x0500,
+	startup  = 0x0600,
+
+	// Flags
+	deassert = 0x0000,
+	assert   = 0x4000,
+	edge     = 0x0000, ///< edge-triggered
+	level    = 0x8000, ///< level-triggered
+};
+IS_BITFIELD(ipi);
 
 /// Controller for processor-local APICs
 class lapic :
@@ -26,8 +43,26 @@ class lapic :
 public:
 	/// Constructor
 	/// \arg base      Physicl base address of the APIC's MMIO registers
-	/// \arg spurious  Vector of the spurious interrupt handler
-	lapic(uintptr_t base, isr spurious);
+	lapic(uintptr_t base);
+
+	/// Get the local APIC's ID
+	uint8_t get_id();
+
+	/// Send an inter-processor interrupt.
+	/// \arg mode   The sending mode
+	/// \arg vector The interrupt vector
+	/// \arg dest   The APIC ID of the destination
+	void send_ipi(ipi mode, uint8_t vector, uint8_t dest);
+
+	/// Send an inter-processor broadcast interrupt to all other CPUs
+	/// \arg mode   The sending mode
+	/// \arg self   If true, include this CPU in the broadcast
+	/// \arg vector The interrupt vector
+	void send_ipi_broadcast(ipi mode, bool self, uint8_t vector);
+
+	/// Wait for an IPI to finish sending. This is done automatically
+	/// before sending another IPI with send_ipi().
+	void ipi_wait();
 
 	/// Enable interrupts for the LAPIC timer.
 	/// \arg vector   Interrupt vector the timer should use
@@ -57,19 +92,14 @@ public:
 	void calibrate_timer();
 
 private:
-	inline uint64_t ticks_to_us(uint32_t ticks) const {
-		return static_cast<uint64_t>(ticks) / m_ticks_per_us;
-	}
-
-	inline uint64_t us_to_ticks(uint64_t interval) const {
-		return interval * m_ticks_per_us;
-	}
+	inline static uint64_t ticks_to_us(uint64_t ticks)    { return ticks / s_ticks_per_us; }
+	inline static uint64_t us_to_ticks(uint64_t interval) { return interval * s_ticks_per_us; }
 
 	void set_divisor(uint8_t divisor);
 	void set_repeat(bool repeat);
 
 	uint32_t m_divisor;
-	uint32_t m_ticks_per_us;
+	static uint64_t s_ticks_per_us;
 };
 
 

src/kernel/boot.s

@@ -14,8 +14,8 @@ _header:
 
 section .text
 align 16
-global _start:function (_start.end - _start)
-_start:
+global _kernel_start:function (_kernel_start.end - _kernel_start)
+_kernel_start:
 	cli
 
 	mov rsp, idle_stack_end

src/kernel/clock.cpp

@@ -16,7 +16,7 @@ clock::clock(uint64_t rate, clock::source source_func, void *data) :
 void
 clock::spinwait(uint64_t us) const
 {
-	uint64_t when = m_source(m_data) + us;
-	while (value() < when);
+	uint64_t when = value() + us;
+	while (value() < when) asm ("pause");
 }
 

src/kernel/console.cpp

@@ -4,8 +4,6 @@
 #include "kutil/no_construct.h"
 #include "kutil/printf.h"
 #include "console.h"
-#include "font.h"
-#include "screen.h"
 #include "serial.h"
 
 
@@ -15,212 +13,12 @@ static kutil::no_construct<console> __g_console_storage;
 console &g_console = __g_console_storage.value;
 
 
-class console::screen_out
-{
-public:
-	screen_out(screen *s, font *f) :
-		m_font(f),
-		m_screen(s),
-		m_size(s->width() / f->width(), s->height() / f->height()),
-		m_fg(0xffffff),
-		m_bg(0),
-		m_first(0),
-		m_data(nullptr),
-		m_attrs(nullptr),
-		m_palette(nullptr)
-	{
-		const unsigned count = m_size.size();
-		const size_t attrs_size = 2 * count;
-
-		m_data = new char[count];
-		kutil::memset(m_data, 0, count);
-
-		m_palette = new screen::pixel_t[256];
-		fill_palette();
-
-		m_attrs = new uint16_t[count];
-		set_color(7, 0); // Grey on black default
-		for (unsigned i = 0; i < count; ++i) m_attrs[i] = m_attr;
-
-		repaint();
-	}
-
-	~screen_out()
-	{
-		delete [] m_data;
-		delete [] m_palette;
-		delete [] m_attrs;
-	}
-
-	void fill_palette()
-	{
-		unsigned index = 0;
-
-		// Manually add the 16 basic ANSI colors
-		m_palette[index++] = m_screen->color(0x00, 0x00, 0x00);
-		m_palette[index++] = m_screen->color(0xcd, 0x00, 0x00);
-		m_palette[index++] = m_screen->color(0x00, 0xcd, 0x00);
-		m_palette[index++] = m_screen->color(0xcd, 0xcd, 0x00);
-		m_palette[index++] = m_screen->color(0x00, 0x00, 0xee);
-		m_palette[index++] = m_screen->color(0xcd, 0x00, 0xcd);
-		m_palette[index++] = m_screen->color(0x00, 0xcd, 0xcd);
-		m_palette[index++] = m_screen->color(0xe5, 0xe5, 0xe5);
-		m_palette[index++] = m_screen->color(0x7f, 0x7f, 0x7f);
-		m_palette[index++] = m_screen->color(0xff, 0x00, 0x00);
-		m_palette[index++] = m_screen->color(0x00, 0xff, 0x00);
-		m_palette[index++] = m_screen->color(0xff, 0xff, 0x00);
-		m_palette[index++] = m_screen->color(0x00, 0x50, 0xff);
-		m_palette[index++] = m_screen->color(0xff, 0x00, 0xff);
-		m_palette[index++] = m_screen->color(0x00, 0xff, 0xff);
-		m_palette[index++] = m_screen->color(0xff, 0xff, 0xff);
-
-		// Build the high-color portion of the table
-		const uint32_t intensity[] = {0, 0x5f, 0x87, 0xaf, 0xd7, 0xff};
-		const uint32_t intensities = sizeof(intensity) / sizeof(uint32_t);
-
-		for (uint32_t r = 0; r < intensities; ++r) {
-			for (uint32_t g = 0; g < intensities; ++g) {
-				for (uint32_t b = 0; b < intensities; ++b) {
-					m_palette[index++] = m_screen->color(
-							intensity[r], intensity[g], intensity[b]);
-				}
-			}
-		}
-
-		// Build the greyscale portion of the table
-		for (uint8_t i = 0x08; i <= 0xee; i += 10)
-			m_palette[index++] = m_screen->color(i, i, i);
-	}
-
-	void repaint()
-	{
-		m_screen->fill(m_bg);
-		if (!m_data) return;
-
-		for (unsigned y = 0; y < m_size.y; ++y) {
-			const char *line = line_pointer(y);
-			const uint16_t *attrs = attr_pointer(y);
-			for (unsigned x = 0; x < m_size.x; ++x) {
-				const uint16_t attr = attrs[x];
-
-				set_color(static_cast<uint8_t>(attr),
-						static_cast<uint8_t>(attr >> 8));
-
-				m_font->draw_glyph(
-					m_screen,
-					line[x] ? line[x] : ' ',
-					m_fg,
-					m_bg,
-					x * m_font->width(),
-					y * m_font->height());
-			}
-		}
-	}
-
-	void scroll(unsigned lines)
-	{
-		if (!m_data) {
-			m_pos.x = 0;
-			m_pos.y = 0;
-		} else {
-			unsigned bytes = lines * m_size.x;
-			char *line = line_pointer(0);
-			for (unsigned i = 0; i < bytes; ++i)
-				*line++ = 0;
-
-			m_first = (m_first + lines) % m_size.y;
-			m_pos.y -= lines;
-		}
-		repaint();
-	}
-
-	void set_color(uint8_t fg, uint8_t bg)
-	{
-		m_bg = m_palette[bg];
-		m_fg = m_palette[fg];
-		m_attr = (bg << 8) | fg;
-	}
-
-	void putc(char c)
-	{
-		char *line = line_pointer(m_pos.y);
-		uint16_t *attrs = attr_pointer(m_pos.y);
-
-
-		switch (c) {
-		case '\t':
-			m_pos.x = (m_pos.x + 4) / 4 * 4;
-			break;
-
-		case '\r':
-			m_pos.x = 0;
-			break;
-
-		case '\n':
-			m_pos.x = 0;
-			m_pos.y++;
-			break;
-
-		default: {
-				if (line) line[m_pos.x] = c;
-				if (attrs) attrs[m_pos.x] = m_attr;
-
-				const unsigned x = m_pos.x * m_font->width();
-				const unsigned y = m_pos.y * m_font->height();
-				m_font->draw_glyph(m_screen, c, m_fg, m_bg, x, y);
-
-				m_pos.x++;
-			}
-		}
-
-		if (m_pos.x >= m_size.x) {
-			m_pos.x = m_pos.x % m_size.x;
-			m_pos.y++;
-		}
-
-		if (m_pos.y >= m_size.y) {
-			scroll(1);
-			line = line_pointer(m_pos.y);
-		}
-	}
-
-private:
-	char * line_pointer(unsigned line)
-	{
-		if (!m_data) return nullptr;
-		return m_data + ((m_first + line) % m_size.y) * m_size.x;
-	}
-
-	uint16_t * attr_pointer(unsigned line)
-	{
-		if (!m_attrs) return nullptr;
-		return m_attrs + ((m_first + line) % m_size.y) * m_size.x;
-	}
-
-	font *m_font;
-	screen *m_screen;
-
-	kutil::coord<unsigned> m_size;
-	kutil::coord<unsigned> m_pos;
-	screen::pixel_t m_fg, m_bg;
-	uint16_t m_attr;
-
-	size_t m_first;
-
-	char *m_data;
-	uint16_t *m_attrs;
-	screen::pixel_t *m_palette;
-};
-
-
 console::console() :
-	m_screen(nullptr),
 	m_serial(nullptr)
 {
 }
 
 console::console(serial_port *serial) :
-	m_screen(nullptr),
 	m_serial(serial)
 {
 	if (m_serial) {
@@ -239,9 +37,6 @@ console::echo()
 void
 console::set_color(uint8_t fg, uint8_t bg)
 {
-	if (m_screen)
-		m_screen->set_color(fg, bg);
-
 	if (m_serial) {
 		const char *fgseq = "\x1b[38;5;";
 		while (*fgseq)
@@ -276,8 +71,6 @@ console::puts(const char *message)
 void
 console::putc(char c)
 {
-	if (m_screen) m_screen->putc(c);
-
 	if (m_serial) {
 		if (c == '\n') m_serial->write('\r');
 		m_serial->write(c);
@@ -291,9 +84,3 @@ void console::vprintf(const char *fmt, va_list args)
 	vsnprintf_(buffer, buf_size, fmt, args);
 	puts(buffer);
 }
-
-void
-console::init_screen(screen *s, font *f)
-{
-	m_screen = new screen_out(s, f);
-}

src/kernel/console.h

@@ -2,9 +2,7 @@
 #include <stdarg.h>
 #include <stdint.h>
 
-class font;
 struct kernel_data;
-class screen;
 class serial_port;
 
 class console
@@ -35,13 +33,9 @@ public:
 
 	void echo();
 
-	void init_screen(screen *s, font *f);
-
 	static console * get();
 
 private:
-	class screen_out;
-	screen_out *m_screen;
 	serial_port *m_serial;
 };
 

src/kernel/cpu.cpp

@@ -1,116 +1,66 @@
 #include <stdint.h>
+#include "kutil/assert.h"
 #include "kutil/memory.h"
 #include "cpu.h"
+#include "cpu/cpu_id.h"
+#include "device_manager.h"
+#include "gdt.h"
+#include "idt.h"
+#include "kernel_memory.h"
 #include "log.h"
+#include "msr.h"
+#include "objects/vm_area.h"
+#include "syscall.h"
+#include "tss.h"
 
-cpu_data bsp_cpu_data;
-
-static constexpr uint32_t cpuid_extended = 0x80000000;
-
-
-inline static void
-__cpuid(
-	uint32_t leaf,
-	uint32_t subleaf,
-	uint32_t *eax,
-	uint32_t *ebx = nullptr,
-	uint32_t *ecx = nullptr,
-	uint32_t *edx = nullptr)
-{
-	uint32_t a, b, c, d;
-	__asm__ __volatile__ ( "cpuid"
-			: "=a"(a), "=b"(b), "=c"(c), "=d"(d)
-			: "a"(leaf), "c"(subleaf)
-			);
-	if (eax) *eax = a;
-	if (ebx) *ebx = b;
-	if (ecx) *ecx = c;
-	if (edx) *edx = d;
-}
-
-cpu_id::cpu_id() :
-	m_features(0)
-{
-	__cpuid(0, 0,
-		&m_high_basic,
-		reinterpret_cast<uint32_t *>(&m_vendor_id[0]),
-		reinterpret_cast<uint32_t *>(&m_vendor_id[8]),
-		reinterpret_cast<uint32_t *>(&m_vendor_id[4]));
-
-	__cpuid(cpuid_extended, 0, &m_high_ext);
-
-	if (m_high_ext >= cpuid_extended + 4) {
-		__cpuid(cpuid_extended + 2, 0,
-			reinterpret_cast<uint32_t *>(&m_brand_name[0]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[4]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[8]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[12]));
-		__cpuid(cpuid_extended + 3, 0,
-			reinterpret_cast<uint32_t *>(&m_brand_name[16]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[20]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[24]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[28]));
-		__cpuid(cpuid_extended + 4, 0,
-			reinterpret_cast<uint32_t *>(&m_brand_name[32]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[36]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[40]),
-			reinterpret_cast<uint32_t *>(&m_brand_name[44]));
-	} else {
-		m_brand_name[0] = 0;
-	}
-}
-
-cpu_id::regs
-cpu_id::get(uint32_t leaf, uint32_t sub) const
-{
-	regs ret {0, 0, 0, 0};
-
-	if ((leaf & cpuid_extended) == 0 && leaf > m_high_basic) return ret;
-	if ((leaf & cpuid_extended) != 0 && leaf > m_high_ext) return ret;
-
-	__cpuid(leaf, sub, &ret.eax, &ret.ebx, &ret.ecx, &ret.edx);
-	return ret;
-}
+cpu_data g_bsp_cpu_data;
 
 void
-cpu_id::validate()
+cpu_validate()
 {
-	bool fail = false;
-	uint32_t leaf = 0;
-	uint32_t sub = 0;
-	regs r;
+	cpu::cpu_id cpu;
 
-	log::info(logs::boot, "CPU: %s", brand_name());
-	log::debug(logs::boot, "    Vendor is %s", vendor_id());
+	log::info(logs::boot, "CPU: %s", cpu.brand_name());
+	log::debug(logs::boot, "    Vendor is %s", cpu.vendor_id());
 
-	log::debug(logs::boot, "    Higest basic CPUID: 0x%02x", highest_basic());
-	log::debug(logs::boot, "    Higest ext CPUID:   0x%02x", highest_ext() & ~cpuid_extended);
+	log::debug(logs::boot, "    Higest basic CPUID: 0x%02x", cpu.highest_basic());
+	log::debug(logs::boot, "    Higest ext CPUID:   0x%02x", cpu.highest_ext() & ~cpu::cpu_id::cpuid_extended);
 
-#define CPU_FEATURE_OPT(name, feat_leaf, feat_sub, regname, bit) \
-	if (leaf != feat_leaf || sub != feat_sub) { \
-		leaf = feat_leaf; sub = feat_sub; r = get(leaf, sub); \
-	} \
-	if (r.regname & (1ull << bit)) \
-		m_features |= (1ull << static_cast<uint64_t>(cpu_feature::name)); \
-	log::debug(logs::boot, "    Supports %9s: %s", #name, (r.regname & (1ull << bit)) ? "yes" : "no");
+#define CPU_FEATURE_OPT(name, ...) \
+	log::debug(logs::boot, "    Supports %9s: %s", #name, cpu.has_feature(cpu::feature::name) ? "yes" : "no");
 
 #define CPU_FEATURE_REQ(name, feat_leaf, feat_sub, regname, bit) \
 	CPU_FEATURE_OPT(name, feat_leaf, feat_sub, regname, bit); \
-	if ((r.regname & (1ull << bit)) == 0) { \
-		log::error(logs::boot, "CPU missing required feature " #name); \
-		fail = true; \
-	}
+	kassert(cpu.has_feature(cpu::feature::name), "Missing required CPU feature " #name );
 
-#include "cpu_features.inc"
+#include "cpu/features.inc"
 #undef CPU_FEATURE_OPT
 #undef CPU_FEATURE_REQ
-
-	if (fail)
-		log::fatal(logs::boot, "CPU not supported.");
 }
 
-bool
-cpu_id::has_feature(cpu_feature feat)
+void
+cpu_early_init(cpu_data *cpu)
 {
-	return (m_features & (1 << static_cast<uint64_t>(feat))) != 0;
+	IDT::get().install();
+	cpu->gdt->install();
+
+	// Install the GS base pointint to the cpu_data
+	wrmsr(msr::ia32_gs_base, reinterpret_cast<uintptr_t>(cpu));
+}
+
+void
+cpu_init(cpu_data *cpu, bool bsp)
+{
+	if (!bsp) {
+		// The BSP already called cpu_early_init
+		cpu_early_init(cpu);
+	}
+
+	// Set up the syscall MSRs
+	syscall_enable();
+
+	// Set up the page attributes table
+	uint64_t pat = rdmsr(msr::ia32_pat);
+	pat = (pat & 0x00ffffffffffffffull) | (0x01ull << 56); // set PAT 7 to WC
+	wrmsr(msr::ia32_pat, pat);
 }

src/kernel/cpu.h

@@ -2,9 +2,12 @@
 
 #include <stdint.h>
 
+class GDT;
+class lapic;
+class process;
 struct TCB;
 class thread;
-class process;
+class TSS;
 
 struct cpu_state
 {
@@ -18,78 +21,39 @@ struct cpu_state
 /// version in 'tasking.inc'
 struct cpu_data
 {
+	cpu_data *self;
+	uint16_t id;
+	uint16_t index;
+	uint32_t reserved;
 	uintptr_t rsp0;
 	uintptr_t rsp3;
 	TCB *tcb;
-	thread *t;
-	process *p;
+	thread *thread;
+	process *process;
+	TSS *tss;
+	GDT *gdt;
+
+	// Members beyond this point do not appear in
+	// the assembly version
+	lapic *apic;
 };
 
-extern cpu_data bsp_cpu_data;
+extern "C" cpu_data * _current_gsbase();
 
-/// Enum of the cpu features jsix cares about
-enum class cpu_feature {
-#define CPU_FEATURE_REQ(name, ...)  name,
-#define CPU_FEATURE_OPT(name, ...)  name,
-#include "cpu_features.inc"
-#undef CPU_FEATURE_OPT
-#undef CPU_FEATURE_REQ
-	max
-};
+/// Set up the running CPU. This sets GDT, IDT, and necessary MSRs as well as creating
+/// the cpu_data structure for this processor.
+/// \arg cpu  The cpu_data structure for this CPU
+/// \arg bsp  True if this CPU is the BSP
+void cpu_init(cpu_data *cpu, bool bsp);
 
-class cpu_id
-{
-public:
-	/// CPUID result register values
-	struct regs {
-		union {
-			uint32_t reg[4];
-			uint32_t eax, ebx, ecx, edx;
-		};
+/// Do early (before cpu_init) initialization work. Only needs to be called manually for
+/// the BSP, otherwise cpu_init will call it.
+/// \arg cpu  The cpu_data structure for this CPU
+void cpu_early_init(cpu_data *cpu);
 
-		/// Return true if bit |bit| of EAX is set
-		bool eax_bit(unsigned bit) { return (eax >> bit) & 0x1; }
+/// Get the cpu_data struct for the current executing CPU
+inline cpu_data & current_cpu() { return *_current_gsbase(); }
 
-		/// Return true if bit |bit| of EBX is set
-		bool ebx_bit(unsigned bit) { return (ebx >> bit) & 0x1; }
-
-		/// Return true if bit |bit| of ECX is set
-		bool ecx_bit(unsigned bit) { return (ecx >> bit) & 0x1; }
-
-		/// Return true if bit |bit| of EDX is set
-		bool edx_bit(unsigned bit) { return (edx >> bit) & 0x1; }
-	};
-
-	cpu_id();
-
-	/// The the result of a given CPUID leaf/subleaf
-	/// \arg leaf     The leaf selector (initial EAX)
-	/// \arg subleaf  The subleaf selector (initial ECX)
-	/// \returns      A |regs| struct of the values retuned
-	regs get(uint32_t leaf, uint32_t sub = 0) const;
-
-	/// Get the name of the cpu vendor (eg, "GenuineIntel")
-	inline const char * vendor_id() const	{ return m_vendor_id; }
-
-	/// Get the brand name of this processor model
-	inline const char * brand_name() const	{ return m_brand_name; }
-
-	/// Get the highest basic CPUID leaf supported
-	inline uint32_t highest_basic() const	{ return m_high_basic; }
-
-	/// Get the highest extended CPUID leaf supported
-	inline uint32_t highest_ext() const		{ return m_high_ext; }
-
-	/// Validate the CPU supports the necessary options for jsix
-	void validate();
-
-	/// Return true if the CPU claims to support the given feature
-	bool has_feature(cpu_feature feat);
-
-private:
-	uint32_t m_high_basic;
-	uint32_t m_high_ext;
-	char m_vendor_id[13];
-	char m_brand_name[48];
-	uint64_t m_features;
-};
+/// Validate the required CPU features are present. Really, the bootloader already
+/// validated the required features, but still iterate the options and log about them.
+void cpu_validate();

src/kernel/debug.cpp

@@ -13,6 +13,7 @@ void
 print_regs(const cpu_state &regs)
 {
 	console *cons = console::get();
+	cpu_data &cpu = current_cpu();
 
 	uint64_t cr2 = 0;
 	__asm__ __volatile__ ("mov %%cr2, %0" : "=r"(cr2));
@@ -20,8 +21,8 @@ print_regs(const cpu_state &regs)
 	uintptr_t cr3 = 0;
 	__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (cr3) );
 
-	cons->printf("       process: %llx", bsp_cpu_data.p->koid());
-	cons->printf("   thread: %llx\n", bsp_cpu_data.t->koid());
+	cons->printf("       process: %llx", cpu.process->koid());
+	cons->printf("   thread: %llx\n", cpu.thread->koid());
 
 	print_regL("rax", regs.rax);
 	print_regM("rbx", regs.rbx);
@@ -43,7 +44,7 @@ print_regs(const cpu_state &regs)
 	cons->puts("\n\n");
 	print_regL("rbp", regs.rbp);
 	print_regM("rsp", regs.user_rsp);
-	print_regR("sp0", bsp_cpu_data.rsp0);
+	print_regR("sp0", cpu.rsp0);
 
 	print_regL("rip", regs.rip);
 	print_regM("cr3", cr3);

src/kernel/debug.h

@@ -4,6 +4,8 @@
 
 #include <stdint.h>
 
+struct cpu_state;
+
 extern "C" {
 	uintptr_t get_rsp();
 	uintptr_t get_rip();

src/kernel/device_manager.cpp

@@ -38,7 +38,7 @@ struct acpi2_rsdp
 	uint32_t rsdt_address;
 
 	uint32_t length;
-	uint64_t xsdt_address;
+	acpi_table_header *xsdt_address;
 	uint8_t checksum20;
 	uint8_t reserved[3];
 } __attribute__ ((packed));
@@ -63,7 +63,7 @@ void irq4_callback(void *)
 
 
 device_manager::device_manager() :
-	m_lapic(0)
+	m_lapic_base(0)
 {
 	m_irqs.ensure_capacity(32);
 	m_irqs.set_size(16);
@@ -73,13 +73,20 @@ device_manager::device_manager() :
 	m_irqs[2] = ignore_endpoint;
 }
 
+template <typename T> static const T *
+check_get_table(const acpi_table_header *header)
+{
+	kassert(header && header->validate(T::type_id), "Invalid ACPI table.");
+	return reinterpret_cast<const T *>(header);
+}
+
 void
 device_manager::parse_acpi(const void *root_table)
 {
 	kassert(root_table != 0, "ACPI root table pointer is null.");
 
-	const acpi1_rsdp *acpi1 =
-		reinterpret_cast<const acpi1_rsdp *>(root_table);
+	const acpi1_rsdp *acpi1 = memory::to_virtual(
+		reinterpret_cast<const acpi1_rsdp *>(root_table));
 
 	for (int i = 0; i < sizeof(acpi1->signature); ++i)
 		kassert(acpi1->signature[i] == expected_signature[i],
@@ -96,7 +103,27 @@ device_manager::parse_acpi(const void *root_table)
 	sum = kutil::checksum(acpi2, sizeof(acpi2_rsdp), sizeof(acpi1_rsdp));
 	kassert(sum == 0, "ACPI 2.0 RSDP checksum mismatch.");
 
-	load_xsdt(reinterpret_cast<const acpi_xsdt *>(acpi2->xsdt_address));
+	load_xsdt(memory::to_virtual(acpi2->xsdt_address));
+}
+
+const device_manager::apic_nmi *
+device_manager::get_lapic_nmi(uint8_t id) const
+{
+	for (const auto &nmi : m_nmis) {
+		if (nmi.cpu == 0xff || nmi.cpu == id)
+			return &nmi;
+	}
+
+	return nullptr;
+}
+
+const device_manager::irq_override *
+device_manager::get_irq_override(uint8_t irq) const
+{
+	for (const auto &o : m_overrides)
+		if (o.source == irq) return &o;
+
+	return nullptr;
 }
 
 ioapic *
@@ -112,9 +139,9 @@ put_sig(char *into, uint32_t type)
 }
 
 void
-device_manager::load_xsdt(const acpi_xsdt *xsdt)
+device_manager::load_xsdt(const acpi_table_header *header)
 {
-	kassert(xsdt && acpi_validate(xsdt), "Invalid ACPI XSDT.");
+	const auto *xsdt = check_get_table<acpi_xsdt>(header);
 
 	char sig[5] = {0,0,0,0,0};
 	log::info(logs::device, "ACPI 2.0+ tables loading");
@@ -124,7 +151,8 @@ device_manager::load_xsdt(const acpi_xsdt *xsdt)
 
 	size_t num_tables = acpi_table_entries(xsdt, sizeof(void*));
 	for (size_t i = 0; i < num_tables; ++i) {
-		const acpi_table_header *header = xsdt->headers[i];
+		const acpi_table_header *header =
+			memory::to_virtual(xsdt->headers[i]);
 
 		put_sig(sig, header->type);
 		log::debug(logs::device, "  Found table %s", sig);
@@ -133,15 +161,15 @@ device_manager::load_xsdt(const acpi_xsdt *xsdt)
 
 		switch (header->type) {
 		case acpi_apic::type_id:
-			load_apic(reinterpret_cast<const acpi_apic *>(header));
+			load_apic(header);
 			break;
 
 		case acpi_mcfg::type_id:
-			load_mcfg(reinterpret_cast<const acpi_mcfg *>(header));
+			load_mcfg(header);
 			break;
 
 		case acpi_hpet::type_id:
-			load_hpet(reinterpret_cast<const acpi_hpet *>(header));
+			load_hpet(header);
 			break;
 
 		default:
@@ -151,40 +179,42 @@ device_manager::load_xsdt(const acpi_xsdt *xsdt)
 }
 
 void
-device_manager::load_apic(const acpi_apic *apic)
+device_manager::load_apic(const acpi_table_header *header)
 {
-	uintptr_t local = apic->local_address;
-	m_lapic = new lapic(local, isr::isrSpurious);
+	const auto *apic = check_get_table<acpi_apic>(header);
+
+	m_lapic_base = apic->local_address;
 
 	size_t count = acpi_table_entries(apic, 1);
 	uint8_t const *p = apic->controller_data;
 	uint8_t const *end = p + count;
 
-	// Pass one: count IOAPIC objcts
-	int num_ioapics = 0;
+	// Pass one: count objcts
+	unsigned num_lapics = 0;
+	unsigned num_ioapics = 0;
+	unsigned num_overrides = 0;
+	unsigned num_nmis = 0;
 	while (p < end) {
 		const uint8_t type = p[0];
 		const uint8_t length = p[1];
-		if (type == 1) num_ioapics++;
-		p += length;
-	}
 
-	m_ioapics.set_capacity(num_ioapics);
-
-	// Pass two: set up IOAPIC objcts
-	p = apic->controller_data;
-	while (p < end) {
-		const uint8_t type = p[0];
-		const uint8_t length = p[1];
-		if (type == 1) {
-			uintptr_t base = kutil::read_from<uint32_t>(p+4);
-			uint32_t base_gsr = kutil::read_from<uint32_t>(p+8);
-			m_ioapics.emplace(base, base_gsr);
+		switch (type) {
+		case 0: ++num_lapics; break;
+		case 1: ++num_ioapics; break;
+		case 2: ++num_overrides; break;
+		case 4: ++num_nmis; break;
+		default: break;
 		}
+
 		p += length;
 	}
 
-	// Pass three: configure APIC objects
+	m_apic_ids.set_capacity(num_lapics);
+	m_ioapics.set_capacity(num_ioapics);
+	m_overrides.set_capacity(num_overrides);
+	m_nmis.set_capacity(num_nmis);
+
+	// Pass two: configure objects
 	p = apic->controller_data;
 	while (p < end) {
 		const uint8_t type = p[0];
@@ -194,38 +224,42 @@ device_manager::load_apic(const acpi_apic *apic)
 		case 0: { // Local APIC
 				uint8_t uid = kutil::read_from<uint8_t>(p+2);
 				uint8_t id = kutil::read_from<uint8_t>(p+3);
-				log::debug(logs::device, "    Local APIC uid %x id %x", id);
+				m_apic_ids.append(id);
+
+				log::debug(logs::device, "    Local APIC uid %x id %x", uid, id);
 			}
 			break;
 
-		case 1: // I/O APIC
+		case 1: { // I/O APIC
+				uintptr_t base = kutil::read_from<uint32_t>(p+4);
+				uint32_t base_gsi = kutil::read_from<uint32_t>(p+8);
+				m_ioapics.emplace(base, base_gsi);
+
+				log::debug(logs::device, "    IO APIC gsi %x base %x", base_gsi, base);
+			}
 			break;
 
 		case 2: { // Interrupt source override
-				uint8_t source = kutil::read_from<uint8_t>(p+3);
-				isr gsi = isr::irq00 + kutil::read_from<uint32_t>(p+4);
-				uint16_t flags = kutil::read_from<uint16_t>(p+8);
+				irq_override o;
+				o.source = kutil::read_from<uint8_t>(p+3);
+				o.gsi = kutil::read_from<uint32_t>(p+4);
+				o.flags = kutil::read_from<uint16_t>(p+8);
+				m_overrides.append(o);
 
 				log::debug(logs::device, "    Intr source override IRQ %d -> %d Pol %d Tri %d",
-						source, gsi, (flags & 0x3), ((flags >> 2) & 0x3));
-
-				// TODO: in a multiple-IOAPIC system this might be elsewhere
-				m_ioapics[0].redirect(source, static_cast<isr>(gsi), flags, true);
+						o.source, o.gsi, (o.flags & 0x3), ((o.flags >> 2) & 0x3));
 			}
 			break;
 
 		case 4: {// LAPIC NMI
-			uint8_t cpu = kutil::read_from<uint8_t>(p + 2);
-			uint8_t num = kutil::read_from<uint8_t>(p + 5);
-			uint16_t flags = kutil::read_from<uint16_t>(p + 3);
+			apic_nmi nmi;
+			nmi.cpu = kutil::read_from<uint8_t>(p + 2);
+			nmi.lint = kutil::read_from<uint8_t>(p + 5);
+			nmi.flags = kutil::read_from<uint16_t>(p + 3);
+			m_nmis.append(nmi);
 
-			log::debug(logs::device, "    LAPIC NMI Proc %d LINT%d Pol %d Tri %d",
-					kutil::read_from<uint8_t>(p+2),
-					kutil::read_from<uint8_t>(p+5),
-					kutil::read_from<uint16_t>(p+3) & 0x3,
-					(kutil::read_from<uint16_t>(p+3) >> 2) & 0x3);
-
-			m_lapic->enable_lint(num, num == 0 ? isr::isrLINT0 : isr::isrLINT1, true, flags);
+			log::debug(logs::device, "    LAPIC NMI Proc %02x LINT%d Pol %d Tri %d",
+					nmi.cpu, nmi.lint, nmi.flags & 0x3, (nmi.flags >> 2) & 0x3);
 			}
 			break;
 
@@ -235,20 +269,13 @@ device_manager::load_apic(const acpi_apic *apic)
 
 		p += length;
 	}
-
-	for (uint8_t i = 0; i < m_ioapics[0].get_num_gsi(); ++i) {
-		switch (i) {
-			case 2: break;
-			default: m_ioapics[0].mask(i, false);
-		}
-	}
-
-	m_lapic->enable();
 }
 
 void
-device_manager::load_mcfg(const acpi_mcfg *mcfg)
+device_manager::load_mcfg(const acpi_table_header *header)
 {
+	const auto *mcfg = check_get_table<acpi_mcfg>(header);
+
 	size_t count = acpi_table_entries(mcfg, sizeof(acpi_mcfg_entry));
 	m_pci.set_size(count);
 	m_devices.set_capacity(16);
@@ -269,8 +296,10 @@ device_manager::load_mcfg(const acpi_mcfg *mcfg)
 }
 
 void
-device_manager::load_hpet(const acpi_hpet *hpet)
+device_manager::load_hpet(const acpi_table_header *header)
 {
+	const auto *hpet = check_get_table<acpi_hpet>(header);
+
 	log::debug(logs::device, "  Found HPET device #%3d: base %016lx  pmin %d  attr %02x",
 			hpet->index, hpet->base_address.address, hpet->periodic_min, hpet->attributes);
 
@@ -311,6 +340,13 @@ device_manager::probe_pci()
 	}
 }
 
+static uint64_t
+fake_clock_source(void*)
+{
+	static uint64_t value = 0;
+	return value++;
+}
+
 void
 device_manager::init_drivers()
 {
@@ -329,18 +365,20 @@ device_manager::init_drivers()
 		ahcid.register_device(&device);
 	}
 	*/
+	clock *master_clock = nullptr;
 	if (m_hpets.count() > 0) {
 		hpet &h = m_hpets[0];
 		h.enable();
 
 		// becomes the singleton
-		clock *master_clock = new clock(h.rate(), hpet_clock_source, &h);
-		kassert(master_clock, "Failed to allocate master clock");
+		master_clock = new clock(h.rate(), hpet_clock_source, &h);
 		log::info(logs::clock, "Created master clock using HPET 0: Rate %d", h.rate());
 	} else {
-		//TODO: APIC clock?
-		kassert(0, "No HPET master clock");
+		//TODO: Other clocks, APIC clock?
+		master_clock = new clock(5000, fake_clock_source, nullptr);
 	}
+
+	kassert(master_clock, "Failed to allocate master clock");
 }
 
 bool

src/kernel/device_manager.h

@@ -6,10 +6,7 @@
 #include "hpet.h"
 #include "pci.h"
 
-struct acpi_xsdt;
-struct acpi_apic;
-struct acpi_mcfg;
-struct acpi_hpet;
+struct acpi_table_header;
 class block_device;
 class endpoint;
 
@@ -27,10 +24,6 @@ public:
 	/// \returns  A reference to the system device manager
 	static device_manager & get() { return s_instance; }
 
-	/// Get the LAPIC
-	/// \returns An object representing the local APIC
-	lapic * get_lapic() { return m_lapic; }
-
 	/// Get an IOAPIC
 	/// \arg i   Index of the requested IOAPIC
 	/// \returns An object representing the given IOAPIC if it exists,
@@ -71,6 +64,39 @@ public:
 	/// \returns  True if the interrupt was handled
 	bool dispatch_irq(unsigned irq);
 
+	struct apic_nmi
+	{
+		uint8_t cpu;
+		uint8_t lint;
+		uint16_t flags;
+	};
+
+	struct irq_override
+	{
+		uint8_t source;
+		uint16_t flags;
+		uint32_t gsi;
+	};
+
+	/// Get the list of APIC ids for other CPUs
+	inline const kutil::vector<uint8_t> & get_apic_ids() const { return m_apic_ids; }
+
+	/// Get the LAPIC base address
+	/// \returns The physical base address of the local apic registers
+	uintptr_t get_lapic_base() const { return m_lapic_base; }
+
+	/// Get the NMI mapping for the given local APIC
+	/// \arg id  ID of the local APIC
+	/// \returns apic_nmi structure describing the NMI configuration,
+	///          or null if no configuration was provided
+	const apic_nmi * get_lapic_nmi(uint8_t id) const;
+
+	/// Get the IRQ source override for the given IRQ
+	/// \arg irq  IRQ number (not isr vector)
+	/// \returns  irq_override structure describing that IRQ's
+	///           configuration, or null if no configuration was provided
+	const irq_override * get_irq_override(uint8_t irq) const;
+
 	/// Register the existance of a block device.
 	/// \arg blockdev  Pointer to the block device
 	void register_block_device(block_device *blockdev);
@@ -100,19 +126,19 @@ public:
 private:
 	/// Parse the ACPI XSDT and load relevant sub-tables.
 	/// \arg xsdt  Pointer to the XSDT from the firmware
-	void load_xsdt(const acpi_xsdt *xsdt);
+	void load_xsdt(const acpi_table_header *xsdt);
 
 	/// Parse the ACPI MADT and initialize APICs from it.
 	/// \arg apic  Pointer to the MADT from the XSDT
-	void load_apic(const acpi_apic *apic);
+	void load_apic(const acpi_table_header *apic);
 
 	/// Parse the ACPI MCFG and initialize PCIe from it.
 	/// \arg mcfg  Pointer to the MCFG from the XSDT
-	void load_mcfg(const acpi_mcfg *mcfg);
+	void load_mcfg(const acpi_table_header *mcfg);
 
 	/// Parse the ACPI HPET and initialize an HPET from it.
 	/// \arg hpet  Pointer to the HPET from the XSDT
-	void load_hpet(const acpi_hpet *hpet);
+	void load_hpet(const acpi_table_header *hpet);
 
 	/// Probe the PCIe busses and add found devices to our
 	/// device list. The device list is destroyed and rebuilt.
@@ -122,9 +148,13 @@ private:
 	/// that has no callback.
 	void bad_irq(uint8_t irq);
 
-	lapic *m_lapic;
+	uintptr_t m_lapic_base;
+
 	kutil::vector<ioapic> m_ioapics;
 	kutil::vector<hpet> m_hpets;
+	kutil::vector<uint8_t> m_apic_ids;
+	kutil::vector<apic_nmi> m_nmis;
+	kutil::vector<irq_override> m_overrides;
 
 	kutil::vector<pci_group> m_pci;
 	kutil::vector<pci_device> m_devices;

src/kernel/font.h

@@ -1,33 +0,0 @@
-#pragma once
-#include <stdint.h>
-
-#include "kutil/coord.h"
-#include "screen.h"
-
-class font
-{
-public:
-	font(void const *data);
-
-	unsigned glyph_bytes() const { return m_size.y * ((m_size.x + 7) / 8); }
-	unsigned count() const { return m_count; }
-	unsigned width() const { return m_size.x; }
-	unsigned height() const { return m_size.y; }
-	bool valid() const { return m_count > 0; }
-
-	void draw_glyph(
-			screen *s,
-			uint32_t glyph,
-			screen::pixel_t fg,
-			screen::pixel_t bg,
-			unsigned x,
-			unsigned y) const;
-
-private:
-	kutil::coord<unsigned> m_size;
-	unsigned m_count;
-	uint8_t const *m_data;
-
-	font() = delete;
-};
-

src/kernel/frame_allocator.cpp

@@ -1,21 +1,12 @@
-#include "kernel_memory.h"
 #include "kutil/assert.h"
 #include "kutil/memory.h"
+
 #include "frame_allocator.h"
+#include "kernel_args.h"
+#include "kernel_memory.h"
+#include "log.h"
 
 using memory::frame_size;
-using memory::page_offset;
-using frame_block_node = kutil::list_node<frame_block>;
-
-int
-frame_block::compare(const frame_block &rhs) const
-{
-	if (address < rhs.address)
-		return -1;
-	else if (address > rhs.address)
-		return 1;
-	return 0;
-}
 
 
 frame_allocator &
@@ -25,54 +16,148 @@ frame_allocator::get()
 	return g_frame_allocator;
 }
 
-frame_allocator::frame_allocator() {}
+frame_allocator::frame_allocator(kernel::args::frame_block *frames, size_t count) :
+	m_blocks {frames},
+	m_count {count}
+{
+}
+
+inline unsigned
+bsf(uint64_t v)
+{
+	asm ("tzcntq %q0, %q1" : "=r"(v) : "0"(v) : "cc");
+	return v;
+}
 
 size_t
 frame_allocator::allocate(size_t count, uintptr_t *address)
 {
-	kassert(!m_free.empty(), "frame_allocator::pop_frames ran out of free frames!");
-	if (m_free.empty())
-		return 0;
+	kutil::scoped_lock lock {m_lock};
 
-	auto *first = m_free.front();
+	for (long i = m_count - 1; i >= 0; --i) {
+		frame_block &block = m_blocks[i];
 
-	if (count >= first->count) {
-		*address = first->address;
-		m_free.remove(first);
-		return first->count;
-	} else {
-		first->count -= count;
-		*address = first->address + (first->count * frame_size);
-		return count;
+		if (!block.map1)
+			continue;
+
+		// Tree walk to find the first available page
+		unsigned o1 = bsf(block.map1);
+
+		uint64_t m2 = block.map2[o1];
+		unsigned o2 = bsf(m2);
+
+		uint64_t m3 = block.bitmap[(o1 << 6) + o2];
+		unsigned o3 = bsf(m3);
+
+		unsigned frame = (o1 << 12) + (o2 << 6) + o3;
+
+		// See how many contiguous pages are here
+		unsigned n = bsf(~m3 >> o3);
+		if (n > count)
+			n = count;
+
+		*address = block.base + frame * frame_size;
+
+		// Clear the bits to mark these pages allocated
+		m3 &= ~(((1 << n) - 1) << o3);
+		block.bitmap[(o1 << 6) + o2] = m3;
+		if (!m3) {
+			// if that was it for this group, clear the next level bit
+			m2 &= ~(1 << o2);
+			block.map2[o1] = m2;
+
+			if (!m2) {
+				// if that was cleared too, update the top level
+				block.map1 &= ~(1 << o1);
+			}
+		}
+
+		return n;
 	}
-}
 
-inline uintptr_t end(frame_block *node) { return node->address + node->count * frame_size; }
+	kassert(false, "frame_allocator ran out of free frames!");
+	return 0;
+}
 
 void
 frame_allocator::free(uintptr_t address, size_t count)
 {
+	kutil::scoped_lock lock {m_lock};
+
 	kassert(address % frame_size == 0, "Trying to free a non page-aligned frame!");
 
-	frame_block_node *node =
-		reinterpret_cast<frame_block_node*>(address + page_offset);
+	if (!count)
+		return;
 
-	kutil::memset(node, 0, sizeof(frame_block_node));
-	node->address = address;
-	node->count = count;
+	for (long i = 0; i < m_count; ++i) {
+		frame_block &block = m_blocks[i];
+		uintptr_t end = block.base + block.count * frame_size;
 
-	m_free.sorted_insert(node);
+		if (address < block.base || address >= end)
+			continue;
 
-	frame_block_node *next = node->next();
-	if (next && end(node) == next->address) {
-		node->count += next->count;
-		m_free.remove(next);
-	}
+		uint64_t frame = (address - block.base) >> 12;
+		unsigned o1 = (frame >> 12) & 0x3f;
+		unsigned o2 = (frame >> 6) & 0x3f;
+		unsigned o3 = frame & 0x3f;
 
-	frame_block_node *prev = node->prev();
-	if (prev && end(prev) == address) {
-		prev->count += node->count;
-		m_free.remove(node);
+		while (count--) {
+			block.map1 |= (1 << o1);
+			block.map2[o1] |= (1 << o2);
+			block.bitmap[o2] |= (1 << o3);
+			if (++o3 == 64) {
+				o3 = 0;
+				if (++o2 == 64) {
+					o2 = 0;
+					++o1;
+					kassert(o1 < 64, "Tried to free pages past the end of a block");
+				}
+			}
+		}
+	}
+}
+
+void
+frame_allocator::used(uintptr_t address, size_t count)
+{
+	kutil::scoped_lock lock {m_lock};
+
+	kassert(address % frame_size == 0, "Trying to mark a non page-aligned frame!");
+
+	if (!count)
+		return;
+
+	for (long i = 0; i < m_count; ++i) {
+		frame_block &block = m_blocks[i];
+		uintptr_t end = block.base + block.count * frame_size;
+
+		if (address < block.base || address >= end)
+			continue;
+
+		uint64_t frame = (address - block.base) >> 12;
+		unsigned o1 = (frame >> 12) & 0x3f;
+		unsigned o2 = (frame >> 6) & 0x3f;
+		unsigned o3 = frame & 0x3f;
+
+		while (count--) {
+			block.bitmap[o2] &= ~(1 << o3);
+			if (!block.bitmap[o2]) {
+				block.map2[o1] &= ~(1 << o2);
+
+				if (!block.map2[o1]) {
+					block.map1 &= ~(1 << o1);
+				}
+			}
+
+			if (++o3 == 64) {
+				o3 = 0;
+				if (++o2 == 64) {
+					o2 = 0;
+					++o1;
+					kassert(o1 < 64, "Tried to mark pages past the end of a block");
+				}
+			}
+		}
 	}
 }
 

src/kernel/frame_allocator.h

@@ -3,18 +3,23 @@
 /// Allocator for physical memory frames
 
 #include <stdint.h>
+#include "kutil/spinlock.h"
 
-#include "kutil/linked_list.h"
-
-struct frame_block;
-using frame_block_list = kutil::linked_list<frame_block>;
+namespace kernel {
+namespace args {
+	struct frame_block;
+}}
 
 /// Allocator for physical memory frames
 class frame_allocator
 {
 public:
-	/// Default constructor
-	frame_allocator();
+	using frame_block = kernel::args::frame_block;
+
+	/// Constructor
+	/// \arg blocks The bootloader-supplied frame bitmap block list
+	/// \arg count  Number of entries in the block list
+	frame_allocator(frame_block *frames, size_t count);
 
 	/// Get free frames from the free list. Only frames from the first free block
 	/// are returned, so the number may be less than requested, but they will
@@ -29,26 +34,20 @@ public:
 	/// \arg count    The number of frames to be freed
 	void free(uintptr_t address, size_t count);
 
+	/// Mark frames as used
+	/// \arg address  The physical address of the first frame to free
+	/// \arg count    The number of frames to be freed
+	void used(uintptr_t address, size_t count);
+
 	/// Get the global frame allocator
 	static frame_allocator & get();
 
 private:
-	frame_block_list m_free; ///< Free frames list
+	frame_block *m_blocks;
+	size_t m_count;
 
+	kutil::spinlock m_lock;
+
+	frame_allocator() = delete;
 	frame_allocator(const frame_allocator &) = delete;
 };
-
-
-/// A block of contiguous frames. Each `frame_block` represents contiguous
-/// physical frames with the same attributes.
-struct frame_block
-{
-	uintptr_t address;
-	uint32_t count;
-
-	/// Compare two blocks by address.
-	/// \arg rhs   The right-hand comparator
-	/// \returns   <0 if this is sorts earlier, >0 if this sorts later, 0 for equal
-	int compare(const frame_block &rhs) const;
-};
-

src/kernel/gdt.cpp

@@ -1,35 +1,80 @@
 #include <stdint.h>
 
 #include "kutil/assert.h"
-#include "kutil/enum_bitfields.h"
 #include "kutil/memory.h"
+#include "kutil/no_construct.h"
 #include "console.h"
+#include "cpu.h"
+#include "gdt.h"
 #include "log.h"
+#include "tss.h"
+
+extern "C" void gdt_write(const void *gdt_ptr, uint16_t cs, uint16_t ds, uint16_t tr);
+
+static constexpr uint8_t kern_cs_index   = 1;
+static constexpr uint8_t kern_ss_index   = 2;
+static constexpr uint8_t user_cs32_index = 3;
+static constexpr uint8_t user_ss_index   = 4;
+static constexpr uint8_t user_cs64_index = 5;
+static constexpr uint8_t tss_index       = 6; // Note that this takes TWO GDT entries
+
+// The BSP's GDT is initialized _before_ global constructors are called,
+// so we don't want it to have a global constructor, lest it overwrite
+// the previous initialization.
+static kutil::no_construct<GDT> __g_bsp_gdt_storage;
+GDT &g_bsp_gdt = __g_bsp_gdt_storage.value;
 
 
-enum class gdt_type : uint8_t
+GDT::GDT(TSS *tss) :
+	m_tss(tss)
 {
-	accessed	= 0x01,
-	read_write	= 0x02,
-	conforming	= 0x04,
-	execute		= 0x08,
-	system		= 0x10,
-	ring1		= 0x20,
-	ring2		= 0x40,
-	ring3		= 0x60,
-	present		= 0x80
-};
-IS_BITFIELD(gdt_type);
+	kutil::memset(this, 0, sizeof(GDT));
 
-struct gdt_descriptor
+	m_ptr.limit = sizeof(m_entries) - 1;
+	m_ptr.base = &m_entries[0];
+
+	// Kernel CS/SS - always 64bit
+	set(kern_cs_index, 0, 0xfffff, true, gdt_type::read_write | gdt_type::execute);
+	set(kern_ss_index, 0, 0xfffff, true, gdt_type::read_write);
+
+	// User CS32/SS/CS64 - layout expected by SYSRET
+	set(user_cs32_index, 0, 0xfffff, false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+	set(user_ss_index,   0, 0xfffff, true,  gdt_type::ring3 | gdt_type::read_write);
+	set(user_cs64_index, 0, 0xfffff, true,  gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+
+	set_tss(tss);
+}
+
+GDT &
+GDT::current()
 {
-	uint16_t limit_low;
-	uint16_t base_low;
-	uint8_t base_mid;
-	gdt_type type;
-	uint8_t size;
-	uint8_t base_high;
-} __attribute__ ((packed));
+	cpu_data &cpu = current_cpu();
+	return *cpu.gdt;
+}
+
+void
+GDT::install() const
+{
+	gdt_write(
+		static_cast<const void*>(&m_ptr),
+		kern_cs_index << 3,
+		kern_ss_index << 3,
+		tss_index << 3);
+}
+
+void
+GDT::set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
+{
+	m_entries[i].limit_low = limit & 0xffff;
+	m_entries[i].size = (limit >> 16) & 0xf;
+	m_entries[i].size |= (is64 ? 0xa0 : 0xc0);
+
+	m_entries[i].base_low = base & 0xffff;
+	m_entries[i].base_mid = (base >> 16) & 0xff;
+	m_entries[i].base_high = (base >> 24) & 0xff;
+
+	m_entries[i].type = type | gdt_type::system | gdt_type::present;
+}
 
 struct tss_descriptor
 {
@@ -43,72 +88,16 @@ struct tss_descriptor
 	uint32_t reserved;
 } __attribute__ ((packed));
 
-struct tss_entry
-{
-	uint32_t reserved0;
-
-	uint64_t rsp[3]; // stack pointers for CPL 0-2
-	uint64_t ist[8]; // ist[0] is reserved
-
-	uint64_t reserved1;
-	uint16_t reserved2;
-	uint16_t iomap_offset;
-} __attribute__ ((packed));
-
-struct idt_descriptor
-{
-	uint16_t base_low;
-	uint16_t selector;
-	uint8_t ist;
-	uint8_t flags;
-	uint16_t base_mid;
-	uint32_t base_high;
-	uint32_t reserved;		// must be zero
-} __attribute__ ((packed));
-
-struct table_ptr
-{
-	uint16_t limit;
-	uint64_t base;
-} __attribute__ ((packed));
-
-
-gdt_descriptor g_gdt_table[10];
-idt_descriptor g_idt_table[256];
-table_ptr g_gdtr;
-table_ptr g_idtr;
-tss_entry g_tss;
-
-
-extern "C" {
-	void idt_write();
-	void idt_load();
-
-	void gdt_write(uint16_t cs, uint16_t ds, uint16_t tr);
-	void gdt_load();
-}
-
 void
-gdt_set_entry(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
-{
-	g_gdt_table[i].limit_low = limit & 0xffff;
-	g_gdt_table[i].size = (limit >> 16) & 0xf;
-	g_gdt_table[i].size |= (is64 ? 0xa0 : 0xc0);
-
-	g_gdt_table[i].base_low = base & 0xffff;
-	g_gdt_table[i].base_mid = (base >> 16) & 0xff;
-	g_gdt_table[i].base_high = (base >> 24) & 0xff;
-
-	g_gdt_table[i].type = type | gdt_type::system | gdt_type::present;
-}
-
-void
-tss_set_entry(uint8_t i, uint64_t base, uint64_t limit)
+GDT::set_tss(TSS *tss)
 {
 	tss_descriptor tssd;
+
+	size_t limit = sizeof(TSS);
 	tssd.limit_low = limit & 0xffff;
 	tssd.size = (limit >> 16) & 0xf;
 
+	uintptr_t base = reinterpret_cast<uintptr_t>(tss);
 	tssd.base_00 = base & 0xffff;
 	tssd.base_16 = (base >> 16) & 0xff;
 	tssd.base_24 = (base >> 24) & 0xff;
@@ -120,87 +109,26 @@ tss_set_entry(uint8_t i, uint64_t base, uint64_t limit)
 		gdt_type::execute |
 		gdt_type::ring3 |
 		gdt_type::present;
-	kutil::memcpy(&g_gdt_table[i], &tssd, sizeof(tss_descriptor));
+
+	kutil::memcpy(&m_entries[tss_index], &tssd, sizeof(tss_descriptor));
 }
 
 void
-idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags)
+GDT::dump(unsigned index) const
 {
-	g_idt_table[i].base_low = addr & 0xffff;
-	g_idt_table[i].base_mid = (addr >> 16) & 0xffff;
-	g_idt_table[i].base_high = (addr >> 32) & 0xffffffff;
-	g_idt_table[i].selector = selector;
-	g_idt_table[i].flags = flags;
-	g_idt_table[i].ist = 0;
-	g_idt_table[i].reserved = 0;
-}
-
-void
-tss_set_stack(int ring, uintptr_t rsp)
-{
-	kassert(ring < 3, "Bad ring passed to tss_set_stack.");
-	g_tss.rsp[ring] = rsp;
-}
-
-uintptr_t
-tss_get_stack(int ring)
-{
-	kassert(ring < 3, "Bad ring passed to tss_get_stack.");
-	return g_tss.rsp[ring];
-}
-
-void
-gdt_init()
-{
-	kutil::memset(&g_gdt_table, 0, sizeof(g_gdt_table));
-	kutil::memset(&g_idt_table, 0, sizeof(g_idt_table));
-
-	g_gdtr.limit = sizeof(g_gdt_table) - 1;
-	g_gdtr.base = reinterpret_cast<uint64_t>(&g_gdt_table);
-
-	// Kernel CS/SS - always 64bit
-	gdt_set_entry(1, 0, 0xfffff,  true, gdt_type::read_write | gdt_type::execute);
-	gdt_set_entry(2, 0, 0xfffff,  true, gdt_type::read_write);
-
-	// User CS32/SS/CS64 - layout expected by SYSRET
-	gdt_set_entry(3, 0, 0xfffff,  false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
-	gdt_set_entry(4, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write);
-	gdt_set_entry(5, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
-
-	kutil::memset(&g_tss, 0, sizeof(tss_entry));
-	g_tss.iomap_offset = sizeof(tss_entry);
-
-	uintptr_t tss_base = reinterpret_cast<uintptr_t>(&g_tss);
-
-	// Note that this takes TWO GDT entries
-	tss_set_entry(6, tss_base, sizeof(tss_entry));
-
-	gdt_write(1 << 3, 2 << 3, 6 << 3);
-
-	g_idtr.limit = sizeof(g_idt_table) - 1;
-	g_idtr.base = reinterpret_cast<uint64_t>(&g_idt_table);
-
-	idt_write();
-}
-
-void
-gdt_dump(int index)
-{
-	const table_ptr &table = g_gdtr;
-
 	console *cons = console::get();
 
-	int start = 0;
-	int count = (table.limit + 1) / sizeof(gdt_descriptor);
+	unsigned start = 0;
+	unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
 	if (index != -1) {
 		start = index;
 		count = 1;
 	} else {
-		cons->printf("         GDT: loc:%lx size:%d\n", table.base, table.limit+1);
+		cons->printf("         GDT: loc:%lx size:%d\n", m_ptr.base, m_ptr.limit+1);
 	}
 
-	const gdt_descriptor *gdt =
-		reinterpret_cast<const gdt_descriptor *>(table.base);
+	const descriptor *gdt =
+		reinterpret_cast<const descriptor *>(m_ptr.base);
 
 	for (int i = start; i < start+count; ++i) {
 		uint32_t base =
@@ -238,51 +166,3 @@ gdt_dump(int index)
 				(gdt[i].size & 0x60) == 0x40 ? "32" : "16");
 	}
 }
-
-void
-idt_dump(int index)
-{
-	const table_ptr &table = g_idtr;
-
-
-	int start = 0;
-	int count = (table.limit + 1) / sizeof(idt_descriptor);
-	if (index != -1) {
-		start = index;
-		count = 1;
-		log::info(logs::boot, "IDT FOR INDEX %02x", index);
-	} else {
-		log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", table.base, table.limit+1);
-	}
-
-	const idt_descriptor *idt =
-		reinterpret_cast<const idt_descriptor *>(table.base);
-
-	for (int i = start; i < start+count; ++i) {
-		uint64_t base =
-			(static_cast<uint64_t>(idt[i].base_high) << 32) |
-			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
-			idt[i].base_low;
-
-		char const *type;
-		switch (idt[i].flags & 0xf) {
-			case 0x5: type = " 32tsk "; break;
-			case 0x6: type = " 16int "; break;
-			case 0x7: type = " 16trp "; break;
-			case 0xe: type = " 32int "; break;
-			case 0xf: type = " 32trp "; break;
-			default:  type = " ????? "; break;
-		}
-
-		if (idt[i].flags & 0x80) {
-			log::debug(logs::boot,
-					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
-					(idt[i].selector & 0x3),
-					((idt[i].selector & 0x4) >> 2),
-					(idt[i].selector >> 3),
-					idt[i].ist,
-					type,
-					((idt[i].flags >> 5) & 0x3));
-		}
-	}
-}

src/kernel/gdt.h

@@ -1,33 +1,66 @@
 #pragma once
 /// \file gdt.h
-/// Definitions relating to system descriptor tables: GDT, IDT, TSS
+/// Definitions relating to a CPU's GDT table
 #include <stdint.h>
 
-/// Set up the GDT and TSS, and switch segment registers to point
-/// to them.
-void gdt_init();
+#include "kutil/enum_bitfields.h"
 
-/// Set an entry in the IDT
-/// \arg i         Index in the IDT (vector of the interrupt this handles)
-/// \arg addr      Address of the handler
-/// \arg selector  GDT selector to set when invoking this handler
-/// \arg flags     Descriptor flags to set
-void idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags);
+class TSS;
 
-/// Set the stack pointer for a given ring in the TSS
-/// \arg ring  Ring to set for (0-2)
-/// \arg rsp   Stack pointer to set
-void tss_set_stack(int ring, uintptr_t rsp);
+enum class gdt_type : uint8_t
+{
+	accessed	= 0x01,
+	read_write	= 0x02,
+	conforming	= 0x04,
+	execute		= 0x08,
+	system		= 0x10,
+	ring1		= 0x20,
+	ring2		= 0x40,
+	ring3		= 0x60,
+	present		= 0x80
+};
+IS_BITFIELD(gdt_type);
 
-/// Get the stack pointer for a given ring in the TSS
-/// \arg ring  Ring to get (0-2)
-/// \returns   Stack pointers for that ring
-uintptr_t tss_get_stack(int ring);
+class GDT
+{
+public:
+	GDT(TSS *tss);
 
-/// Dump information about the current GDT to the screen
-/// \arg index  Which entry to print, or -1 for all entries
-void gdt_dump(int index = -1);
+	/// Get the currently running CPU's GDT
+	static GDT & current();
 
-/// Dump information about the current IDT to the screen
-/// \arg index  Which entry to print, or -1 for all entries
-void idt_dump(int index = -1);
+	/// Install this GDT to the current CPU
+	void install() const;
+
+	/// Get the addrss of the pointer
+	inline const void * pointer() const { return static_cast<const void*>(&m_ptr); }
+
+	/// Dump debug information about the GDT to the console.
+	/// \arg index  Which entry to print, or -1 for all entries
+	void dump(unsigned index = -1) const;
+
+private:
+	void set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type);
+	void set_tss(TSS *tss);
+
+	struct descriptor
+	{
+		uint16_t limit_low;
+		uint16_t base_low;
+		uint8_t base_mid;
+		gdt_type type;
+		uint8_t size;
+		uint8_t base_high;
+	} __attribute__ ((packed, align(8)));
+
+	struct ptr
+	{
+		uint16_t limit;
+		descriptor *base;
+	} __attribute__ ((packed, align(4)));
+
+	descriptor m_entries[8];
+	TSS *m_tss;
+
+	ptr m_ptr;
+};

src/kernel/gdt.s

@@ -1,35 +0,0 @@
-extern g_idtr
-extern g_gdtr
-
-global idt_write
-idt_write:
-	lidt [rel g_idtr]
-	ret
-
-global idt_load
-idt_load:
-	sidt [rel g_idtr]
-	ret
-
-global gdt_write
-gdt_write:
-	lgdt [rel g_gdtr]
-	mov ax, si ; second arg is data segment
-	mov ds, ax
-	mov es, ax
-	mov fs, ax
-	mov gs, ax
-	mov ss, ax
-	push qword rdi ; first arg is code segment
-	lea rax, [rel .next]
-	push rax
-	o64 retf
-.next:
-	ltr dx ; third arg is the TSS
-	ret
-
-global gdt_load
-gdt_load:
-	sgdt [rel g_gdtr]
-	ret
-

src/kernel/gdtidt.s

@@ -0,0 +1,35 @@
+
+global idt_write
+idt_write:
+	lidt [rdi] ; first arg is the IDT pointer location
+	ret
+
+global idt_load
+idt_load:
+	sidt [rdi] ; first arg is where to write the idtr value
+	ret
+
+global gdt_write
+gdt_write:
+	lgdt [rdi] ; first arg is the GDT pointer location
+
+	mov ax, dx ; third arg is data segment
+	mov ds, ax
+	mov es, ax
+	mov fs, ax
+	mov gs, ax
+	mov ss, ax
+
+	push qword rsi ; second arg is code segment
+	lea rax, [rel .next]
+	push rax
+	o64 retf
+.next:
+	ltr cx ; fourth arg is the TSS
+	ret
+
+global gdt_load
+gdt_load:
+	sgdt [rdi] ; first arg is where to write the gdtr value
+	ret
+

src/kernel/idt.cpp

@@ -0,0 +1,137 @@
+#include "kutil/memory.h"
+#include "kutil/no_construct.h"
+#include "idt.h"
+#include "log.h"
+
+extern "C" {
+	void idt_write(const void *idt_ptr);
+
+#define ISR(i, s, name)  extern void name ();
+#define EISR(i, s, name) extern void name ();
+#define IRQ(i, q, name)  extern void name ();
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef EISR
+#undef ISR
+}
+
+// The IDT is initialized _before_ global constructors are called,
+// so we don't want it to have a global constructor, lest it overwrite
+// the previous initialization.
+static kutil::no_construct<IDT> __g_idt_storage;
+IDT &g_idt = __g_idt_storage.value;
+
+
+IDT::IDT()
+{
+	kutil::memset(this, 0, sizeof(IDT));
+	m_ptr.limit = sizeof(m_entries) - 1;
+	m_ptr.base = &m_entries[0];
+
+#define ISR(i, s, name)  set(i, & name, 0x08, 0x8e);
+#define EISR(i, s, name) set(i, & name, 0x08, 0x8e);
+#define IRQ(i, q, name)  set(i, & name, 0x08, 0x8e);
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef EISR
+#undef ISR
+}
+
+IDT &
+IDT::get()
+{
+	return g_idt;
+}
+
+void
+IDT::install() const
+{
+	idt_write(static_cast<const void*>(&m_ptr));
+}
+
+void
+IDT::add_ist_entries()
+{
+#define ISR(i, s, name)   if (s) { set_ist(i, s); }
+#define EISR(i, s, name)  if (s) { set_ist(i, s); }
+#define IRQ(i, q, name)
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef EISR
+#undef ISR
+}
+
+uint8_t
+IDT::used_ist_entries() const
+{
+	uint8_t entries = 0;
+
+#define ISR(i, s, name)   if (s) { entries |= (1 << s); }
+#define EISR(i, s, name)  if (s) { entries |= (1 << s); }
+#define IRQ(i, q, name)
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef EISR
+#undef ISR
+
+	return entries;
+}
+
+void
+IDT::set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags)
+{
+	uintptr_t addr = reinterpret_cast<uintptr_t>(handler);
+
+	m_entries[i].base_low = addr & 0xffff;
+	m_entries[i].base_mid = (addr >> 16) & 0xffff;
+	m_entries[i].base_high = (addr >> 32) & 0xffffffff;
+	m_entries[i].selector = selector;
+	m_entries[i].flags = flags;
+	m_entries[i].ist = 0;
+	m_entries[i].reserved = 0;
+}
+
+void
+IDT::dump(unsigned index) const
+{
+	unsigned start = 0;
+	unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
+	if (index != -1) {
+		start = index;
+		count = 1;
+		log::info(logs::boot, "IDT FOR INDEX %02x", index);
+	} else {
+		log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", m_ptr.base, m_ptr.limit+1);
+	}
+
+	const descriptor *idt =
+		reinterpret_cast<const descriptor *>(m_ptr.base);
+
+	for (int i = start; i < start+count; ++i) {
+		uint64_t base =
+			(static_cast<uint64_t>(idt[i].base_high) << 32) |
+			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
+			idt[i].base_low;
+
+		char const *type;
+		switch (idt[i].flags & 0xf) {
+			case 0x5: type = " 32tsk "; break;
+			case 0x6: type = " 16int "; break;
+			case 0x7: type = " 16trp "; break;
+			case 0xe: type = " 32int "; break;
+			case 0xf: type = " 32trp "; break;
+			default:  type = " ????? "; break;
+		}
+
+		if (idt[i].flags & 0x80) {
+			log::debug(logs::boot,
+					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
+					(idt[i].selector & 0x3),
+					((idt[i].selector & 0x4) >> 2),
+					(idt[i].selector >> 3),
+					idt[i].ist,
+					type,
+					((idt[i].flags >> 5) & 0x3));
+		}
+	}
+}

src/kernel/idt.h

@@ -0,0 +1,63 @@
+#pragma once
+/// \file idt.h
+/// Definitions relating to a CPU's IDT table
+#include <stdint.h>
+
+class IDT
+{
+public:
+	IDT();
+
+	/// Install this IDT to the current CPU
+	void install() const;
+
+	/// Add the IST entries listed in the ISR table into the IDT.
+	/// This can't be done until after memory is set up so the
+	/// stacks can be created.
+	void add_ist_entries();
+
+	/// Get the IST entry used by an entry.
+	/// \arg i   Which IDT entry to look in
+	/// \returns The IST index used by entry i, or 0 for none
+	inline uint8_t get_ist(uint8_t i) const {
+		return m_entries[i].ist;
+	}
+
+	/// Set the IST entry used by an entry.
+	/// \arg i   Which IDT entry to set
+	/// \arg ist The IST index for entry i, or 0 for none
+	void set_ist(uint8_t i, uint8_t ist) { m_entries[i].ist = ist; }
+
+	/// Get the IST entries that are used by this table, as a bitmap
+	uint8_t used_ist_entries() const;
+
+	/// Dump debug information about the IDT to the console.
+	/// \arg index  Which entry to print, or -1 for all entries
+	void dump(unsigned index = -1) const;
+
+	/// Get the global IDT
+	static IDT & get();
+
+private:
+	void set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags);
+
+	struct descriptor
+	{
+		uint16_t base_low;
+		uint16_t selector;
+		uint8_t ist;
+		uint8_t flags;
+		uint16_t base_mid;
+		uint32_t base_high;
+		uint32_t reserved;		// must be zero
+	} __attribute__ ((packed, aligned(16)));
+
+	struct ptr
+	{
+		uint16_t limit;
+		descriptor *base;
+	} __attribute__ ((packed, aligned(4)));
+
+	descriptor m_entries[256];
+	ptr m_ptr;
+};

src/kernel/interrupt_isrs.inc

@@ -1,36 +1,36 @@
-ISR (0x00, isrDivideByZero)
-ISR (0x01, isrDebug)
-ISR (0x02, isrNMI)
-ISR (0x03, isrBreakpoint)
-ISR (0x04, isrOverflow)
-ISR (0x05, isrBRE)
-ISR (0x06, isrInvalidOp)
-ISR (0x07, isrDNA)
-EISR(0x08, isrDoubleFault)
-ISR (0x09, isrCoprocessor)
-EISR(0x0a, isrInvalidTSS)
-EISR(0x0b, isrSegmentNP)
-EISR(0x0c, isrSSFault)
-EISR(0x0d, isrGPFault)
-EISR(0x0e, isrPageFault)
-ISR (0x0f, isr15)
+ISR (0x00, 0, isrDivideByZero)
+ISR (0x01, 0, isrDebug)
+ISR (0x02, 1, isrNMI)
+ISR (0x03, 0, isrBreakpoint)
+ISR (0x04, 0, isrOverflow)
+ISR (0x05, 0, isrBRE)
+ISR (0x06, 0, isrInvalidOp)
+ISR (0x07, 0, isrDNA)
+EISR(0x08, 2, isrDoubleFault)
+ISR (0x09, 0, isrCoprocessor)
+EISR(0x0a, 0, isrInvalidTSS)
+EISR(0x0b, 0, isrSegmentNP)
+EISR(0x0c, 0, isrSSFault)
+EISR(0x0d, 0, isrGPFault)
+EISR(0x0e, 3, isrPageFault)
+ISR (0x0f, 0, isr15)
 
-ISR (0x10, isrX87FPE)
-ISR (0x11, isrAlignmentChk)
-ISR (0x12, isrMachineChk)
-ISR (0x13, isrSIMDFPE)
-ISR (0x14, isrVirt)
-ISR (0x15, isr21)
-ISR (0x16, isr22)
-ISR (0x17, isr23)
-ISR (0x18, isr24)
-ISR (0x19, isr25)
-ISR (0x1a, isr26)
-ISR (0x1b, isr27)
-ISR (0x1c, isr28)
-ISR (0x1d, isr29)
-ISR (0x1e, isrSecurity)
-ISR (0x1f, isr31)
+ISR (0x10, 0, isrX87FPE)
+ISR (0x11, 0, isrAlignmentChk)
+ISR (0x12, 0, isrMachineChk)
+ISR (0x13, 0, isrSIMDFPE)
+ISR (0x14, 0, isrVirt)
+ISR (0x15, 0, isr21)
+ISR (0x16, 0, isr22)
+ISR (0x17, 0, isr23)
+ISR (0x18, 0, isr24)
+ISR (0x19, 0, isr25)
+ISR (0x1a, 0, isr26)
+ISR (0x1b, 0, isr27)
+ISR (0x1c, 0, isr28)
+ISR (0x1d, 0, isr29)
+ISR (0x1e, 0, isrSecurity)
+ISR (0x1f, 0, isr31)
 
 IRQ (0x20, 0x00, irq00)
 IRQ (0x21, 0x01, irq01)
@@ -237,28 +237,28 @@ IRQ (0xde, 0xbe, irqBE)
 IRQ (0xdf, 0xbf, irqBF)
 
 
-ISR (0xe0, isrTimer)
-ISR (0xe1, isrLINT0)
-ISR (0xe2, isrLINT1)
-ISR (0xe4, isrAssert)
+ISR (0xe0, 0, isrTimer)
+ISR (0xe1, 0, isrLINT0)
+ISR (0xe2, 0, isrLINT1)
+ISR (0xe3, 0, isrAPICError)
+ISR (0xe4, 0, isrAssert)
 
-UISR(0xee, isrSyscall)
-ISR (0xef, isrSpurious)
+ISR (0xef, 0, isrSpurious)
 
-ISR (0xf0, isrIgnore0)
-ISR (0xf1, isrIgnore1)
-ISR (0xf2, isrIgnore2)
-ISR (0xf3, isrIgnore3)
-ISR (0xf4, isrIgnore4)
-ISR (0xf5, isrIgnore5)
-ISR (0xf6, isrIgnore6)
-ISR (0xf7, isrIgnore7)
+ISR (0xf0, 0, isrIgnore0)
+ISR (0xf1, 0, isrIgnore1)
+ISR (0xf2, 0, isrIgnore2)
+ISR (0xf3, 0, isrIgnore3)
+ISR (0xf4, 0, isrIgnore4)
+ISR (0xf5, 0, isrIgnore5)
+ISR (0xf6, 0, isrIgnore6)
+ISR (0xf7, 0, isrIgnore7)
 
-ISR (0xf8, isrIgnore8)
-ISR (0xf9, isrIgnore9)
-ISR (0xfa, isrIgnoreA)
-ISR (0xfb, isrIgnoreB)
-ISR (0xfc, isrIgnoreC)
-ISR (0xfd, isrIgnoreD)
-ISR (0xfe, isrIgnoreE)
-ISR (0xff, isrIgnoreF)
+ISR (0xf8, 0, isrIgnore8)
+ISR (0xf9, 0, isrIgnore9)
+ISR (0xfa, 0, isrIgnoreA)
+ISR (0xfb, 0, isrIgnoreB)
+ISR (0xfc, 0, isrIgnoreC)
+ISR (0xfd, 0, isrIgnoreD)
+ISR (0xfe, 0, isrIgnoreE)
+ISR (0xff, 0, isrIgnoreF)

src/kernel/interrupts.cpp

@@ -8,6 +8,7 @@
 #include "debug.h"
 #include "device_manager.h"
 #include "gdt.h"
+#include "idt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "kernel_memory.h"
@@ -15,6 +16,7 @@
 #include "objects/process.h"
 #include "scheduler.h"
 #include "syscall.h"
+#include "tss.h"
 #include "vm_space.h"
 
 static const uint16_t PIC1 = 0x20;
@@ -22,21 +24,14 @@ static const uint16_t PIC2 = 0xa0;
 
 constexpr uintptr_t apic_eoi_addr = 0xfee000b0 + ::memory::page_offset;
 
+constexpr size_t increment_offset = 0x1000;
+
 extern "C" {
 	void _halt();
 
 	void isr_handler(cpu_state*);
 	void irq_handler(cpu_state*);
 
-#define ISR(i, name)     extern void name ();
-#define EISR(i, name)    extern void name ();
-#define UISR(i, name)    extern void name ();
-#define IRQ(i, q, name)  extern void name ();
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef UISR
-#undef EISR
-#undef ISR
 }
 
 isr
@@ -50,13 +45,11 @@ uint8_t
 get_irq(unsigned vector)
 {
 	switch (vector) {
-#define ISR(i, name)
-#define EISR(i, name)
-#define UISR(i, name)
+#define ISR(i, s, name)
+#define EISR(i, s, name)
 #define IRQ(i, q, name) case i : return q;
 #include "interrupt_isrs.inc"
 #undef IRQ
-#undef UISR
 #undef EISR
 #undef ISR
 
@@ -64,7 +57,7 @@ get_irq(unsigned vector)
 	}
 }
 
-static void
+void
 disable_legacy_pic()
 {
 	// Mask all interrupts
@@ -84,30 +77,19 @@ disable_legacy_pic()
 	outb(PIC2+1, 0x02); io_wait();
 }
 
-void
-interrupts_init()
-{
-#define ISR(i, name)     idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
-#define EISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
-#define UISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0xee);
-#define IRQ(i, q, name)  idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef UISR
-#undef EISR
-#undef ISR
-
-	disable_legacy_pic();
-
-	log::info(logs::boot, "Interrupts enabled.");
-}
-
 void
 isr_handler(cpu_state *regs)
 {
 	console *cons = console::get();
+	uint8_t vector = regs->interrupt & 0xff;
 
-	switch (static_cast<isr>(regs->interrupt & 0xff)) {
+	// Clear out the IST for this vector so we just keep using
+	// this stack
+	uint8_t old_ist = IDT::get().get_ist(vector);
+	if (old_ist)
+		IDT::get().set_ist(vector, 0);
+
+	switch (static_cast<isr>(vector)) {
 
 	case isr::isrDebug: {
 			cons->set_color(11);
@@ -141,6 +123,16 @@ isr_handler(cpu_state *regs)
 		}
 		break;
 
+	case isr::isrDoubleFault:
+		cons->set_color(9);
+		cons->printf("\nDouble Fault:\n");
+
+		cons->set_color();
+		print_regs(*regs);
+		print_stacktrace(2);
+		_halt();
+		break;
+
 	case isr::isrGPFault: {
 			cons->set_color(9);
 			cons->puts("\nGeneral Protection Fault:\n");
@@ -154,13 +146,13 @@ isr_handler(cpu_state *regs)
 				switch ((regs->errorcode & 0x07) >> 1) {
 				case 0:
 					cons->printf(" GDT[%x]\n", index);
-					gdt_dump(index);
+					GDT::current().dump(index);
 					break;
 
 				case 1:
 				case 3:
 					cons->printf(" IDT[%x]\n", index);
-					idt_dump(index);
+					IDT::get().dump(index);
 					break;
 
 				default:
@@ -279,6 +271,10 @@ isr_handler(cpu_state *regs)
 		print_stacktrace(2);
 		_halt();
 	}
+
+	// Return the IST for this vector to what it was
+	if (old_ist)
+		IDT::get().set_ist(vector, old_ist);
 	*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;
 }
 
@@ -292,8 +288,8 @@ irq_handler(cpu_state *regs)
 		cons->printf("\nReceived unknown IRQ: %d (vec %d)\n",
 				irq, regs->interrupt);
 		cons->set_color();
+
 		print_regs(*regs);
-		_halt();
 	}
 
 	*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;

src/kernel/interrupts.h

@@ -7,13 +7,11 @@
 /// Enum of all defined ISR/IRQ vectors
 enum class isr : uint8_t
 {
-#define ISR(i, name)     name = i,
-#define EISR(i, name)    name = i,
-#define UISR(i, name)    name = i,
+#define ISR(i, s, name)     name = i,
+#define EISR(i, s, name)    name = i,
 #define IRQ(i, q, name)  name = i,
 #include "interrupt_isrs.inc"
 #undef IRQ
-#undef UISR
 #undef EISR
 #undef ISR
 
@@ -31,6 +29,5 @@ extern "C" {
 	void interrupts_disable();
 }
 
-/// Fill the IDT with our ISRs, and disable the legacy
-/// PIC interrupts.
-void interrupts_init();
+/// Disable the legacy PIC
+void disable_legacy_pic();

src/kernel/interrupts.s

@@ -1,8 +1,14 @@
 %include "push_all.inc"
 
+section .text
+
 extern isr_handler
-global isr_handler_prelude
+global isr_handler_prelude:function (isr_handler_prelude.end - isr_handler_prelude)
 isr_handler_prelude:
+	push rbp     ; Never executed, fake function prelude
+	mov rbp, rsp ; to calm down gdb
+
+.real:
 	push_all
 	check_swap_gs
 
@@ -10,10 +16,15 @@ isr_handler_prelude:
 	mov rsi, rsp
 	call isr_handler
 	jmp isr_handler_return
+.end:
 
 extern irq_handler
-global irq_handler_prelude
+global irq_handler_prelude:function (irq_handler_prelude.end - irq_handler_prelude)
 irq_handler_prelude:
+	push rbp     ; Never executed, fake function prelude
+	mov rbp, rsp ; to calm down gdb
+
+.real:
 	push_all
 	check_swap_gs
 
@@ -21,41 +32,45 @@ irq_handler_prelude:
 	mov rsi, rsp
 	call irq_handler
 	; fall through to isr_handler_return
+.end:
 
-global isr_handler_return
+global isr_handler_return:function (isr_handler_return.end - isr_handler_return)
 isr_handler_return:
 	check_swap_gs
 	pop_all
 
 	add rsp, 16		; because the ISRs added err/num
 	iretq
+.end:
 
 %macro EMIT_ISR 2
-	global %1
+	global %1:function (%1.end - %1)
 	%1:
 		push 0
 		push %2
-		jmp isr_handler_prelude
+		jmp isr_handler_prelude.real
+	.end:
 %endmacro
 
 %macro EMIT_EISR 2
-	global %1
+	global %1:function (%1.end - %1)
 	%1:
 		push %2
-		jmp isr_handler_prelude
+		jmp isr_handler_prelude.real
+	.end:
 %endmacro
 
 %macro EMIT_IRQ 2
-	global %1
+	global %1:function (%1.end - %1)
 	%1:
 		push 0
 		push %2
-		jmp irq_handler_prelude
+		jmp irq_handler_prelude.real
+	.end:
 %endmacro
 
-%define EISR(i, name)     EMIT_EISR name, i   ; ISR with error code
-%define UISR(i, name)     EMIT_ISR name, i    ; ISR callable from user space
-%define  ISR(i, name)     EMIT_ISR name, i
+%define EISR(i, s, name)     EMIT_EISR name, i   ; ISR with error code
+%define  ISR(i, s, name)     EMIT_ISR name, i
 %define  IRQ(i, q, name)  EMIT_IRQ name, i
 
 section .isrs

src/kernel/loader.s

@@ -1,24 +0,0 @@
-%include "push_all.inc"
-
-extern load_process_image
-
-global preloaded_process_init
-preloaded_process_init:
-
-	; create_process already pushed the arguments for load_process_image and
-	; the following iretq onto the stack for us
-
-	pop rdi ; the physical address of the program image
-	pop rsi ; the virtual address of the program image
-	pop rdx ; the size in bytes of the program image
-	pop rcx ; the address of this thread's TCB
-
-	call load_process_image
-
-	; user rsp is now in rax, put it in the right place for iret
-	mov [rsp + 0x18], rax
-
-	; the entrypoint should already be on the stack
-	swapgs
-	iretq
-

src/kernel/log.cpp

@@ -1,8 +1,10 @@
+#include "j6/signals.h"
 #include "kutil/memory.h"
 #include "kutil/no_construct.h"
 #include "console.h"
 #include "log.h"
-#include "scheduler.h"
+#include "objects/system.h"
+#include "objects/thread.h"
 
 static uint8_t log_buffer[0x10000];
 
@@ -10,7 +12,7 @@ static uint8_t log_buffer[0x10000];
 // so that we can start log output immediately. Keep its constructor
 // from being called here so as to not overwrite the previous initialization.
 static kutil::no_construct<log::logger> __g_logger_storage;
-static log::logger &g_logger = __g_logger_storage.value;
+log::logger &g_logger = __g_logger_storage.value;
 
 static const uint8_t level_colors[] = {0x07, 0x07, 0x0f, 0x0b, 0x09};
 
@@ -26,29 +28,54 @@ output_log(log::area_t area, log::level severity, const char *message)
 	cons->set_color();
 }
 
+static void
+log_flush()
+{
+	system &sys = system::get();
+	sys.assert_signal(j6_signal_system_has_log);
+}
+
 void
 logger_task()
 {
-	uint8_t buffer[257];
-	auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
 	auto *cons = console::get();
 
 	log::info(logs::task, "Starting kernel logger task");
 	g_logger.set_immediate(nullptr);
+	g_logger.set_flush(log_flush);
 
-	scheduler &s = scheduler::get();
+	thread &self = thread::current();
+	system &sys = system::get();
+
+	size_t buffer_size = 1;
+	uint8_t *buffer = nullptr;
 
 	while (true) {
-		if(g_logger.get_entry(buffer, sizeof(buffer))) {
+		size_t size = g_logger.get_entry(buffer, buffer_size);
+		if (size > buffer_size) {
+			while (size > buffer_size) buffer_size *= 2;
+			kutil::kfree(buffer);
+			buffer = reinterpret_cast<uint8_t*>(kutil::kalloc(buffer_size));
+			kassert(buffer, "Could not allocate logger task buffer");
+			continue;
+		}
+
+		if(size) {
+			auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
 			buffer[ent->bytes] = 0;
+
 			cons->set_color(level_colors[static_cast<int>(ent->severity)]);
 			cons->printf("%7s %5s: %s\n",
 				g_logger.area_name(ent->area),
 				g_logger.level_name(ent->severity),
 				ent->message);
 			cons->set_color();
-		} else {
-			s.schedule();
+		}
+
+		if (!g_logger.has_log()) {
+			sys.deassert_signal(j6_signal_system_has_log);
+			sys.add_blocked_thread(&self);
+			self.wait_on_signals(&sys, j6_signal_system_has_log);
 		}
 	}
 }
@@ -57,3 +84,8 @@ void logger_init()
 {
 	new (&g_logger) log::logger(log_buffer, sizeof(log_buffer), output_log);
 }
+
+void logger_clear_immediate()
+{
+	g_logger.set_immediate(nullptr);
+}

src/kernel/log.h

@@ -6,4 +6,5 @@ namespace log = kutil::log;
 namespace logs = kutil::logs;
 
 void logger_init();
+void logger_clear_immediate();
 void logger_task();

src/kernel/main.cpp

@@ -6,21 +6,28 @@
 #include "kutil/assert.h"
 #include "apic.h"
 #include "block_device.h"
+#include "clock.h"
 #include "console.h"
 #include "cpu.h"
 #include "device_manager.h"
 #include "gdt.h"
+#include "idt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "kernel_args.h"
 #include "kernel_memory.h"
 #include "log.h"
+#include "msr.h"
 #include "objects/channel.h"
 #include "objects/event.h"
+#include "objects/thread.h"
+#include "objects/vm_area.h"
 #include "scheduler.h"
 #include "serial.h"
 #include "symbol_table.h"
 #include "syscall.h"
+#include "tss.h"
+#include "vm_space.h"
 
 #ifndef GIT_VERSION
 #define GIT_VERSION
@@ -30,6 +37,39 @@ extern "C" {
 	void kernel_main(kernel::args::header *header);
 	void (*__ctors)(void);
 	void (*__ctors_end)(void);
+	void long_ap_startup(cpu_data *cpu);
+	void ap_startup();
+	void ap_idle();
+	void init_ap_trampoline(void*, cpu_data *, void (*)());
+}
+
+extern void __kernel_assert(const char *, unsigned, const char *);
+
+using namespace kernel;
+
+volatile size_t ap_startup_count;
+static bool scheduler_ready = false;
+
+/// Bootstrap the memory managers.
+void memory_initialize_pre_ctors(args::header &kargs);
+void memory_initialize_post_ctors(args::header &kargs);
+process * load_simple_process(args::program &program);
+
+unsigned start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4);
+
+/// TODO: not this. this is awful.
+args::framebuffer *fb = nullptr;
+
+void
+init_console()
+{
+	serial_port *com1 = new (&g_com1) serial_port(COM1);
+	console *cons = new (&g_console) console(com1);
+
+	cons->set_color(0x21, 0x00);
+	cons->puts("jsix OS ");
+	cons->set_color(0x08, 0x00);
+	cons->puts(GIT_VERSION " booting...\n");
 }
 
 void
@@ -42,85 +82,72 @@ run_constructors()
 	}
 }
 
-extern void __kernel_assert(const char *, unsigned, const char *);
-
-/// Bootstrap the memory managers.
-void memory_initialize_pre_ctors(kernel::args::header *kargs);
-void memory_initialize_post_ctors(kernel::args::header *kargs);
-
-using namespace kernel;
-
-void
-init_console()
-{
-	serial_port *com1 = new (&g_com1) serial_port(COM1);
-	console *cons = new (&g_console) console(com1);
-
-	cons->set_color(0x21, 0x00);
-	cons->puts("jsix OS ");
-	cons->set_color(0x08, 0x00);
-	cons->puts(GIT_VERSION " booting...\n");
-	logger_init();
-}
-
-channel *std_out = nullptr;
-
-void
-stdout_task()
-{
-	uint8_t buffer[257];
-	auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
-	auto *cons = console::get();
-
-	log::info(logs::task, "Starting kernel stdout task");
-
-	scheduler &s = scheduler::get();
-	thread *th = thread::from_tcb(s.current());
-
-	while (true) {
-		j6_signal_t current = std_out->signals();
-		if (!(current & j6_signal_channel_can_recv)) {
-			th->wait_on_signals(std_out, j6_signal_channel_can_recv);
-			s.schedule();
-		}
-
-		size_t n = 256;
-		j6_status_t status = std_out->dequeue(&n, buffer);
-		if (status != j6_status_ok) {
-			log::warn(logs::task, "Kernel stdout error: %x", status);
-			return;
-		}
-
-		buffer[n] = 0;
-		const char *s = reinterpret_cast<const char *>(buffer);
-
-		while (n) {
-			size_t r = cons->puts(s);
-			n -= r + 1;
-			s += r + 1;
-		}
-	}
-}
-
-
 void
 kernel_main(args::header *header)
 {
 	kutil::assert_set_callback(__kernel_assert);
+
 	init_console();
+	logger_init();
 
-	gdt_init();
-	interrupts_init();
+	cpu_validate();
 
-	memory_initialize_pre_ctors(header);
+	log::debug(logs::boot, "    jsix header is at: %016lx", header);
+	log::debug(logs::boot, "     Memory map is at: %016lx", header->mem_map);
+	log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
+	log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
+	log::debug(logs::boot, "    Kernel PML4 is at: %016lx", header->pml4);
+
+	uint64_t cr0, cr4;
+	asm ("mov %%cr0, %0" : "=r"(cr0));
+	asm ("mov %%cr4, %0" : "=r"(cr4));
+	uint64_t efer = rdmsr(msr::ia32_efer);
+	log::debug(logs::boot, "Control regs: cr0:%lx cr4:%lx efer:%lx", cr0, cr4, efer);
+
+	bool has_video = false;
+	if (header->video.size > 0) {
+		has_video = true;
+		fb = &header->video;
+
+		const args::framebuffer &video = header->video;
+		log::debug(logs::boot, "Framebuffer: %dx%d[%d] type %d @ %llx size %llx",
+			video.horizontal,
+			video.vertical,
+			video.scanline,
+			video.type,
+			video.phys_addr,
+			video.size);
+		logger_clear_immediate();
+	}
+
+	extern IDT &g_idt;
+	extern TSS &g_bsp_tss;
+	extern GDT &g_bsp_gdt;
+	extern cpu_data g_bsp_cpu_data;
+	extern uintptr_t idle_stack_end;
+
+	IDT *idt = new (&g_idt) IDT;
+
+	cpu_data *cpu = &g_bsp_cpu_data;
+	kutil::memset(cpu, 0, sizeof(cpu_data));
+
+	cpu->self = cpu;
+	cpu->tss = new (&g_bsp_tss) TSS;
+	cpu->gdt = new (&g_bsp_gdt) GDT {cpu->tss};
+	cpu->rsp0 = idle_stack_end;
+	cpu_early_init(cpu);
+
+	disable_legacy_pic();
+
+	memory_initialize_pre_ctors(*header);
 	run_constructors();
-	memory_initialize_post_ctors(header);
+	memory_initialize_post_ctors(*header);
 
-	cpu_id cpu;
-	cpu.validate();
+	cpu->tss->create_ist_stacks(idt->used_ist_entries());
 
 	for (size_t i = 0; i < header->num_modules; ++i) {
 		args::module &mod = header->modules[i];
+
 		switch (mod.type) {
 		case args::mod_type::symbol_table:
 			new symbol_table {mod.location, mod.size};
@@ -131,18 +158,30 @@ kernel_main(args::header *header)
 		}
 	}
 
-	log::debug(logs::boot, "    jsix header is at: %016lx", header);
-	log::debug(logs::boot, "     Memory map is at: %016lx", header->mem_map);
-	log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
-	log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
+	syscall_initialize();
 
 	device_manager &devices = device_manager::get();
 	devices.parse_acpi(header->acpi_table);
 
-	interrupts_enable();
-	devices.init_drivers();
+	// Need the local APIC to get the BSP's id
+	uintptr_t apic_base = devices.get_lapic_base();
 
-	devices.get_lapic()->calibrate_timer();
+	lapic *apic = new lapic(apic_base);
+	apic->enable();
+
+	cpu->id = apic->get_id();
+	cpu->apic = apic;
+
+	cpu_init(cpu, true);
+
+	devices.init_drivers();
+	apic->calibrate_timer();
+
+	const auto &apic_ids = devices.get_apic_ids();
+	unsigned num_cpus = start_aps(*apic, apic_ids, header->pml4);
+
+	idt->add_ist_entries();
+	interrupts_enable();
 
 	/*
 	block_device *disk = devices->get_block_device(0);
@@ -168,23 +207,138 @@ kernel_main(args::header *header)
 	}
 	*/
 
-	syscall_enable();
-	scheduler *sched = new scheduler(devices.get_lapic());
-
-	std_out = new channel;
+	scheduler *sched = new scheduler {num_cpus};
+	scheduler_ready = true;
 
 	// Skip program 0, which is the kernel itself
-	for (size_t i = 1; i < header->num_programs; ++i) {
-		args::program &prog = header->programs[i];
-		sched->load_process(prog.phys_addr, prog.virt_addr, prog.size, prog.entrypoint); 
-	}
+	for (unsigned i = 1; i < header->num_programs; ++i)
+		load_simple_process(header->programs[i]);
 
-	sched->create_kernel_task(logger_task, scheduler::max_priority-1, true);
-	sched->create_kernel_task(stdout_task, scheduler::max_priority-1, true);
-
-	const char stdout_message[] = "Hello on the fake stdout channel\n";
-	size_t message_size = sizeof(stdout_message);
-	std_out->enqueue(&message_size, reinterpret_cast<const void*>(stdout_message));
+	if (!has_video)
+		sched->create_kernel_task(logger_task, scheduler::max_priority/2, true);
 
 	sched->start();
 }
+
+unsigned
+start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4)
+{
+	using memory::frame_size;
+	using memory::kernel_stack_pages;
+
+	extern size_t ap_startup_code_size;
+	extern process &g_kernel_process;
+	extern vm_area_guarded &g_kernel_stacks;
+
+	clock &clk = clock::get();
+
+	ap_startup_count = 1; // BSP processor
+	log::info(logs::boot, "Starting %d other CPUs", ids.count() - 1);
+
+	// Since we're using address space outside kernel space, make sure
+	// the kernel's vm_space is used
+	cpu_data &bsp = current_cpu();
+	bsp.process = &g_kernel_process;
+
+	uint16_t index = bsp.index;
+
+	// Copy the startup code somwhere the real mode trampoline can run
+	uintptr_t addr = 0x8000; // TODO: find a valid address, rewrite addresses
+	uint8_t vector = addr >> 12;
+	vm_area *vma = new vm_area_fixed(addr, 0x1000, vm_flags::write);
+	vm_space::kernel_space().add(addr, vma);
+	kutil::memcpy(
+		reinterpret_cast<void*>(addr),
+		reinterpret_cast<void*>(&ap_startup),
+		ap_startup_code_size);
+
+	// AP idle stacks need less room than normal stacks, so pack multiple
+	// into a normal stack area
+	static constexpr size_t idle_stack_bytes = 2048; // 2KiB is generous
+	static constexpr size_t full_stack_bytes = kernel_stack_pages * frame_size;
+	static constexpr size_t idle_stacks_per = full_stack_bytes / idle_stack_bytes;
+
+	uint8_t ist_entries = IDT::get().used_ist_entries();
+
+	size_t free_stack_count = 0;
+	uintptr_t stack_area_start = 0;
+
+	ipi mode = ipi::init | ipi::level | ipi::assert;
+	apic.send_ipi_broadcast(mode, false, 0);
+
+	for (uint8_t id : ids) {
+		if (id == bsp.id) continue;
+
+		// Set up the CPU data structures
+		TSS *tss = new TSS;
+		GDT *gdt = new GDT {tss};
+		cpu_data *cpu = new cpu_data;
+		kutil::memset(cpu, 0, sizeof(cpu_data));
+
+		cpu->self = cpu;
+		cpu->id = id;
+		cpu->index = ++index;
+		cpu->gdt = gdt;
+		cpu->tss = tss;
+
+		tss->create_ist_stacks(ist_entries);
+
+		// Set up the CPU's idle task stack
+		if (free_stack_count == 0) {
+			stack_area_start = g_kernel_stacks.get_section();
+			free_stack_count = idle_stacks_per;
+		}
+
+		uintptr_t stack_end = stack_area_start + free_stack_count-- * idle_stack_bytes;
+		stack_end -= 2 * sizeof(void*); // Null frame
+		*reinterpret_cast<uint64_t*>(stack_end) = 0; // pre-fault the page
+		cpu->rsp0 = stack_end;
+
+		// Set up the trampoline with this CPU's data
+		init_ap_trampoline(kpml4, cpu, ap_idle);
+
+		// Kick it off!
+		size_t current_count = ap_startup_count;
+		log::debug(logs::boot, "Starting AP %d: stack %llx", cpu->index, stack_end);
+
+		ipi startup = ipi::startup | ipi::assert;
+
+		apic.send_ipi(startup, vector, id);
+		for (unsigned i = 0; i < 20; ++i) {
+			if (ap_startup_count > current_count) break;
+			clk.spinwait(20);
+		}
+
+		// If the CPU already incremented ap_startup_count, it's done
+		if (ap_startup_count > current_count)
+			continue;
+
+		// Send the second SIPI (intel recommends this)
+		apic.send_ipi(startup, vector, id);
+		for (unsigned i = 0; i < 100; ++i) {
+			if (ap_startup_count > current_count) break;
+			clk.spinwait(100);
+		}
+
+		log::warn(logs::boot, "No response from AP %d within timeout", id);
+	}
+
+	log::info(logs::boot, "%d CPUs running", ap_startup_count);
+	vm_space::kernel_space().remove(vma);
+	return ap_startup_count;
+}
+
+void
+long_ap_startup(cpu_data *cpu)
+{
+	cpu_init(cpu, false);
+	++ap_startup_count;
+	while (!scheduler_ready) asm ("pause");
+
+	uintptr_t apic_base =
+		device_manager::get().get_lapic_base();
+	cpu->apic = new lapic(apic_base);
+	cpu->apic->enable();
+
+	scheduler::get().start();
+}

src/kernel/memory_bootstrap.cpp

@@ -1,30 +1,31 @@
 #include <utility>
 
 #include "kernel_args.h"
+#include "j6/init.h"
 
 #include "kutil/assert.h"
 #include "kutil/heap_allocator.h"
 #include "kutil/no_construct.h"
 
+#include "device_manager.h"
 #include "frame_allocator.h"
+#include "gdt.h"
 #include "io.h"
 #include "log.h"
+#include "msr.h"
 #include "objects/process.h"
+#include "objects/thread.h"
+#include "objects/system.h"
 #include "objects/vm_area.h"
 #include "vm_space.h"
 
-using memory::frame_size;
 using memory::heap_start;
 using memory::kernel_max_heap;
-using memory::kernel_offset;
-using memory::heap_start;
-using memory::page_offset;
-using memory::pml4e_kernel;
-using memory::pml4e_offset;
-using memory::table_entries;
 
 using namespace kernel;
 
+extern "C" void initialize_main_thread();
+extern "C" uintptr_t initialize_main_user_stack();
 
 // These objects are initialized _before_ global constructors are called,
 // so we don't want them to have global constructors at all, lest they
@@ -35,20 +36,17 @@ kutil::heap_allocator &g_kernel_heap = __g_kernel_heap_storage.value;
 static kutil::no_construct<frame_allocator> __g_frame_allocator_storage;
 frame_allocator &g_frame_allocator = __g_frame_allocator_storage.value;
 
-static kutil::no_construct<vm_area_open> __g_kernel_heap_area_storage;
-vm_area_open &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
+static kutil::no_construct<vm_area_untracked> __g_kernel_heap_area_storage;
+vm_area_untracked &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
 
-vm_area_buffers g_kernel_stacks {
-	memory::kernel_max_stacks,
-	vm_space::kernel_space(),
-	vm_flags::write,
-	memory::kernel_stack_pages};
+static kutil::no_construct<vm_area_guarded> __g_kernel_stacks_storage;
+vm_area_guarded &g_kernel_stacks = __g_kernel_stacks_storage.value;
 
-vm_area_buffers g_kernel_buffers {
+vm_area_guarded g_kernel_buffers {
+	memory::buffers_start,
+	memory::kernel_buffer_pages,
 	memory::kernel_max_buffers,
-	vm_space::kernel_space(),
-	vm_flags::write,
-	memory::kernel_buffer_pages};
+	vm_flags::write};
 
 void * operator new(size_t size)           { return g_kernel_heap.allocate(size); }
 void * operator new [] (size_t size)       { return g_kernel_heap.allocate(size); }
@@ -56,103 +54,242 @@ void operator delete (void *p) noexcept    { return g_kernel_heap.free(p); }
 void operator delete [] (void *p) noexcept { return g_kernel_heap.free(p); }
 
 namespace kutil {
-void * kalloc(size_t size) { return g_kernel_heap.allocate(size); }
-void kfree(void *p) { return g_kernel_heap.free(p); }
+	void * kalloc(size_t size) { return g_kernel_heap.allocate(size); }
+	void kfree(void *p) { return g_kernel_heap.free(p); }
 }
 
-/*
-void walk_page_table(
-	page_table *table,
-	page_table::level level,
-	uintptr_t &current_start,
-	size_t &current_bytes,
-	vm_area &karea)
-{
-	constexpr size_t huge_page_size = (1ull<<30);
-	constexpr size_t large_page_size = (1ull<<21);
-
-	for (unsigned i = 0; i < table_entries; ++i) {
-		page_table *next = table->get(i);
-		if (!next) {
-			if (current_bytes)
-				karea.commit(current_start, current_bytes);
-			current_start = 0;
-			current_bytes = 0;
-			continue;
-		} else if (table->is_page(level, i)) {
-			if (!current_bytes)
-				current_start = reinterpret_cast<uintptr_t>(next);
-			current_bytes +=
-				(level == page_table::level::pt
-					? frame_size
-					: level == page_table::level::pd
-						? large_page_size
-						: huge_page_size);
-		} else {
-			page_table::level deeper =
-				static_cast<page_table::level>(
-					static_cast<unsigned>(level) + 1);
-
-			walk_page_table(
-				next, deeper, current_start, current_bytes, kspace);
-		}
-	}
+template <typename T>
+uintptr_t
+get_physical_page(T *p) {
+	return memory::page_align_down(reinterpret_cast<uintptr_t>(p));
 }
-*/
 
 void
-memory_initialize_pre_ctors(args::header *kargs)
+memory_initialize_pre_ctors(args::header &kargs)
 {
-	new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
-	new (&g_frame_allocator) frame_allocator;
+	using kernel::args::frame_block;
 
-	args::mem_entry *entries = kargs->mem_map;
-	const size_t count = kargs->map_count;
-	for (unsigned i = 0; i < count; ++i) {
-		// TODO: use entry attributes
-		args::mem_entry &e = entries[i];
-		if (e.type == args::mem_type::free)
-			g_frame_allocator.free(e.start, e.pages);
+	page_table *kpml4 = static_cast<page_table*>(kargs.pml4);
+
+	new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
+
+	frame_block *blocks = reinterpret_cast<frame_block*>(memory::bitmap_start);
+	new (&g_frame_allocator) frame_allocator {blocks, kargs.frame_block_count};
+
+	// Mark all the things the bootloader allocated for us as used
+	g_frame_allocator.used(
+		get_physical_page(&kargs),
+		memory::page_count(sizeof(kargs)));
+
+	g_frame_allocator.used(
+		get_physical_page(kargs.frame_blocks),
+		kargs.frame_block_pages);
+
+	g_frame_allocator.used(
+		get_physical_page(kargs.pml4),
+		kargs.table_pages);
+
+	for (unsigned i = 0; i < kargs.num_modules; ++i) {
+		const kernel::args::module &mod = kargs.modules[i];
+		g_frame_allocator.used(
+			get_physical_page(mod.location),
+			memory::page_count(mod.size));
+	}
+
+	for (unsigned i = 0; i < kargs.num_programs; ++i) {
+		const kernel::args::program &prog = kargs.programs[i];
+		for (auto &sect : prog.sections) {
+			if (!sect.size) continue;
+			g_frame_allocator.used(
+				sect.phys_addr,
+				memory::page_count(sect.size));
+		}
 	}
 
-	page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
 	process *kp = process::create_kernel_process(kpml4);
 	vm_space &vm = kp->space();
 
 	vm_area *heap = new (&g_kernel_heap_area)
-		vm_area_open(memory::kernel_max_heap, vm, vm_flags::write);
+		vm_area_untracked(kernel_max_heap, vm_flags::write);
 
-	vm.add(memory::heap_start, heap);
+	vm.add(heap_start, heap);
+
+	vm_area *stacks = new (&g_kernel_stacks) vm_area_guarded {
+		memory::stacks_start,
+		memory::kernel_stack_pages,
+		memory::kernel_max_stacks,
+		vm_flags::write};
+	vm.add(memory::stacks_start, &g_kernel_stacks);
+
+	// Clean out any remaning bootloader page table entries
+	for (unsigned i = 0; i < memory::pml4e_kernel; ++i)
+		kpml4->entries[i] = 0;
 }
 
 void
-memory_initialize_post_ctors(args::header *kargs)
+memory_initialize_post_ctors(args::header &kargs)
 {
-	/*
-	uintptr_t current_start = 0;
-	size_t current_bytes = 0;
-
-	// TODO: Should we exclude the top of this area? (eg, buffers, stacks, etc)
-	page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
-	for (unsigned i = pml4e_kernel; i < pml4e_offset; ++i) {
-		page_table *pdp = kpml4->get(i);
-		kassert(pdp, "Bootloader did not create all kernelspace PDs");
-
-		walk_page_table(
-			pdp, page_table::level::pdp,
-			current_start, current_bytes,
-			g_kernel_space);
-	}
-
-	if (current_bytes)
-		g_kernel_space.commit(current_start, current_bytes);
-	*/
 	vm_space &vm = vm_space::kernel_space();
-	vm.add(memory::stacks_start, &g_kernel_stacks);
 	vm.add(memory::buffers_start, &g_kernel_buffers);
 
 	g_frame_allocator.free(
-		reinterpret_cast<uintptr_t>(kargs->page_tables),
-		kargs->table_count);
-
+		get_physical_page(kargs.page_tables),
+		kargs.table_count);
+}
+
+static void
+log_mtrrs()
+{
+	uint64_t mtrrcap = rdmsr(msr::ia32_mtrrcap);
+	uint64_t mtrrdeftype = rdmsr(msr::ia32_mtrrdeftype);
+	unsigned vcap = mtrrcap & 0xff;
+	log::debug(logs::boot, "MTRRs: vcap=%d %s %s def=%02x %s %s",
+		vcap,
+		(mtrrcap & (1<< 8)) ? "fix" : "",
+		(mtrrcap & (1<<10)) ? "wc" : "",
+		mtrrdeftype & 0xff,
+		(mtrrdeftype & (1<<10)) ? "fe" : "",
+		(mtrrdeftype & (1<<11)) ? "enabled" : ""
+		);
+
+	for (unsigned i = 0; i < vcap; ++i) {
+		uint64_t base = rdmsr(find_mtrr(msr::ia32_mtrrphysbase, i));
+		uint64_t mask = rdmsr(find_mtrr(msr::ia32_mtrrphysmask, i));
+		log::debug(logs::boot, "       vcap[%2d] base:%016llx mask:%016llx type:%02x %s", i,
+			(base & ~0xfffull),
+			(mask & ~0xfffull),
+			(base & 0xff),
+			(mask & (1<<11)) ? "valid" : "");
+ 	}
+
+	msr mtrr_fixed[] = {
+		msr::ia32_mtrrfix64k_00000,
+		msr::ia32_mtrrfix16k_80000,
+		msr::ia32_mtrrfix16k_a0000,
+		msr::ia32_mtrrfix4k_c0000,
+		msr::ia32_mtrrfix4k_c8000,
+		msr::ia32_mtrrfix4k_d0000,
+		msr::ia32_mtrrfix4k_d8000,
+		msr::ia32_mtrrfix4k_e0000,
+		msr::ia32_mtrrfix4k_e8000,
+		msr::ia32_mtrrfix4k_f0000,
+		msr::ia32_mtrrfix4k_f8000,
+	};
+
+	for (int i = 0; i < 11; ++i) {
+		uint64_t v = rdmsr(mtrr_fixed[i]);
+		log::debug(logs::boot, "      fixed[%2d] %02x %02x %02x %02x %02x %02x %02x %02x", i,
+			((v <<  0) & 0xff), ((v <<  8) & 0xff), ((v << 16) & 0xff), ((v << 24) & 0xff),
+			((v << 32) & 0xff), ((v << 40) & 0xff), ((v << 48) & 0xff), ((v << 56) & 0xff));
+	}
+
+	uint64_t pat = rdmsr(msr::ia32_pat);
+	static const char *pat_names[] = {"UC ","WC ","XX ","XX ","WT ","WP ","WB ","UC-"};
+	log::debug(logs::boot, "      PAT: 0:%s 1:%s 2:%s 3:%s 4:%s 5:%s 6:%s 7:%s",
+		pat_names[(pat >> (0*8)) & 7], pat_names[(pat >> (1*8)) & 7],
+		pat_names[(pat >> (2*8)) & 7], pat_names[(pat >> (3*8)) & 7],
+		pat_names[(pat >> (4*8)) & 7], pat_names[(pat >> (5*8)) & 7],
+		pat_names[(pat >> (6*8)) & 7], pat_names[(pat >> (7*8)) & 7]);
+}
+
+
+process *
+load_simple_process(args::program &program)
+{
+	using kernel::args::section_flags;
+
+	process *p = new process;
+	vm_space &space = p->space();
+
+	for (const auto &sect : program.sections) {
+		vm_flags flags =
+			(bitfield_has(sect.type, section_flags::execute) ? vm_flags::exec : vm_flags::none) |
+			(bitfield_has(sect.type, section_flags::write) ? vm_flags::write : vm_flags::none);
+
+		vm_area *vma = new vm_area_fixed(sect.phys_addr, sect.size, flags);
+		space.add(sect.virt_addr, vma);
+	}
+
+	uint64_t iopl = (3ull << 12);
+	uintptr_t trampoline = reinterpret_cast<uintptr_t>(initialize_main_thread);
+
+	thread *main = p->create_thread();
+	main->add_thunk_user(program.entrypoint, trampoline, iopl);
+	main->set_state(thread::state::ready);
+
+	return p;
+}
+
+template <typename T>
+inline T * push(uintptr_t &rsp, size_t size = sizeof(T)) {
+	rsp -= size;
+	T *p = reinterpret_cast<T*>(rsp);
+	rsp &= ~(sizeof(uint64_t)-1); // Align the stack
+	return p;
+}
+
+uintptr_t
+initialize_main_user_stack()
+{
+	process &proc = process::current();
+	thread &th = thread::current();
+	TCB *tcb = th.tcb();
+
+	const char message[] = "Hello from the kernel!";
+	char *message_arg = push<char>(tcb->rsp3, sizeof(message));
+	kutil::memcpy(message_arg, message, sizeof(message));
+
+	j6_init_value *initv = nullptr;
+	unsigned n = 0;
+
+	extern args::framebuffer *fb;
+	if (fb) {
+		j6_init_framebuffer *fb_desc = push<j6_init_framebuffer>(tcb->rsp3);
+		kutil::memset(fb_desc, 0, sizeof(j6_init_framebuffer));
+
+		fb_desc->addr = fb->phys_addr;
+		fb_desc->size = fb->size;
+		fb_desc->vertical = fb->vertical;
+		fb_desc->horizontal = fb->horizontal;
+		fb_desc->scanline = fb->scanline;
+
+		if (fb->type == kernel::args::fb_type::bgr8)
+			fb_desc->flags |= 1;
+
+		initv = push<j6_init_value>(tcb->rsp3);
+		initv->type = j6_init_desc_framebuffer;
+		initv->data = fb_desc;
+		++n;
+	}
+
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_other;
+	initv->handle.type = j6_object_type_system;
+	initv->handle.handle = proc.add_handle(&system::get());
+	++n;
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_self;
+	initv->handle.type = j6_object_type_process;
+	initv->handle.handle = proc.self_handle();
+	++n;
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_self;
+	initv->handle.type = j6_object_type_thread;
+	initv->handle.handle = th.self_handle();
+	++n;
+
+	uint64_t *initc = push<uint64_t>(tcb->rsp3);
+	*initc = n;
+
+	char **argv0 = push<char*>(tcb->rsp3);
+	*argv0 = message_arg;
+
+	uint64_t *argc = push<uint64_t>(tcb->rsp3);
+	*argc = 1;
+
+	th.clear_state(thread::state::loading);
+	return tcb->rsp3;
 }

src/kernel/msr.cpp

@@ -1,5 +1,11 @@
 #include "msr.h"
 
+msr
+find_mtrr(msr type, unsigned index)
+{
+	return static_cast<msr>(static_cast<uint32_t>(type) + (2 * index));
+}
+
 uint64_t
 rdmsr(msr addr)
 {

src/kernel/msr.h

@@ -6,22 +6,46 @@
 
 enum class msr : uint32_t
 {
-	ia32_efer			= 0xc0000080,
-	ia32_star			= 0xc0000081,
-	ia32_lstar			= 0xc0000082,
-	ia32_fmask			= 0xc0000084,
+	ia32_mtrrcap           = 0x000000fe,
+	ia32_mtrrdeftype       = 0x000002ff,
 
-	ia32_gs_base		= 0xc0000101,
-	ia32_kernel_gs_base = 0xc0000102
+	ia32_mtrrphysbase      = 0x00000200,
+	ia32_mtrrphysmask      = 0x00000201,
+
+	ia32_mtrrfix64k_00000  = 0x00000250,
+
+	ia32_mtrrfix16k_80000  = 0x00000258,
+	ia32_mtrrfix16k_a0000  = 0x00000259,
+
+	ia32_mtrrfix4k_c0000   = 0x00000268,
+	ia32_mtrrfix4k_c8000   = 0x00000269,
+	ia32_mtrrfix4k_d0000   = 0x0000026A,
+	ia32_mtrrfix4k_d8000   = 0x0000026B,
+	ia32_mtrrfix4k_e0000   = 0x0000026C,
+	ia32_mtrrfix4k_e8000   = 0x0000026D,
+	ia32_mtrrfix4k_f0000   = 0x0000026E,
+	ia32_mtrrfix4k_f8000   = 0x0000026F,
+
+	ia32_pat               = 0x00000277,
+	ia32_efer              = 0xc0000080,
+	ia32_star              = 0xc0000081,
+	ia32_lstar             = 0xc0000082,
+	ia32_fmask             = 0xc0000084,
+
+	ia32_gs_base           = 0xc0000101,
+	ia32_kernel_gs_base    = 0xc0000102
 };
 
+/// Find the msr for MTRR physical base or mask
+msr find_mtrr(msr type, unsigned index);
+
 /// Read the value of a MSR
-/// \arg addr   The MSR address
-/// \returns    The current value of the MSR
+/// \arg addr  The MSR address
+/// \returns   The current value of the MSR
 uint64_t rdmsr(msr addr);
 
 /// Write to a MSR
-/// \arg addr   The MSR address
-/// \arg value  The value to write
+/// \arg addr  The MSR address
+/// \arg value The value to write
 void wrmsr(msr addr, uint64_t value);
 

src/kernel/objects/channel.cpp

@@ -4,13 +4,13 @@
 #include "objects/channel.h"
 #include "objects/vm_area.h"
 
-extern vm_area_buffers g_kernel_buffers;
+extern vm_area_guarded g_kernel_buffers;
 
 constexpr size_t buffer_bytes = memory::kernel_buffer_pages * memory::frame_size;
 
 channel::channel() :
 	m_len(0),
-	m_data(g_kernel_buffers.get_buffer()),
+	m_data(g_kernel_buffers.get_section()),
 	m_buffer(reinterpret_cast<uint8_t*>(m_data), buffer_bytes),
 	kobject(kobject::type::channel, j6_signal_channel_can_send)
 {
@@ -79,7 +79,7 @@ void
 channel::close()
 {
 	kobject::close();
-	g_kernel_buffers.return_buffer(m_data);
+	g_kernel_buffers.return_section(m_data);
 }
 
 void

src/kernel/objects/kobject.h

@@ -16,17 +16,9 @@ public:
 	/// Types of kernel objects.
 	enum class type : uint16_t
 	{
-		none,
-		system,
-
-		event,
-		channel,
-		endpoint,
-
-		vma,
-
-		process,
-		thread,
+#define OBJECT_TYPE( name, val ) name = val,
+#include "j6/tables/object_types.inc"
+#undef OBJECT_TYPE
 
 		max
 	};

src/kernel/objects/process.cpp

@@ -13,21 +13,20 @@ static kutil::no_construct<process> __g_kernel_process_storage;
 process &g_kernel_process = __g_kernel_process_storage.value;
 
 
-kutil::vector<process*> process::s_processes;
-
 process::process() :
 	kobject {kobject::type::process},
-	m_next_handle {0},
+	m_next_handle {1},
 	m_state {state::running}
 {
-	s_processes.append(this);
+	j6_handle_t self = add_handle(this);
+	kassert(self == self_handle(), "Process self-handle is not 1");
 }
 
 // The "kernel process"-only constructor
 process::process(page_table *kpml4) :
 	kobject {kobject::type::process},
 	m_space {kpml4},
-	m_next_handle {0},
+	m_next_handle {self_handle()+1},
 	m_state {state::running}
 {
 }
@@ -36,10 +35,9 @@ process::~process()
 {
 	for (auto &it : m_handles)
 		if (it.val) it.val->handle_release();
-	s_processes.remove_swap(this);
 }
 
-process & process::current() { return *bsp_cpu_data.p; }
+process & process::current() { return *current_cpu().process; }
 process & process::kernel_process() { return g_kernel_process; }
 
 process *
@@ -49,21 +47,18 @@ process::create_kernel_process(page_table *pml4)
 }
 
 void
-process::exit(unsigned code)
+process::exit(int32_t code)
 {
 	// TODO: make this thread-safe
-	if (m_state != state::running)
-		return;
-	else
-		m_state = state::exited;
+	m_state = state::exited;
+	m_return_code = code;
+	close();
 
 	for (auto *thread : m_threads) {
 		thread->exit(code);
 	}
-	m_return_code = code;
-	close();
 
-	if (this == bsp_cpu_data.p)
+	if (this == current_cpu().process)
 		scheduler::get().schedule();
 }
 
@@ -93,7 +88,7 @@ process::update()
 thread *
 process::create_thread(uint8_t priority, bool user)
 {
-	if (priority == default_pri)
+	if (priority == default_priority)
 		priority = scheduler::default_priority;
 
 	thread *th = new thread(*this, priority);
@@ -102,11 +97,13 @@ process::create_thread(uint8_t priority, bool user)
 	if (user) {
 		uintptr_t stack_top = stacks_top - (m_threads.count() * stack_size);
 
-		vm_area *vma = new vm_area_open(stack_size, m_space,
-				vm_flags::zero|vm_flags::write);
+		vm_flags flags = vm_flags::zero|vm_flags::write;
+		vm_area *vma = new vm_area_open(stack_size, flags);
 		m_space.add(stack_top - stack_size, vma);
 
-		th->tcb()->rsp3 = stack_top;
+		// Space for null frame - because the page gets zeroed on
+		// allocation, just pointing rsp here does the trick
+		th->tcb()->rsp3 = stack_top - 2 * sizeof(uint64_t);
 	}
 
 	m_threads.append(th);
@@ -120,8 +117,11 @@ process::thread_exited(thread *th)
 	kassert(&th->m_parent == this, "Process got thread_exited for non-child!");
 	uint32_t status = th->m_return_code;
 	m_threads.remove_swap(th);
+	remove_handle(th->self_handle());
 	delete th;
 
+	// TODO: delete the thread's stack VMA
+
 	if (m_threads.count() == 0) {
 		exit(status);
 		return true;