Merge branch 'real-hardware' into fb-driver

Since it's often needed when debugging between the bootloader and
kernel, log the address of the table pages the bootloader allocated.

.gitignore

@@ -3,10 +3,8 @@
 *.bak
 tags
 jsix.log
-*.out
 *.o
 *.a
 sysroot
 .gdb_history
 .peru
-__pycache__

README.md

@@ -1,18 +1,15 @@
-![jsix](assets/jsix.svg)
+# jsix: A toy OS kernel
 
-# The jsix operating system
-
-**jsix** is a custom multi-core x64 operating system that I am building from
-scratch. It's far from finished, or even being usable - see the *Status and
-Roadmap* section, below.
+**jsix** is the kernel for the hobby OS that I am currently building. It's
+far from finished, or even being usable. Instead, it's a sandbox for me to play
+with kernel-level code and explore architectures.
 
 The design goals of the project are:
 
 * Modernity - I'm not interested in designing for legacy systems, or running on
   all hardware out there. My target is only 64 bit architecutres, and modern
   commodity hardware. Currently that means x64 systems with Nehalem or newer
-  CPUs and UEFI firmware. (See [this list][cpu_features] for the currently
-  required CPU features.) Eventually I'd like to work on an AArch64 port,
+  CPUs and UEFI firmware. Eventually I'd like to work on an AArch64 port,
   partly to force myself to factor out the architecture-dependent pieces of the
   code base.
 
@@ -20,11 +17,13 @@ The design goals of the project are:
   processes as possible, in the microkernel fashion. A sub-goal of this is to
   explore where the bottlenecks of such a microkernel are now, and whether
   eschewing legacy hardware will let me design a system that's less bogged down
-  by the traditional microkernel problems.
+  by the traditional microkernel problems. Given that there are no processes
+  yet, the kernel is monolithic by default.
 
 * Exploration - I'm really mostly doing this to have fun learning and exploring
-  modern OS development. Initial feature implementations may temporarily throw
-  away modular design to allow for exploration of the related hardware.
+  modern OS development. Modular design may be tossed out (hopefully
+  temporarily) in some places to allow me to play around with the related
+  hardware.
 
 A note on the name: This kernel was originally named Popcorn, but I have since
 discovered that the Popcorn Linux project is also developing a kernel with that
@@ -32,69 +31,6 @@ name, started around the same time as this project. So I've renamed this kernel
 jsix (Always styled _jsix_ or `j6`, never capitalized) as an homage to L4, xv6,
 and my wonderful wife.
 
-[cpu_features]: https://github.com/justinian/jsix/blob/master/src/libraries/cpu/include/cpu/features.inc
-
-## Status and Roadmap
-
-The following major feature areas are targets for jsix development:
-
-#### UEFI boot loader
-
-_Done._ The bootloader loads the kernel and initial userspace programs, and
-sets up necessary kernel arguments about the memory map and EFI GOP
-framebuffer. Possible future ideas:
-
-- take over more init-time functions from the kernel
-- rewrite it in Zig
-
-#### Memory
-
-_Virtual memory: Sufficient._ The kernel manages virtual memory with a number
-of kinds of `vm_area` objects representing mapped areas, which can belong to
-one or more `vm_space` objects which represent a whole virtual memory space.
-(Each process has a `vm_space`, and so does the kernel itself.)
-
-Remaining to do:
-
-- TLB shootdowns
-- Page swapping
-
-_Physical page allocation: Sufficient._ The current physical page allocator
-implementation suses a group of block representing up-to-1GiB areas of usable
-memory as defined by the bootloader. Each block has a three-level bitmap
-denoting free/used pages.
-
-#### Multitasking
-
-_Sufficient._ The global scheduler object keeps separate ready/blocked lists
-per core. Cores periodically attempt to balance load via work stealing.
-
-User-space tasks are able to create threads as well as other processes.
-
-Several kernel-only tasks exist, though I'm trying to reduce that. Eventually
-only the timekeeping task should be a separate kernel-only thread.
-
-#### API
-
-_In progress._ User-space tasks are able to make syscalls to the kernel via
-fast SYSCALL/SYSRET instructions.
-
-Major tasks still to do:
-
-- The process initialization protocol needs to be re-built entirely.
-- Processes' handles to kernel objects need the ability to check capabilities
-
-#### Hardware Support
-
-  * Framebuffer driver: _In progress._ Currently on machines with a video
-	device accessible by UEFI, jsix starts a user-space framebuffer driver that
-	only prints out kernel logs.
-  * Serial driver: _To do._ Machines without a video device should have a
-	user-space log output task like the framebuffer driver, but currently this
-	is done inside the kernel.
-  * USB driver: _To do_
-  * AHCI (SATA) driver: _To do_
-
 ## Building
 
 jsix uses the [Ninja][] build tool, and generates the build files for it with a
@@ -102,7 +38,7 @@ custom tool called [Bonnibel][]. Bonnibel can be installed with [Cargo][], or
 downloaded as a prebuilt binary from its Github repository.
 
 [Ninja]:    https://ninja-build.org
-[Bonnibel]: https://github.com/justinian/bonnibel_rs
+[Bonnibel]: https://github.com/justinian/bonnibel
 [Cargo]:    https://crates.io/crates/bonnibel
 
 Requrirements:
@@ -135,8 +71,8 @@ I personally run this either from a real debian amd64 testing/buster machine or
 a windows WSL debian testing/buster installation. The following should be
 enough to set up such a system to build the kernel:
 
-    sudo apt install qemu-system-x86 nasm clang-10 mtools curl ninja-build
-    sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-10 1000
-    sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-10 1000
-	curl -L -o pb https://github.com/justinian/bonnibel_rs/releases/download/v2.3.0/pb-linux-amd64 && chmod a+x pb
+    sudo apt install qemu-system-x86 nasm clang-6.0 mtools curl
+    sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-6.0 1000
+    sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-6.0 1000
+	curl -L -o pb https://github.com/justinian/bonnibel/releases/download/2.0.0/pb_linux_amd64 && chmod a+x pb
 

assets/debugging/jsix.elf-gdb.py

@@ -56,77 +56,8 @@ class PrintBacktraceCommand(gdb.Command):
                 return
 
 
-class TableWalkCommand(gdb.Command):
-    def __init__(self):
-        super().__init__("j6tw", gdb.COMMAND_DATA)
-
-    def invoke(self, arg, from_tty):
-        args = gdb.string_to_argv(arg)
-        if len(args) < 2:
-            raise Exception("Must be: j6tw <pml4> <addr>")
-
-        pml4 = int(gdb.parse_and_eval(args[0]))
-        addr = int(gdb.parse_and_eval(args[1]))
-
-        indices = [
-            (addr >> 39) & 0x1ff,
-            (addr >> 30) & 0x1ff,
-            (addr >> 21) & 0x1ff,
-            (addr >> 12) & 0x1ff,
-            ]
-
-        names = ["PML4", "PDP", "PD", "PT"]
-
-        table_flags = [
-            (0x0001, "present"),
-            (0x0002, "write"),
-            (0x0004, "user"),
-            (0x0008, "pwt"),
-            (0x0010, "pcd"),
-            (0x0020, "accessed"),
-            (0x0040, "dirty"),
-            (0x0080, "largepage"),
-            (0x0100, "global"),
-            (0x1080, "pat"),
-            ((1<<63), "xd"),
-        ]
-
-        page_flags = [
-            (0x0001, "present"),
-            (0x0002, "write"),
-            (0x0004, "user"),
-            (0x0008, "pwt"),
-            (0x0010, "pcd"),
-            (0x0020, "accessed"),
-            (0x0040, "dirty"),
-            (0x0080, "pat"),
-            (0x0100, "global"),
-            ((1<<63), "xd"),
-        ]
-
-        flagsets = [table_flags, table_flags, table_flags, page_flags]
-
-        table = pml4
-        entry = 0
-        for i in range(len(indices)):
-            entry = int(gdb.parse_and_eval(f'((uint64_t*){table})[{indices[i]}]'))
-            flagset = flagsets[i]
-            flag_names = " | ".join([f[1] for f in flagset if (entry & f[0]) == f[0]])
-
-            print(f"{names[i]:>4}: {table:016x}")
-            print(f"      index: {indices[i]:3} {entry:016x}")
-            print(f"      flags: {flag_names}")
-
-            if (entry & 1) == 0 or (i < 3 and (entry & 0x80)):
-                break
-
-            table = (entry & 0x7ffffffffffffe00) | 0xffffc00000000000
-
-
-
 PrintStackCommand()
 PrintBacktraceCommand()
-TableWalkCommand()
 
 gdb.execute("target remote :1234")
 gdb.execute("display/i $rip")

assets/jsix.svg

@@ -1 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?><!-- Generator: Gravit.io --><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" style="isolation:isolate" viewBox="176.562 356.069 211.11 113" width="211.11pt" height="113pt"><g><g><rect x="176.562" y="356.069" width="211.11" height="113" transform="matrix(1,0,0,1,0,0)" fill="rgb(255,255,255)"/><g><path d=" M 212.981 372.36 L 219.564 376.16 L 226.147 379.961 L 226.147 387.563 L 226.147 395.164 L 219.564 398.965 L 212.981 402.766 L 206.398 398.965 L 199.815 395.164 L 199.815 387.563 L 199.815 379.961 L 206.398 376.16 L 212.981 372.36 L 212.981 372.36 L 212.981 372.36 Z  M 256.292 397.366 L 262.875 401.166 L 269.458 404.967 L 269.458 412.569 L 269.458 420.17 L 262.875 423.971 L 256.292 427.772 L 249.709 423.971 L 243.126 420.17 L 243.126 412.569 L 243.126 404.967 L 249.709 401.166 L 256.292 397.366 L 256.292 397.366 Z  M 183.622 387.283 L 205.52 374.64 L 227.418 361.997 L 249.316 374.64 L 271.214 387.283 L 271.214 412.569 L 271.214 437.854 L 249.316 450.497 L 227.418 463.14 L 205.52 450.497 L 183.622 437.854 L 183.622 412.569 L 183.622 387.283 L 183.622 387.283 L 183.622 387.283 Z  M 241.855 372.36 L 248.438 376.16 L 255.021 379.961 L 255.021 387.563 L 255.021 395.164 L 248.438 398.965 L 241.855 402.766 L 235.272 398.965 L 228.689 395.164 L 228.689 387.563 L 228.689 379.961 L 235.272 376.16 L 241.855 372.36 Z " fill-rule="evenodd" fill="rgb(49,79,128)"/><path d=" M 298.642 379.579 L 291.621 379.579 L 291.621 372.558 L 298.642 372.558 L 298.642 379.579 Z  M 285.214 446.718 L 285.214 441.452 L 287.32 441.452 L 287.32 441.452 Q 289.339 441.452 290.524 440.092 L 290.524 440.092 L 290.524 440.092 Q 291.708 438.731 291.708 436.625 L 291.708 436.625 L 291.708 387.039 L 298.729 387.039 L 298.729 436.011 L 298.729 436.011 Q 298.729 440.925 295.921 443.822 L 295.921 443.822 L 295.921 443.822 Q 293.113 446.718 288.286 446.718 L 288.286 446.718 L 285.214 446.718 Z  M 306.628 432.676 L 306.628 427.41 L 314.088 427.41 L 314.088 427.41 Q 317.862 427.41 319.573 425.347 L 319.573 425.347 L 319.573 425.347 Q 321.285 423.285 321.285 419.95 L 321.285 419.95 L 321.285 419.95 Q 321.285 417.317 319.705 415.474 L 319.705 415.474 L 319.705 415.474 Q 318.125 413.631 314.966 411.174 L 314.966 411.174 L 314.966 411.174 Q 312.245 408.98 310.621 407.356 L 310.621 407.356 L 310.621 407.356 Q 308.998 405.732 307.813 403.319 L 307.813 403.319 L 307.813 403.319 Q 306.628 400.905 306.628 397.746 L 306.628 397.746 L 306.628 397.746 Q 306.628 393.095 309.744 390.067 L 309.744 390.067 L 309.744 390.067 Q 312.859 387.039 318.125 387.039 L 318.125 387.039 L 325.76 387.039 L 325.76 392.305 L 319.441 392.305 L 319.441 392.305 Q 313.21 392.305 313.21 398.185 L 313.21 398.185 L 313.21 398.185 Q 313.21 400.467 314.615 402.134 L 314.615 402.134 L 314.615 402.134 Q 316.019 403.802 319.003 406.083 L 319.003 406.083 L 319.003 406.083 Q 321.723 408.19 323.479 409.901 L 323.479 409.901 L 323.479 409.901 Q 325.234 411.613 326.463 414.202 L 326.463 414.202 L 326.463 414.202 Q 327.691 416.791 327.691 420.301 L 327.691 420.301 L 327.691 420.301 Q 327.691 426.532 324.4 429.604 L 324.4 429.604 L 324.4 429.604 Q 321.109 432.676 315.141 432.676 L 315.141 432.676 L 306.628 432.676 Z  M 342.611 379.579 L 335.59 379.579 L 335.59 372.558 L 342.611 372.558 L 342.611 379.579 Z  M 342.611 432.676 L 335.59 432.676 L 335.59 387.039 L 342.611 387.039 L 342.611 432.676 Z  M 356.126 432.676 L 348.754 432.676 L 361.392 409.77 L 349.632 387.039 L 356.39 387.039 L 364.639 403.187 L 372.977 387.039 L 379.735 387.039 L 367.974 409.77 L 380.612 432.676 L 373.24 432.676 L 364.639 416.001 L 356.126 432.676 Z " fill="rgb(49,79,128)"/></g></g></g></svg>

external/uefi/boot_services.h

@@ -14,10 +14,8 @@
 namespace uefi {
 namespace bs_impl {
 	using allocate_pages = status (*)(allocate_type, memory_type, size_t, void**);
-	using free_pages = status (*)(void*, size_t);
 	using get_memory_map = status (*)(size_t*, memory_descriptor*, size_t*, size_t*, uint32_t*);
 	using allocate_pool = status (*)(memory_type, uint64_t, void**);
-	using free_pool = status (*)(void*);
 	using handle_protocol = status (*)(handle, const guid*, void**);
 	using create_event = status (*)(evt, tpl, event_notify, void*, event*);
 	using exit_boot_services = status (*)(handle, size_t);
@@ -37,10 +35,10 @@ struct boot_services {
 
 	// Memory Services
 	bs_impl::allocate_pages allocate_pages;
-	bs_impl::free_pages free_pages;
+	void *free_pages;
 	bs_impl::get_memory_map get_memory_map;
 	bs_impl::allocate_pool allocate_pool;
-	bs_impl::free_pool free_pool;
+	void *free_pool;
 
 	// Event & Timer Services
 	bs_impl::create_event create_event;

external/uefi/networking.h

@@ -1,390 +0,0 @@
-#pragma once
-#ifndef _uefi_networking_h_
-#define _uefi_networking_h_
-
-// This Source Code Form is part of the j6-uefi-headers project and is subject
-// to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was
-// not distributed with this file, You can obtain one at
-// http://mozilla.org/MPL/2.0/.
-
-#include <stdint.h>
-#include <uefi/types.h>
-
-namespace uefi {
-
-//
-// IPv4 definitions
-//
-struct ipv4_address
-{
-	uint8_t addr[4];
-};
-
-//
-// IPv6 definitions
-//
-struct ipv6_address
-{
-	uint8_t addr[16];
-};
-
-struct ip6_address_info
-{
-    ipv6_address address;
-    uint8_t prefix_length;
-};
-
-struct ip6_route_table
-{
-    ipv6_address gateway;
-    ipv6_address destination;
-    uint8_t prefix_length;
-};
-
-enum class ip6_neighbor_state : int {
-    incomplete,
-    reachable,
-    stale,
-    delay,
-    probe,
-}
-
-struct ip6_neighbor_cache
-{
-    ipv6_address neighbor;
-    mac_address link_address;
-    ip6_neighbor_state state;
-};
-
-enum class icmpv6_type : uint8_t
-{
-    dest_unreachable = 0x1,
-    packet_too_big = 0x2,
-    time_exceeded = 0x3,
-    parameter_problem = 0x4,
-    echo_request = 0x80,
-    echo_reply = 0x81,
-    listener_query = 0x82,
-    listener_report = 0x83,
-    listener_done = 0x84,
-    router_solicit = 0x85,
-    router_advertise = 0x86,
-    neighbor_solicit = 0x87,
-    neighbor_advertise = 0x88,
-    redirect = 0x89,
-    listener_report_2 = 0x8f,
-};
-
-enum class icmpv6_code : uint8_t
-{
-    // codes for icmpv6_type::dest_unreachable
-    no_route_to_dest = 0x0,
-    comm_prohibited = 0x1,
-    beyond_scope = 0x2,
-    addr_unreachable = 0x3,
-    port_unreachable = 0x4,
-    source_addr_failed = 0x5,
-    route_rejected = 0x6,
-
-    // codes for icmpv6_type::time_exceeded
-    timeout_hop_limit = 0x0,
-    timeout_reassemble = 0x1,
-
-    // codes for icmpv6_type::parameter_problem
-    erroneous_header = 0x0,
-    unrecognize_next_hdr = 0x1,
-    unrecognize_option = 0x2,
-};
-
-struct ip6_icmp_type
-{
-    icmpv6_type type;
-    icmpv6_code code;
-};
-
-struct ip6_config_data
-{
-    uint8_t default_protocol;
-    bool accept_any_protocol;
-    bool accept_icmp_errors;
-    bool accept_promiscuous;
-    ipv6_address destination_address;
-    ipv6_address station_address;
-    uint8_t traffic_class;
-    uint8_t hop_limit;
-    uint32_t flow_label;
-    uint32_t receive_timeout;
-    uint32_t transmit_timeout;
-};
-
-struct ip6_mode_data
-{
-    bool is_started;
-    uint32_t max_packet_size;
-    ip6_config_data config_data;
-    bool is_configured;
-    uint32_t address_count;
-    ip6_address_info * address_list;
-    uint32_t group_count;
-    ipv6_address * group_table;
-    uint32_t route_count;
-    ip6_route_table * route_table;
-    uint32_t neighbor_count;
-    ip6_neighbor_cache * neighbor_cache;
-    uint32_t prefix_count;
-    ip6_address_info * prefix_table;
-    uint32_t icmp_type_count;
-     * icmp_type_list;
-};
-
-struct ip6_header
-{
-    uint8_t traffic_class_h : 4;
-    uint8_t version : 4;
-    uint8_t flow_label_h : 4;
-    uint8_t traffic_class_l : 4;
-    uint16_t flow_label_l;
-    uint16_t payload_length;
-    uint8_t next_header;
-    uint8_t hop_limit;
-    ipv6_address source_address;
-    ipv6_address destination_address;
-} __attribute__ ((packed));
-
-struct ip6_fragment_data
-{
-    uint32_t fragment_length;
-    void *fragment_buffer;
-};
-
-struct ip6_override_data
-{
-    uint8_t protocol;
-    uint8_t hop_limit;
-    uint32_t flow_label;
-};
-
-struct ip6_receive_data
-{
-    time time_stamp;
-    event recycle_signal;
-    uint32_t header_length;
-    ip6_header *header;
-    uint32_t data_length;
-    uint32_t fragment_count;
-    ip6_fragment_data fragment_table[1];
-};
-
-struct ip6_transmit_data
-{
-    ipv6_address destination_address;
-    ip6_override_data *override_data;
-
-    uint32_t ext_hdrs_length;
-    void *ext_hdrs;
-    uint8_t next_header;
-    uint32_t data_length;
-    uint32_t fragment_count;
-    ip6_fragment_data fragment_table[1];
-};
-
-struct ip6_completion_token
-{
-    event event;
-    status status;
-    union {
-        ip6_receive_data *rx_data;
-        ip6_transmit_data *tx_data;
-    } packet;
-};
-
-enum class ip6_config_data_type : int
-{
-    interface_info,
-    alt_interface_id,
-    policy,
-    dup_addr_detect_transmits,
-    manual_address,
-    gateway,
-    dns_server,
-    maximum
-};
-
-struct ip6_config_interface_info
-{
-    wchar_t name[32];
-    uint8_t if_type;
-    uint32_t hw_address_size;
-    mac_address hw_address;
-    uint32_t address_info_count;
-    ip6_address_info *address_info;
-    uint32_t route_count;
-    ip6_route_table *route_table;
-};
-
-struct ip6_config_interface_id
-{
-    uint8_t id[8];
-};
-
-enum class ip6_config_policy : int
-{
-    manual,
-    automatic
-};
-
-struct ip6_config_dup_addr_detect_transmits
-{
-    uint32_t dup_addr_detect_transmits;
-};
-
-struct ip6_config_manual_address
-{
-    ipv6_address address;
-    bool is_anycast;
-    uint8_t prefix_length;
-};
-
-//
-// IP definitions
-//
-union ip_address
-{
-	uint8_t addr[4];
-	ipv4_address v4;
-	ipv6_address v6;
-};
-
-//
-// HTTP definitions
-//
-struct httpv4_access_point
-{
-	bool use_default_address;
-	ipv4_address local_address;
-	ipv4_address local_subnet;
-	uint16_t local_port;
-};
-
-struct httpv6_access_point
-{
-	ipv6_address local_address;
-	uint16_t local_port;
-};
-
-enum class http_version : int {
-	v10,
-	v11,
-	unsupported,
-};
-
-struct http_config_data
-{
-	http_version http_version;
-	uint32_t time_out_millisec;
-	bool local_address_is_ipv6;
-	union {
-		httpv4_access_point *ipv4_node;
-		httpv6_access_point *ipv6_node;
-	} access_point;
-};
-
-enum class http_method : int {
-	get,
-	post,
-	patch,
-	options,
-	connect,
-	head,
-	put,
-	delete_,
-	trace,
-};
-
-struct http_request_data
-{
-	http_method method;
-	wchar_t *url;
-};
-
-enum class http_status_code : int {
-	unsupported,
-	continue_,
-	switching_protocols,
-	ok,
-	created,
-	accepted,
-	non_authoritative_information,
-	no_content,
-	reset_content,
-	partial_content,
-	multiple_choices,
-	moved_permanently,
-	found,
-	see_other,
-	not_modified,
-	use_proxy,
-	temporary_redirect,
-	bad_request,
-	unauthorized,
-	payment_required,
-	forbidden,
-	not_found,
-	method_not_allowed,
-	not_acceptable,
-	proxy_authentication_required,
-	request_time_out,
-	conflict,
-	gone,
-	length_required,
-	precondition_failed,
-	request_entity_too_large,
-	request_uri_too_large,
-	unsupported_media_type,
-	requested_range_not_satisfied,
-	expectation_failed,
-	internal_server_error,
-	not_implemented,
-	bad_gateway,
-	service_unavailable,
-	gateway_timeout,
-	http_version_not_supported,
-	permanent_redirect,  // I hate your decisions, uefi.
-};
-
-struct http_response_data
-{
-	http_status_code status_code;
-};
-
-struct http_header
-{
-	char *field_name;
-	char *field_value;
-};
-
-struct http_message
-{
-	union {
-		http_request_data *request;
-		http_response_data *response;
-	} data;
-
-	size_t header_count;
-	http_header *headers;
-
-	size_t body_length;
-	void *body;
-};
-
-struct http_token
-{
-	event event;
-	status status;
-	http_message *message;
-};
-
-
-} // namespace uefi
-
-#endif

external/uefi/protos/device_path.h

@@ -16,7 +16,6 @@ struct device_path
 {
     static constexpr uefi::guid guid{ 0x09576e91,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
-
     uint8_t type;
     uint8_t sub_type;
     uint16_t length;

external/uefi/protos/file_info.h

@@ -16,7 +16,6 @@ struct file_info
 {
     static constexpr uefi::guid guid{ 0x09576e92,0x6d3f,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
-
     uint64_t size;
     uint64_t file_size;
     uint64_t physical_size;

external/uefi/protos/graphics_output.h

@@ -17,7 +17,6 @@ struct graphics_output
 {
     static constexpr uefi::guid guid{ 0x9042a9de,0x23dc,0x4a38,{0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a} };
 
-
     inline uefi::status query_mode(uint32_t mode_number, uint64_t * size_of_info, uefi::graphics_output_mode_info ** info) {
         return _query_mode(this, mode_number, size_of_info, info);
     }

external/uefi/protos/http.h

@@ -1,72 +0,0 @@
-#pragma once
-#ifndef _uefi_protos_http_h_
-#define _uefi_protos_http_h_
-
-// This file was auto generated by the j6-uefi-headers project. Please see
-// https://github.com/justinian/j6-uefi-headers for more information.
-
-#include <uefi/guid.h>
-#include <uefi/types.h>
-#include <uefi/networking.h>
-
-namespace uefi {
-namespace protos {
-struct http;
-
-struct http
-{
-    static constexpr uefi::guid guid{ 0x7a59b29b,0x910b,0x4171,{0x82,0x42,0xa8,0x5a,0x0d,0xf2,0x5b,0x5b} };
-    static constexpr uefi::guid service_binding{ 0xbdc8e6af,0xd9bc,0x4379,{0xa7,0x2a,0xe0,0xc4,0xe7,0x5d,0xae,0x1c} };
-
-
-    inline uefi::status get_mode_data(uefi::http_config_data * http_config_data) {
-        return _get_mode_data(this, http_config_data);
-    }
-
-    inline uefi::status configure(uefi::http_config_data * http_config_data) {
-        return _configure(this, http_config_data);
-    }
-
-    inline uefi::status request(uefi::http_token * token) {
-        return _request(this, token);
-    }
-
-    inline uefi::status cancel(uefi::http_token * token) {
-        return _cancel(this, token);
-    }
-
-    inline uefi::status response(uefi::http_token * token) {
-        return _response(this, token);
-    }
-
-    inline uefi::status poll() {
-        return _poll(this);
-    }
-
-
-protected:
-    using _get_mode_data_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
-    _get_mode_data_def _get_mode_data;
-
-    using _configure_def = uefi::status (*)(uefi::protos::http *, uefi::http_config_data *);
-    _configure_def _configure;
-
-    using _request_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
-    _request_def _request;
-
-    using _cancel_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
-    _cancel_def _cancel;
-
-    using _response_def = uefi::status (*)(uefi::protos::http *, uefi::http_token *);
-    _response_def _response;
-
-    using _poll_def = uefi::status (*)(uefi::protos::http *);
-    _poll_def _poll;
-
-public:
-};
-
-} // namespace protos
-} // namespace uefi
-
-#endif // _uefi_protos_http_h_

external/uefi/protos/ip6.h

@@ -1,93 +0,0 @@
-#pragma once
-#ifndef _uefi_protos_ip6_h_
-#define _uefi_protos_ip6_h_
-
-// This file was auto generated by the j6-uefi-headers project. Please see
-// https://github.com/justinian/j6-uefi-headers for more information.
-
-#include <uefi/guid.h>
-#include <uefi/types.h>
-#include <uefi/networking.h>
-
-namespace uefi {
-namespace protos {
-struct ip6;
-
-struct ip6
-{
-    static constexpr uefi::guid guid{ 0x2c8759d5,0x5c2d,0x66ef,{0x92,0x5f,0xb6,0x6c,0x10,0x19,0x57,0xe2} };
-    static constexpr uefi::guid service_binding{ 0xec835dd3,0xfe0f,0x617b,{0xa6,0x21,0xb3,0x50,0xc3,0xe1,0x33,0x88} };
-
-
-    inline uefi::status get_mode_data(uefi::ip6_mode_data * ip6_mode_data, uefi::managed_network_config_data * mnp_config_data, uefi::simple_network_mode * snp_config_data) {
-        return _get_mode_data(this, ip6_mode_data, mnp_config_data, snp_config_data);
-    }
-
-    inline uefi::status configure(uefi::ip6_config_data * ip6_config_data) {
-        return _configure(this, ip6_config_data);
-    }
-
-    inline uefi::status groups(bool join_flag, uefi::ipv6_address * group_address) {
-        return _groups(this, join_flag, group_address);
-    }
-
-    inline uefi::status routes(bool delete_route, uefi::ipv6_address * destination, uint8_t prefix_length, uefi::ipv6_address * gateway_address) {
-        return _routes(this, delete_route, destination, prefix_length, gateway_address);
-    }
-
-    inline uefi::status neighbors(bool delete_flag, uefi::ipv6_address * target_ip6_address, uefi::mac_address * target_link_address, uint32_t timeout, bool override) {
-        return _neighbors(this, delete_flag, target_ip6_address, target_link_address, timeout, override);
-    }
-
-    inline uefi::status transmit(uefi::ip6_completion_token * token) {
-        return _transmit(this, token);
-    }
-
-    inline uefi::status receive(uefi::ip6_completion_token * token) {
-        return _receive(this, token);
-    }
-
-    inline uefi::status cancel(uefi::ip6_completion_token * token) {
-        return _cancel(this, token);
-    }
-
-    inline uefi::status poll() {
-        return _poll(this);
-    }
-
-
-protected:
-    using _get_mode_data_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_mode_data *, uefi::managed_network_config_data *, uefi::simple_network_mode *);
-    _get_mode_data_def _get_mode_data;
-
-    using _configure_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_config_data *);
-    _configure_def _configure;
-
-    using _groups_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *);
-    _groups_def _groups;
-
-    using _routes_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uint8_t, uefi::ipv6_address *);
-    _routes_def _routes;
-
-    using _neighbors_def = uefi::status (*)(uefi::protos::ip6 *, bool, uefi::ipv6_address *, uefi::mac_address *, uint32_t, bool);
-    _neighbors_def _neighbors;
-
-    using _transmit_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
-    _transmit_def _transmit;
-
-    using _receive_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
-    _receive_def _receive;
-
-    using _cancel_def = uefi::status (*)(uefi::protos::ip6 *, uefi::ip6_completion_token *);
-    _cancel_def _cancel;
-
-    using _poll_def = uefi::status (*)(uefi::protos::ip6 *);
-    _poll_def _poll;
-
-public:
-};
-
-} // namespace protos
-} // namespace uefi
-
-#endif // _uefi_protos_ip6_h_

external/uefi/protos/ip6_config.h

@@ -1,57 +0,0 @@
-#pragma once
-#ifndef _uefi_protos_ip6_config_h_
-#define _uefi_protos_ip6_config_h_
-
-// This file was auto generated by the j6-uefi-headers project. Please see
-// https://github.com/justinian/j6-uefi-headers for more information.
-
-#include <uefi/guid.h>
-#include <uefi/types.h>
-#include <uefi/networking.h>
-
-namespace uefi {
-namespace protos {
-struct ip6_config;
-
-struct ip6_config
-{
-    static constexpr uefi::guid guid{ 0x937fe521,0x95ae,0x4d1a,{0x89,0x29,0x48,0xbc,0xd9,0x0a,0xd3,0x1a} };
-
-
-    inline uefi::status set_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
-        return _set_data(this, data_type, data_size, data);
-    }
-
-    inline uefi::status get_data(uefi::ip6_config_data_type data_type, size_t data_size, void * data) {
-        return _get_data(this, data_type, data_size, data);
-    }
-
-    inline uefi::status register_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
-        return _register_data_notify(this, data_type, event);
-    }
-
-    inline uefi::status unregister_data_notify(uefi::ip6_config_data_type data_type, uefi::event event) {
-        return _unregister_data_notify(this, data_type, event);
-    }
-
-
-protected:
-    using _set_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
-    _set_data_def _set_data;
-
-    using _get_data_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, size_t, void *);
-    _get_data_def _get_data;
-
-    using _register_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
-    _register_data_notify_def _register_data_notify;
-
-    using _unregister_data_notify_def = uefi::status (*)(uefi::protos::ip6_config *, uefi::ip6_config_data_type, uefi::event);
-    _unregister_data_notify_def _unregister_data_notify;
-
-public:
-};
-
-} // namespace protos
-} // namespace uefi
-
-#endif // _uefi_protos_ip6_config_h_

external/uefi/protos/load_file.h

@@ -1,36 +0,0 @@
-#pragma once
-#ifndef _uefi_protos_load_file_h_
-#define _uefi_protos_load_file_h_
-
-// This file was auto generated by the j6-uefi-headers project. Please see
-// https://github.com/justinian/j6-uefi-headers for more information.
-
-#include <uefi/guid.h>
-#include <uefi/types.h>
-#include <uefi/protos/device_path.h>
-
-namespace uefi {
-namespace protos {
-struct load_file;
-
-struct load_file
-{
-    static constexpr uefi::guid guid{ {0x56ec3091,0x954c,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
-
-
-    inline uefi::status load_file(uefi::protos::device_path * file_path, bool boot_policy, size_t * buffer_size, void * buffer) {
-        return _load_file(this, file_path, boot_policy, buffer_size, buffer);
-    }
-
-
-protected:
-    using _load_file_def = uefi::status (*)(uefi::protos::load_file *, uefi::protos::device_path *, bool, size_t *, void *);
-    _load_file_def _load_file;
-
-public:
-};
-
-} // namespace protos
-} // namespace uefi
-
-#endif // _uefi_protos_load_file_h_

external/uefi/protos/loaded_image.h

@@ -18,7 +18,6 @@ struct loaded_image
 {
     static constexpr uefi::guid guid{ 0x5b1b31a1,0x9562,0x11d2,{0x8e,0x3f,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
-
     inline uefi::status unload(uefi::handle image_handle) {
         return _unload(image_handle);
     }

external/uefi/protos/service_binding.h

@@ -1,41 +0,0 @@
-#pragma once
-#ifndef _uefi_protos_service_binding_h_
-#define _uefi_protos_service_binding_h_
-
-// This file was auto generated by the j6-uefi-headers project. Please see
-// https://github.com/justinian/j6-uefi-headers for more information.
-
-#include <uefi/guid.h>
-#include <uefi/types.h>
-
-namespace uefi {
-namespace protos {
-struct service_binding;
-
-struct service_binding
-{
-
-
-    inline uefi::status create_child(uefi::handle * child_handle) {
-        return _create_child(this, child_handle);
-    }
-
-    inline uefi::status destroy_child(uefi::handle child_handle) {
-        return _destroy_child(this, child_handle);
-    }
-
-
-protected:
-    using _create_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle *);
-    _create_child_def _create_child;
-
-    using _destroy_child_def = uefi::status (*)(uefi::protos::service_binding *, uefi::handle);
-    _destroy_child_def _destroy_child;
-
-public:
-};
-
-} // namespace protos
-} // namespace uefi
-
-#endif // _uefi_protos_service_binding_h_

external/uefi/protos/simple_file_system.h

@@ -17,7 +17,6 @@ struct simple_file_system
 {
     static constexpr uefi::guid guid{ 0x0964e5b22,0x6459,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
-
     inline uefi::status open_volume(uefi::protos::file ** root) {
         return _open_volume(this, root);
     }

external/uefi/protos/simple_text_output.h

@@ -17,7 +17,6 @@ struct simple_text_output
 {
     static constexpr uefi::guid guid{ 0x387477c2,0x69c7,0x11d2,{0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b} };
 
-
     inline uefi::status reset(bool extended_verification) {
         return _reset(this, extended_verification);
     }

modules.yaml

@@ -12,7 +12,6 @@ modules:
             - src/kernel
         source:
             - src/kernel/apic.cpp
-            - src/kernel/ap_startup.s
             - src/kernel/assert.cpp
             - src/kernel/boot.s
             - src/kernel/clock.cpp
@@ -25,12 +24,12 @@ modules:
             - src/kernel/frame_allocator.cpp
             - src/kernel/fs/gpt.cpp
             - src/kernel/gdt.cpp
-            - src/kernel/gdtidt.s
+            - src/kernel/gdt.s
             - src/kernel/hpet.cpp
-            - src/kernel/idt.cpp
             - src/kernel/interrupts.cpp
             - src/kernel/interrupts.s
             - src/kernel/io.cpp
+            - src/kernel/loader.s
             - src/kernel/log.cpp
             - src/kernel/main.cpp
             - src/kernel/memory_bootstrap.cpp
@@ -43,7 +42,6 @@ modules:
             - src/kernel/objects/system.cpp
             - src/kernel/objects/vm_area.cpp
             - src/kernel/page_table.cpp
-            - src/kernel/page_tree.cpp
             - src/kernel/pci.cpp
             - src/kernel/scheduler.cpp
             - src/kernel/serial.cpp
@@ -58,7 +56,7 @@ modules:
             - src/kernel/syscalls/thread.cpp
             - src/kernel/syscalls/vm_area.cpp
             - src/kernel/task.s
-            - src/kernel/tss.cpp
+            - src/kernel/vm_mapper.cpp
             - src/kernel/vm_space.cpp
 
     boot:
@@ -114,7 +112,6 @@ modules:
             - src/libraries/kutil/logger.cpp
             - src/libraries/kutil/memory.cpp
             - src/libraries/kutil/printf.c
-            - src/libraries/kutil/spinlock.cpp
 
     cpu:
         kind: lib
@@ -122,24 +119,14 @@ modules:
         includes:
             - src/libraries/cpu/include
         source:
-            - src/libraries/cpu/cpu_id.cpp
+            - src/libraries/cpu/cpu.cpp
 
-    j6:
-      kind: lib
-      output: libj6.a
-      includes:
-        - src/libraries/j6/include
-      target: user
-      source:
-        - src/libraries/j6/syscalls.s
 
     libc:
         kind: lib
         output: libc.a
         includes:
           - src/libraries/libc/include
-        deps:
-            - j6
         target: user
         defines:
             - DISABLE_SSE
@@ -158,6 +145,7 @@ modules:
             - src/libraries/libc/arch/x86_64/_Exit.s
             - src/libraries/libc/arch/x86_64/crt0.s
             - src/libraries/libc/arch/x86_64/init_libc.c
+            - src/libraries/libc/arch/x86_64/syscalls.s
             - src/libraries/libc/ctype/isalnum.c
             - src/libraries/libc/ctype/isalpha.c
             - src/libraries/libc/ctype/isblank.c
@@ -326,7 +314,6 @@ modules:
             - src/tests/main.cpp
             - src/tests/map.cpp
             - src/tests/vector.cpp
-
 overlays:
     - url: https://f000.backblazeb2.com/file/jsix-os/sysroot-llvm8-20190706.tar.bz2
       path: sysroot

src/boot/console.cpp

@@ -55,7 +55,7 @@ console::console(uefi::boot_services *bs, uefi::protos::simple_text_output *out)
 	m_out->output_string(GIT_VERSION_WIDE);
 
 	m_out->set_attribute(uefi::attribute::light_gray);
-	m_out->output_string(L" booting...\r\n");
+	m_out->output_string(L" booting...\r\n\n");
 
 	if (m_fb.type != kernel::args::fb_type::none) {
 		wchar_t const * type = nullptr;

src/boot/error.cpp

@@ -35,28 +35,27 @@ message(uefi::status status)
 }
 
 [[ noreturn ]] static void
-cpu_assert(uefi::status s, const wchar_t *message, size_t line)
+cpu_assert(uefi::status s, const wchar_t *message)
 {
 	asm volatile (
 		"movq $0xeeeeeeebadbadbad, %%r8;"
 		"movq %0, %%r9;"
 		"movq %1, %%r10;"
-		"movq %2, %%r11;"
 		"movq $0, %%rdx;"
 		"divq %%rdx;"
 		:
-		: "r"((uint64_t)s), "r"(message), "r"(line)
+		: "r"((uint64_t)s), "r"(message)
 		: "rax", "rdx", "r8", "r9", "r10");
 	while (1) asm("hlt");
 }
 
 [[ noreturn ]] void
-raise(uefi::status status, const wchar_t *message, size_t line)
+raise(uefi::status status, const wchar_t *message)
 {
 	if(status_line::fail(message, status))
 		while (1) asm("hlt");
 	else
-		cpu_assert(status, message, line);
+		cpu_assert(status, message);
 }
 
 

src/boot/error.h

@@ -12,7 +12,7 @@ class console;
 namespace error {
 
 /// Halt or exit the program with the given error status/message
-[[ noreturn ]] void raise(uefi::status status, const wchar_t *message, size_t line = 0);
+[[ noreturn ]] void raise(uefi::status status, const wchar_t *message);
 
 const wchar_t * message(uefi::status status);
 
@@ -28,6 +28,6 @@ void debug_break();
 #define try_or_raise(s, m) \
 	do { \
 		uefi::status _s = (s); \
-		if (uefi::is_error(_s)) ::boot::error::raise(_s, (m), __LINE__); \
+		if (uefi::is_error(_s)) ::boot::error::raise(_s, (m)); \
 	} while(0)
 

src/boot/fs.cpp

@@ -78,7 +78,7 @@ file::load(uefi::memory_type mem_type)
 		m_file->read(&size, data),
 		L"Could not read from file");
 
-	return { .size = size,  .data = data };
+	return { .data = data, .size = size };
 }
 
 file

src/boot/guids.cpp

@@ -0,0 +1,13 @@
+#include "guids.h"
+
+#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) \
+	EFI_GUID name __attribute__((section(".rodata"))) = {dw, w1, w2, {b1, b2, b3, b4, b5, b6, b7, b8}}
+#include "guids.inc"
+#undef GUID
+
+int is_guid(EFI_GUID *a, EFI_GUID *b)
+{
+	uint64_t *ai = (uint64_t *)a;
+	uint64_t *bi = (uint64_t *)b;
+	return ai[0] == bi[0] && ai[1] == bi[1];
+}

src/boot/guids.h

@@ -0,0 +1,8 @@
+#pragma once
+#include <efi/efi.h>
+
+int is_guid(EFI_GUID *a, EFI_GUID *b);
+
+#define GUID(dw, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8, name) extern EFI_GUID name
+#include "guids.inc"
+#undef GUID

src/boot/guids.inc

@@ -0,0 +1,11 @@
+GUID(0xeb9d2d30,0x2d88,0x11d3,0x9a,0x16,0x00,0x90,0x27,0x3f,0xc1,0x4d, guid_acpi1);
+GUID(0x8868e871,0xe4f1,0x11d3,0xbc,0x22,0x00,0x80,0xc7,0x3c,0x88,0x81, guid_acpi2);
+GUID(0x09576e92,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_file_info);
+GUID(0x9042a9de,0x23dc,0x4a38,0x96,0xfb,0x7a,0xde,0xd0,0x80,0x51,0x6a, guid_gfx_out);
+GUID(0x964e5b22,0x6459,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_simple_filesystem);
+GUID(0x09576e91,0x6d3f,0x11d2,0x8e,0x39,0x00,0xa0,0xc9,0x69,0x72,0x3b, guid_device_path);
+GUID(0x8b843e20,0x8132,0x4852,0x90,0xcc,0x55,0x1a,0x4e,0x4a,0x7f,0x1c, guid_device_path_to_text);
+
+GUID(0x10d0669c,0x9ec6,0x4268,0xbc,0x48,0xff,0x74,0x75,0x21,0xfe,0x07, guid_jsix_vendor);
+
+// vim: ft=c

src/boot/hardware.cpp

@@ -37,25 +37,8 @@ find_acpi_table(uefi::system_table *st)
 	return reinterpret_cast<void*>(acpi1_table);
 }
 
-static uint64_t
-rdmsr(uint32_t addr)
-{
-	uint32_t low, high;
-	__asm__ __volatile__ ("rdmsr" : "=a"(low), "=d"(high) : "c"(addr));
-	return (static_cast<uint64_t>(high) << 32) | low;
-}
-
-static void
-wrmsr(uint32_t addr, uint64_t value)
-{
-	uint32_t low = value & 0xffffffff;
-	uint32_t high = value >> 32;
-	__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
-}
-
-
 void
-setup_control_regs()
+setup_cr4()
 {
 	uint64_t cr4 = 0;
 	asm volatile ( "mov %%cr4, %0" : "=r" (cr4) );
@@ -66,15 +49,6 @@ setup_control_regs()
 		0x020000 | // Enable PCIDs
 		0;
 	asm volatile ( "mov %0, %%cr4" :: "r" (cr4) );
-
-	// Set up IA32_EFER
-	constexpr uint32_t IA32_EFER = 0xC0000080;
-	uint64_t efer = rdmsr(IA32_EFER);
-	efer |=
-		0x0001 | // Enable SYSCALL
-		0x0800 | // Enable NX bit
-		0;
-	wrmsr(IA32_EFER, efer);
 }
 
 } // namespace hw

src/boot/hardware.h

@@ -13,8 +13,8 @@ namespace hw {
 /// significant bit set to 1.
 void * find_acpi_table(uefi::system_table *st);
 
-/// Enable CPU options in CR4 etc for the kernel starting state.
-void setup_control_regs();
+/// Enable CPU options in CR4 for the kernel starting state.
+void setup_cr4();
 
 } // namespace hw
 } // namespace boot

src/boot/loader.cpp

@@ -27,7 +27,7 @@ load_file(
 	fs::file file = disk.open(path);
 	buffer b = file.load(type);
 
-	//console::print(L"    Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
+	console::print(L"    Loaded at: 0x%lx, %d bytes\r\n", b.data, b.size);
 	return b;
 }
 
@@ -87,9 +87,6 @@ load_program(
 
 	bs->set_mem(pages, total_size, 0);
 
-	program.base = prog_base;
-	program.total_size = total_size;
-	program.num_sections = 0;
 	for (int i = 0; i < header->ph_num; ++i) {
 		ptrdiff_t offset = header->ph_offset + i * header->ph_entsize;
 		const elf::program_header *pheader =
@@ -98,18 +95,14 @@ load_program(
 		if (pheader->type != elf::PT_LOAD)
 			continue;
 
-		args::program_section &section = program.sections[program.num_sections++];
-
 		void *src_start = offset_ptr<void>(data.data, pheader->offset);
 		void *dest_start = offset_ptr<void>(pages, pheader->vaddr - prog_base);
-
 		bs->copy_mem(dest_start, src_start, pheader->file_size);
-		section.phys_addr = reinterpret_cast<uintptr_t>(dest_start);
-		section.virt_addr = pheader->vaddr;
-		section.size = pheader->mem_size;
-		section.type = static_cast<args::section_flags>(pheader->flags);
 	}
 
+	program.phys_addr = reinterpret_cast<uintptr_t>(pages);
+	program.size = total_size;
+	program.virt_addr = prog_base;
 	program.entrypoint = header->entrypoint;
 }
 

src/boot/main.cpp

@@ -8,7 +8,7 @@
 #include <stdint.h>
 
 #include "console.h"
-#include "cpu/cpu_id.h"
+#include "cpu/cpu.h"
 #include "error.h"
 #include "fs.h"
 #include "hardware.h"
@@ -93,8 +93,6 @@ add_module(args::header *args, args::mod_type type, buffer &data)
 	m.type = type;
 	m.location = data.data;
 	m.size = data.size;
-
-	change_pointer(m.location);
 }
 
 /// Check that all required cpu features are supported
@@ -154,13 +152,18 @@ uefi_preboot(uefi::handle image, uefi::system_table *st)
 		loader::load_program(program, desc.name, buf, bs);
 	}
 
+	for (unsigned i = 0; i < args->num_modules; ++i) {
+		args::module &mod = args->modules[i];
+		change_pointer(mod.location);
+	}
+
 	return args;
 }
 
 memory::efi_mem_map
 uefi_exit(args::header *args, uefi::handle image, uefi::boot_services *bs)
 {
-	status_line status {L"Exiting UEFI", nullptr, false};
+	status_line status {L"Exiting UEFI"};
 
 	memory::efi_mem_map map =
 		memory::build_kernel_mem_map(args, bs);
@@ -188,27 +191,17 @@ efi_main(uefi::handle image, uefi::system_table *st)
 	args->video = con.fb();
 	status_bar status {con.fb()}; // Switch to fb status display
 
-	// Map the kernel to the appropriate address
 	args::program &kernel = args->programs[0];
-	for (auto &section : kernel.sections)
-		if (section.size)
-			paging::map_section(args, section);
-
-	memory::fix_frame_blocks(args);
-
+	paging::map_pages(args, kernel.phys_addr, kernel.virt_addr, kernel.size);
 	kernel::entrypoint kentry =
 		reinterpret_cast<kernel::entrypoint>(kernel.entrypoint);
 	status.next();
 
-	hw::setup_control_regs();
 	memory::virtualize(args->pml4, map, st->runtime_services);
 	status.next();
 
-	change_pointer(args);
 	change_pointer(args->pml4);
-	change_pointer(args->modules);
-	change_pointer(args->programs);
-
+	hw::setup_cr4();
 	status.next();
 
 	kentry(args);

src/boot/memory.cpp

@@ -14,8 +14,6 @@ namespace memory {
 
 using mem_entry = kernel::args::mem_entry;
 using mem_type = kernel::args::mem_type;
-using frame_block = kernel::args::frame_block;
-using kernel::args::frames_per_block;
 
 size_t fixup_pointer_index = 0;
 void **fixup_pointers[64];
@@ -109,144 +107,30 @@ can_merge(mem_entry &prev, mem_type type, uefi::memory_descriptor *next)
 }
 
 void
-get_uefi_mappings(efi_mem_map &map, uefi::boot_services *bs)
+get_uefi_mappings(efi_mem_map *map, bool allocate, uefi::boot_services *bs)
 {
-	size_t length = map.total;
+	size_t length = 0;
 	uefi::status status = bs->get_memory_map(
-		&length, map.entries, &map.key, &map.size, &map.version);
-	map.length = length;
-
-	if (status == uefi::status::success)
-		return;
+		&length, nullptr, &map->key, &map->size, &map->version);
 
 	if (status != uefi::status::buffer_too_small)
 		error::raise(status, L"Error getting memory map size");
 
-	if (map.entries) {
-		try_or_raise(
-			bs->free_pool(reinterpret_cast<void*>(map.entries)),
-			L"Freeing previous memory map space");
-	}
+	map->length = length;
 
-	map.total = length + 10*map.size;
+	if (allocate) {
+		map->length += 10*map->size;
 
 		try_or_raise(
 			bs->allocate_pool(
-			uefi::memory_type::loader_data, map.total,
-			reinterpret_cast<void**>(&map.entries)),
+				uefi::memory_type::loader_data, map->length,
+				reinterpret_cast<void**>(&map->entries)),
 			L"Allocating space for memory map");
 
-	map.length = map.total;
 		try_or_raise(
-		bs->get_memory_map(&map.length, map.entries, &map.key, &map.size, &map.version),
+			bs->get_memory_map(&map->length, map->entries, &map->key, &map->size, &map->version),
 			L"Getting UEFI memory map");
 	}
-
-inline size_t bitmap_size(size_t frames) { return (frames + 63) / 64; }
-inline size_t num_blocks(size_t frames) { return (frames + (frames_per_block-1)) / frames_per_block; }
-
-void
-build_kernel_frame_blocks(const mem_entry *map, size_t nent, kernel::args::header *args, uefi::boot_services *bs)
-{
-	status_line status {L"Creating kernel frame accounting map"};
-
-	size_t block_count = 0;
-	size_t total_bitmap_size = 0;
-	for (size_t i = 0; i < nent; ++i) {
-		const mem_entry &ent = map[i];
-		if (ent.type != mem_type::free)
-			continue;
-
-		block_count += num_blocks(ent.pages);
-		total_bitmap_size += bitmap_size(ent.pages) * sizeof(uint64_t);
-	}
-
-	size_t total_size = block_count * sizeof(frame_block) + total_bitmap_size;
-
-	frame_block *blocks = nullptr;
-	try_or_raise(
-		bs->allocate_pages(
-			uefi::allocate_type::any_pages,
-			uefi::memory_type::loader_data,
-			bytes_to_pages(total_size),
-			reinterpret_cast<void**>(&blocks)),
-		L"Error allocating kernel frame block space");
-
-	frame_block *next_block = blocks;
-	for (size_t i = 0; i < nent; ++i) {
-		const mem_entry &ent = map[i];
-		if (ent.type != mem_type::free)
-			continue;
-
-		size_t page_count = ent.pages;
-		uintptr_t base_addr = ent.start;
-		while (page_count) {
-			frame_block *blk = next_block++;
-			bs->set_mem(blk, sizeof(frame_block), 0);
-
-			blk->attrs = ent.attr;
-			blk->base = base_addr;
-			base_addr += frames_per_block * page_size;
-
-			if (page_count >= frames_per_block) {
-				page_count -= frames_per_block;
-				blk->count = frames_per_block;
-				blk->map1 = ~0ull;
-				bs->set_mem(blk->map2, sizeof(blk->map2), 0xff);
-			} else {
-				blk->count = page_count;
-				unsigned i = 0;
-
-				uint64_t b1 = (page_count + 4095) / 4096;
-				blk->map1 = (1 << b1) - 1;
-
-				uint64_t b2 = (page_count + 63) / 64;
-				uint64_t b2q = b2 / 64;
-				uint64_t b2r = b2 % 64;
-				bs->set_mem(blk->map2, b2q, 0xff);
-				blk->map2[b2q] = (1 << b2r) - 1;
-				break;
-			}
-		}
-	}
-
-	uint64_t *bitmap = reinterpret_cast<uint64_t*>(next_block);
-	bs->set_mem(bitmap, total_bitmap_size, 0);
-	for (unsigned i = 0; i < block_count; ++i) {
-		frame_block &blk = blocks[i];
-		blk.bitmap = bitmap;
-
-		size_t b = blk.count / 64;
-		size_t r = blk.count % 64;
-		bs->set_mem(blk.bitmap, b*8, 0xff);
-		blk.bitmap[b] = (1 << r) - 1;
-
-		bitmap += bitmap_size(blk.count);
-	}
-
-	args->frame_block_count = block_count;
-	args->frame_block_pages = bytes_to_pages(total_size);
-	args->frame_blocks = blocks;
-}
-
-void
-fix_frame_blocks(kernel::args::header *args)
-{
-	// Map the frame blocks to the appropriate address
-	paging::map_pages(args,
-		reinterpret_cast<uintptr_t>(args->frame_blocks),
-		::memory::bitmap_start,
-		args->frame_block_pages,
-		true, false);
-
-	uintptr_t offset = ::memory::bitmap_start -
-		reinterpret_cast<uintptr_t>(args->frame_blocks);
-
-	for (unsigned i = 0; i < args->frame_block_count; ++i) {
-		frame_block &blk = args->frame_blocks[i];
-		blk.bitmap = reinterpret_cast<uint64_t*>(
-			reinterpret_cast<uintptr_t>(blk.bitmap) + offset);
-	}
 }
 
 efi_mem_map
@@ -255,7 +139,7 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 	status_line status {L"Creating kernel memory map"};
 
 	efi_mem_map map;
-	get_uefi_mappings(map, bs);
+	get_uefi_mappings(&map, false, bs);
 
 	size_t map_size = map.num_entries() * sizeof(mem_entry);
 
@@ -269,14 +153,14 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 		L"Error allocating kernel memory map module space");
 
 	bs->set_mem(kernel_map, map_size, 0);
-	get_uefi_mappings(map, bs);
+	get_uefi_mappings(&map, true, bs);
 
-	size_t nent = 0;
+	size_t i = 0;
 	bool first = true;
 	for (auto desc : map) {
 		/*
 		// EFI map dump
-		console::print(L"   eRange %lx (%lx) %x(%s) [%lu]\r\n",
+		console::print(L"   Range %lx (%lx) %x(%s) [%lu]\r\n",
 			desc->physical_start, desc->attribute, desc->type, memory_type_name(desc->type), desc->number_of_pages);
 		*/
 
@@ -292,10 +176,13 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 			case uefi::memory_type::boot_services_code:
 			case uefi::memory_type::boot_services_data:
 			case uefi::memory_type::conventional_memory:
-			case uefi::memory_type::loader_data:
 				type = mem_type::free;
 				break;
 
+			case uefi::memory_type::loader_data:
+				type = mem_type::pending;
+				break;
+
 			case uefi::memory_type::runtime_services_code:
 			case uefi::memory_type::runtime_services_data:
 				type = mem_type::uefi_runtime;
@@ -323,19 +210,18 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 		// TODO: validate uefi's map is sorted
 		if (first) {
 			first = false;
-			mem_entry &ent = kernel_map[nent++];
-			ent.start = desc->physical_start;
-			ent.pages = desc->number_of_pages;
-			ent.type = type;
-			ent.attr = (desc->attribute & 0xffffffff);
+			kernel_map[i].start = desc->physical_start;
+			kernel_map[i].pages = desc->number_of_pages;
+			kernel_map[i].type = type;
+			kernel_map[i].attr = (desc->attribute & 0xffffffff);
 			continue;
 		}
 
-		mem_entry &prev = kernel_map[nent - 1];
+		mem_entry &prev = kernel_map[i];
 		if (can_merge(prev, type, desc)) {
 			prev.pages += desc->number_of_pages;
 		} else {
-			mem_entry &next = kernel_map[nent++];
+			mem_entry &next = kernel_map[++i];
 			next.start = desc->physical_start;
 			next.pages = desc->number_of_pages;
 			next.type = type;
@@ -345,19 +231,17 @@ build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs)
 
 	// Give just the actually-set entries in the header
 	args->mem_map = kernel_map;
-	args->map_count = nent;
+	args->map_count = i;
 
 	/*
 	// kernel map dump
-	for (unsigned i = 0; i < nent; ++i) {
+	for (unsigned i = 0; i < args->map_count; ++i) {
 		const kernel::args::mem_entry &e = kernel_map[i];
-		console::print(L"   kRange %lx (%lx) %x(%s) [%lu]\r\n",
+		console::print(L"   Range %lx (%lx) %x(%s) [%lu]\r\n",
 			e.start, e.attr, e.type, kernel_memory_type_name(e.type), e.pages);
 	}
 	*/
 
-	build_kernel_frame_blocks(kernel_map, nent, args, bs);
-	get_uefi_mappings(map, bs);
 	return map;
 }
 

src/boot/memory.h

@@ -41,13 +41,12 @@ struct efi_mem_map
 	using iterator = offset_iterator<desc>;
 
 	size_t length;     ///< Total length of the map data
-	size_t total;      ///< Total allocated space for map data
 	size_t size;       ///< Size of an entry in the array
 	size_t key;        ///< Key for detecting changes
 	uint32_t version;  ///< Version of the `memory_descriptor` struct
 	desc *entries;     ///< The array of UEFI descriptors
 
-	efi_mem_map() : length(0), total(0), size(0), key(0), version(0), entries(nullptr) {}
+	efi_mem_map() : length(0), size(0), key(0), version(0), entries(nullptr) {}
 
 	/// Get the count of entries in the array
 	inline size_t num_entries() const { return length / size; }
@@ -63,14 +62,6 @@ struct efi_mem_map
 /// \returns  The uefi memory map used to build the kernel map
 efi_mem_map build_kernel_mem_map(kernel::args::header *args, uefi::boot_services *bs);
 
-/// Create the kernel frame allocation maps
-void build_kernel_frame_blocks(
-		const kernel::args::mem_entry *map, size_t nent,
-		kernel::args::header *args, uefi::boot_services *bs);
-
-/// Map the frame allocation maps to the right spot and fix up pointers
-void fix_frame_blocks(kernel::args::header *args);
-
 /// Activate the given memory mappings. Sets the given page tables live as well
 /// as informs UEFI runtime services of the new mappings.
 /// \arg pml4  The root page table for the new mappings

src/boot/paging.cpp

@@ -15,7 +15,7 @@ using memory::page_size;
 using ::memory::pml4e_kernel;
 using ::memory::table_entries;
 
-// Flags: 0 0 0 1  0 0 0 0  0 0 0 1 = 0x0101
+// Flags: 0 0 0 1  0 0 0 0  0 0 1 1 = 0x0103
 //         IGN  |  | | | |  | | | +- Present
 //              |  | | | |  | | +--- Writeable
 //              |  | | | |  | +----- Usermode access (supervisor only)
@@ -26,7 +26,7 @@ using ::memory::table_entries;
 //              |  +---------------- PAT (determining memory type for page)
 //              +------------------- Global
 /// Page table entry flags for entries pointing at a page
-constexpr uint64_t page_flags = 0x101;
+constexpr uint16_t page_flags = 0x103;
 
 // Flags: 0  0 0 0 1  1 0 0 0  1 0 1 1 = 0x018b
 //        |   IGN  |  | | | |  | | | +- Present
@@ -40,7 +40,7 @@ constexpr uint64_t page_flags = 0x101;
 //        |        +------------------- Global
 //        +---------------------------- PAT (determining memory type for page)
 /// Page table entry flags for entries pointing at a huge page
-constexpr uint64_t huge_page_flags = 0x18b;
+constexpr uint16_t huge_page_flags = 0x183;
 
 // Flags: 0 0 0 0  0 0 0 0  0 0 1 1 = 0x0003
 //        IGNORED  | | | |  | | | +- Present
@@ -52,7 +52,7 @@ constexpr uint64_t huge_page_flags = 0x18b;
 //                 | +-------------- Ignored
 //                 +---------------- Reserved 0 (Table pointer, not page)
 /// Page table entry flags for entries pointing at another table
-constexpr uint64_t table_flags = 0x003;
+constexpr uint16_t table_flags = 0x003;
 
 /// Iterator over page table entries.
 template <unsigned D = 4>
@@ -118,7 +118,7 @@ private:
 
 		if (!(parent_ent & 1)) {
 			if (!m_cache_count--)
-				error::raise(uefi::status::out_of_resources, L"Page table cache empty", 0x7ab1e5);
+				error::raise(uefi::status::out_of_resources, L"Page table cache empty");
 
 			page_table *table = reinterpret_cast<page_table*>(m_page_cache);
 			m_page_cache = offset_ptr<void>(m_page_cache, page_size);
@@ -191,7 +191,7 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
 	status_line status(L"Allocating initial page tables");
 
 	static constexpr size_t pd_tables = 256;   // number of pages for kernelspace PDs
-	static constexpr size_t extra_tables = 64; // number of extra pages
+	static constexpr size_t extra_tables = 49; // number of extra pages
 
 	// number of pages for kernelspace PDs + PML4
 	static constexpr size_t kernel_tables = pd_tables + 1;
@@ -212,7 +212,6 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
 	page_table *pml4 = reinterpret_cast<page_table*>(addr);
 
 	args->pml4 = pml4;
-	args->table_pages = tables_needed;
 	args->table_count = tables_needed - 1;
 	args->page_tables = offset_ptr<void>(addr, page_size);
 
@@ -221,42 +220,27 @@ allocate_tables(kernel::args::header *args, uefi::boot_services *bs)
 	add_kernel_pds(pml4, args->page_tables, args->table_count);
 	add_offset_mappings(pml4, args->page_tables, args->table_count);
 
-	//console::print(L"    Set up initial mappings, %d spare tables.\r\n", args->table_count);
-}
-
-template <typename E>
-constexpr bool has_flag(E set, E flag) {
-	return
-		(static_cast<uint64_t>(set) & static_cast<uint64_t>(flag)) ==
-		 static_cast<uint64_t>(flag);
+	console::print(L"    Set up initial mappings, %d spare tables.\r\n", args->table_count);
 }
 
 void
 map_pages(
 	kernel::args::header *args,
 	uintptr_t phys, uintptr_t virt,
-	size_t count, bool write_flag, bool exe_flag)
+	size_t size)
 {
-	if (!count)
-		return;
-
 	paging::page_table *pml4 =
 		reinterpret_cast<paging::page_table*>(args->pml4);
 
+	size_t pages = memory::bytes_to_pages(size);
 	page_entry_iterator<4> iterator{
 		virt, pml4,
 		args->page_tables,
 		args->table_count};
 
-	uint64_t flags = page_flags;
-	if (!exe_flag)
-		flags |= (1ull << 63); // set NX bit
-	if (write_flag)
-		flags |= 2;
-
 	while (true) {
-		*iterator = phys | flags;
-		if (--count == 0)
+		*iterator = phys | page_flags;
+		if (--pages == 0)
 			break;
 
 		iterator.increment();
@@ -264,24 +248,6 @@ map_pages(
 	}
 }
 
-void
-map_section(
-	kernel::args::header *args,
-	const kernel::args::program_section &section)
-{
-	using kernel::args::section_flags;
-
-	size_t pages = memory::bytes_to_pages(section.size);
-
-	map_pages(
-		args,
-		section.phys_addr,
-		section.virt_addr,
-		pages,
-		has_flag(section.type, section_flags::write),
-		has_flag(section.type, section_flags::execute));
-}
-
 
 } // namespace paging
 } // namespace boot

src/boot/paging.h

@@ -38,22 +38,15 @@ void allocate_tables(
 /// tables in the current PML4.
 void add_current_mappings(page_table *new_pml4);
 
-/// Map physical memory pages to virtual addresses in the given page tables.
+/// Map a physical address to a virtual address in the given page tables.
 /// \arg args  The kernel args header, used for the page table cache and pml4
-/// \arg section The program section to load
+/// \arg phys  The phyiscal address to map in
+/// \arg virt  The virtual address to map in
+/// \arg size  The size in bytes of the mapping 
 void map_pages(
 	kernel::args::header *args,
 	uintptr_t phys, uintptr_t virt,
-	size_t count, bool write_flag, bool exe_flag);
-
-/// Map a program section in physical memory to its virtual address in the
-/// given page tables.
-/// \arg args    The kernel args header, used for the page table cache and pml4
-/// \arg section The program section to load
-void map_section(
-	kernel::args::header *args,
-	const kernel::args::program_section &section);
-
+	size_t bytes);
 
 } // namespace paging
 } // namespace boot

src/boot/status.cpp

@@ -30,8 +30,7 @@ status *status::s_current = nullptr;
 unsigned status::s_current_type = 0;
 unsigned status_bar::s_count = 0;
 
-status_line::status_line(const wchar_t *message, const wchar_t *context, bool fails_clean) :
-	status(fails_clean),
+status_line::status_line(const wchar_t *message, const wchar_t *context) :
 	m_level(level_ok),
 	m_depth(0),
 	m_outer(nullptr)
@@ -149,7 +148,6 @@ status_line::do_fail(const wchar_t *message, uefi::status status)
 
 
 status_bar::status_bar(kernel::args::framebuffer const &fb) :
-	status(fb.size),
 	m_outer(nullptr)
 {
 	m_size = (fb.vertical / num_boxes) - 1;

src/boot/status.h

@@ -17,8 +17,6 @@ namespace boot {
 class status
 {
 public:
-	status(bool fails_clean = true) : m_fails_clean(fails_clean) {}
-
 	virtual void do_warn(const wchar_t *message, uefi::status status) = 0;
 	virtual void do_fail(const wchar_t *message, uefi::status status) = 0;
 
@@ -30,7 +28,7 @@ public:
 	inline static bool warn(const wchar_t *message, uefi::status status = uefi::status::success) {
 		if (!s_current) return false;
 		s_current->do_warn(message, status);
-		return s_current->m_fails_clean;
+		return true;
 	}
 
 	/// Set the state to error, and print a message. If the state is already at
@@ -41,15 +39,12 @@ public:
 	inline static bool fail(const wchar_t *message, uefi::status status) {
 		if (!s_current) return false;
 		s_current->do_fail(message, status);
-		return s_current->m_fails_clean;
+		return true;
 	}
 
 protected:
 	static status *s_current;
 	static unsigned s_current_type;
-
-private:
-	bool m_fails_clean;
 };
 
 /// Scoped status line reporter. Prints a message and an "OK" if no errors
@@ -64,11 +59,7 @@ public:
 	/// Constructor.
 	/// \arg message  Description of the operation in progress
 	/// \arg context  If non-null, printed after `message` and a colon
-	/// \arg fails_clean If true, this object can handle printing failure
-	status_line(
-		const wchar_t *message,
-		const wchar_t *context = nullptr,
-		bool fails_clean = true);
+	status_line(const wchar_t *message, const wchar_t *context = nullptr);
 	~status_line();
 
 	virtual void do_warn(const wchar_t *message, uefi::status status) override;

src/drivers/fb/main.cpp

@@ -4,11 +4,11 @@
 
 #include "j6/init.h"
 #include "j6/errors.h"
-#include "j6/flags.h"
 #include "j6/signals.h"
-#include "j6/syscalls.h"
 #include "j6/types.h"
 
+#include <j6libc/syscalls.h>
+
 #include "font.h"
 #include "screen.h"
 #include "scrollback.h"
@@ -19,7 +19,6 @@ extern "C" {
 }
 
 extern j6_handle_t __handle_sys;
-extern j6_handle_t __handle_self;
 
 struct entry
 {
@@ -33,7 +32,7 @@ struct entry
 int
 main(int argc, const char **argv)
 {
-	j6_system_log("fb driver starting");
+	_syscall_system_log("fb driver starting");
 
 	size_t initc = 0;
 	j6_init_value *initv = nullptr;
@@ -42,40 +41,20 @@ main(int argc, const char **argv)
 	j6_init_framebuffer *fb = nullptr;
 	for (unsigned i = 0; i < initc; ++i) {
 		if (initv[i].type == j6_init_desc_framebuffer) {
-			fb = reinterpret_cast<j6_init_framebuffer*>(initv[i].data);
+			fb = reinterpret_cast<j6_init_framebuffer*>(initv[i].value);
 			break;
 		}
 	}
 
-	if (!fb || fb->addr == 0) {
-		j6_system_log("fb driver didn't find a framebuffer, exiting");
+	if (!fb || fb->addr == nullptr) {
+		_syscall_system_log("fb driver didn't find a framebuffer, exiting");
 		return 1;
 	}
 
-	j6_handle_t fb_handle = j6_handle_invalid;
-	uint32_t flags =
-		j6_vm_flag_write |
-		j6_vm_flag_write_combine;
-	j6_status_t s = j6_system_map_mmio(__handle_sys, &fb_handle, fb->addr, fb->size, flags);
-	if (s != j6_status_ok) {
-		return s;
-	}
-
-	s = j6_vma_map(fb_handle, __handle_self, fb->addr);
-	if (s != j6_status_ok) {
-		return s;
-	}
-
 	const screen::pixel_order order = (fb->flags & 1) ?
 		screen::pixel_order::bgr8 : screen::pixel_order::rgb8;
 
-	screen scr(
-		reinterpret_cast<void*>(fb->addr),
-		fb->horizontal,
-		fb->vertical,
-		fb->scanline,
-		order);
-
+	screen scr(fb->addr, fb->horizontal, fb->vertical, order);
 	font fnt;
 
 	screen::pixel_t fg = scr.color(0xb0, 0xb0, 0xb0);
@@ -92,28 +71,14 @@ main(int argc, const char **argv)
 	scrollback scroll(rows, cols);
 
 	int pending = 0;
-	constexpr int pending_threshold = 5;
-
-	j6_handle_t sys = __handle_sys;
-	size_t buffer_size = 0;
-	void *message_buffer = nullptr;
+	constexpr int pending_threshold = 10;
 
+	char message_buffer[256];
 	while (true) {
-		size_t size = buffer_size;
-		j6_status_t s = j6_system_get_log(sys, message_buffer, &size);
-
-		if (s == j6_err_insufficient) {
-			free(message_buffer);
-			message_buffer = malloc(size * 2);
-			buffer_size = size;
-			continue;
-		} else if (s != j6_status_ok) {
-			j6_system_log("fb driver got error from get_log, quitting");
-			return s;
-		}
-
-		if (size > 0) {
-			entry *e = reinterpret_cast<entry*>(message_buffer);
+		size_t size = sizeof(message_buffer);
+		_syscall_system_get_log(__handle_sys, message_buffer, &size);
+		if (size != 0) {
+			entry *e = reinterpret_cast<entry*>(&message_buffer);
 
 			size_t eom = e->bytes - sizeof(entry);
 			e->message[eom] = 0;
@@ -133,7 +98,8 @@ main(int argc, const char **argv)
 		}
 	}
 
-	j6_system_log("fb driver done, exiting");
+
+	_syscall_system_log("fb driver done, exiting");
 	return 0;
 }
 

src/drivers/fb/screen.cpp

@@ -2,15 +2,13 @@
 #include <string.h>
 #include "screen.h"
 
-screen::screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order) :
-	m_fb(static_cast<volatile pixel_t *>(addr)),
+screen::screen(void *addr, unsigned hres, unsigned vres, pixel_order order) :
+	m_fb(static_cast<pixel_t *>(addr)),
 	m_order(order),
-	m_scanline(scanline),
 	m_resx(hres),
 	m_resy(vres)
 {
-	const size_t size = scanline * vres;
-	m_back = reinterpret_cast<pixel_t*>(malloc(size * sizeof(pixel_t)));
+	m_back = reinterpret_cast<pixel_t*>(malloc(hres*vres*sizeof(pixel_t)));
 }
 
 screen::pixel_t
@@ -34,15 +32,19 @@ screen::color(uint8_t r, uint8_t g, uint8_t b) const
 void
 screen::fill(pixel_t color)
 {
-	const size_t len = m_scanline * m_resy;
-	asm volatile ( "rep stosl" : :
-		"a"(color), "c"(len), "D"(m_back) );
+	const size_t len = m_resx * m_resy;
+	for (size_t i = 0; i < len; ++i)
+		m_back[i] = color;
+}
+
+void
+screen::draw_pixel(unsigned x, unsigned y, pixel_t color)
+{
+	m_back[x + y * m_resx] = color;
 }
 
 void
 screen::update()
 {
-	const size_t len = m_scanline * m_resy * sizeof(pixel_t);
-	asm volatile ( "rep movsb" : :
-		"c"(len), "S"(m_back), "D"(m_fb) );
+	memcpy(m_fb, m_back, m_resx*m_resy*sizeof(pixel_t));
 }

src/drivers/fb/screen.h

@@ -9,7 +9,7 @@ public:
 
 	enum class pixel_order : uint8_t { bgr8, rgb8, };
 
-	screen(volatile void *addr, unsigned hres, unsigned vres, unsigned scanline, pixel_order order);
+	screen(void *addr, unsigned hres, unsigned vres, pixel_order order);
 
 	unsigned width() const { return m_resx; }
 	unsigned height() const { return m_resy; }
@@ -17,19 +17,13 @@ public:
 	pixel_t color(uint8_t r, uint8_t g, uint8_t b) const;
 
 	void fill(pixel_t color);
-
-	inline void draw_pixel(unsigned x, unsigned y, pixel_t color) {
-		const size_t index = x + y * m_scanline;
-		m_back[index] = color;
-	}
+	void draw_pixel(unsigned x, unsigned y, pixel_t color);
 
 	void update();
 
 private:
-	volatile pixel_t *m_fb;
-	pixel_t *m_back;
+	pixel_t *m_fb, *m_back;
 	pixel_order m_order;
-	unsigned m_scanline;
 	unsigned m_resx, m_resy;
 
 	screen() = delete;

src/drivers/fb/scrollback.cpp

@@ -12,6 +12,10 @@ scrollback::scrollback(unsigned lines, unsigned cols, unsigned margin) :
 	m_margin {margin}
 {
 	m_data = reinterpret_cast<char*>(malloc(lines*cols));
+	m_lines = reinterpret_cast<char**>(malloc(lines*sizeof(char*)));
+	for (unsigned i = 0; i < lines; ++i)
+		m_lines[i] = &m_data[i*cols];
+
 	memset(m_data, ' ', lines*cols);
 }
 
@@ -23,17 +27,15 @@ scrollback::add_line(const char *line, size_t len)
 	if (len > m_cols)
 		len = m_cols;
 
-	char *start = m_data + (i * m_cols);
-	memcpy(start, line, len);
+	memcpy(m_lines[i], line, len);
 	if (len < m_cols)
-		memset(start + len, ' ', m_cols - len);
+		memset(m_lines[i]+len, ' ', m_cols - len);
 }
 
 char *
 scrollback::get_line(unsigned i)
 {
-	unsigned line = (i + m_count) % m_rows;
-	return &m_data[line*m_cols];
+	return m_lines[(i+m_count)%m_rows];
 }
 
 void
@@ -45,12 +47,8 @@ scrollback::render(screen &scr, font &fnt)
 	const unsigned xstride = (m_margin + fnt.width());
 	const unsigned ystride = (m_margin + fnt.height());
 
-	unsigned start = m_count <= m_rows ? 0 :
-		m_count  % m_rows;
-
 	for (unsigned y = 0; y < m_rows; ++y) {
-		unsigned i = (start + y) % m_rows;
-		char *line = &m_data[i*m_cols];
+		char *line = get_line(y);
 		for (unsigned x = 0; x < m_cols; ++x) {
 			fnt.draw_glyph(scr, line[x], fg, bg, m_margin+x*xstride, m_margin+y*ystride);
 		}

src/drivers/fb/scrollback.h

@@ -17,6 +17,7 @@ public:
 
 private:
 	char *m_data;
+	char **m_lines;
 	unsigned m_rows, m_cols;
 	unsigned m_start;
 	unsigned m_count;

src/drivers/nulldrv/main.cpp

@@ -4,7 +4,8 @@
 #include "j6/types.h"
 #include "j6/errors.h"
 #include "j6/signals.h"
-#include "j6/syscalls.h"
+
+#include <j6libc/syscalls.h>
 
 #include "io.h"
 #include "serial.h"
@@ -14,39 +15,40 @@ extern j6_handle_t __handle_sys;
 j6_handle_t endp = j6_handle_invalid;
 
 extern "C" {
+	void _init_libc(j6_process_init *);
 	int main(int, const char **);
 }
 
 void
 thread_proc()
 {
-	j6_system_log("sub thread starting");
+	_syscall_system_log("sub thread starting");
 
 	char buffer[512];
 	size_t len = sizeof(buffer);
 	j6_tag_t tag = 0;
-	j6_status_t result = j6_endpoint_receive(endp, &tag, &len, (void*)buffer);
+	j6_status_t result = _syscall_endpoint_receive(endp, &tag, &len, (void*)buffer);
 	if (result != j6_status_ok)
-		j6_thread_exit(result);
+		_syscall_thread_exit(result);
 
-	j6_system_log("sub thread received message");
+	_syscall_system_log("sub thread received message");
 
 	for (int i = 0; i < len; ++i)
 		if (buffer[i] >= 'A' && buffer[i] <= 'Z')
 			buffer[i] += 0x20;
 
 	tag++;
-	result = j6_endpoint_send(endp, tag, len, (void*)buffer);
+	result = _syscall_endpoint_send(endp, tag, len, (void*)buffer);
 	if (result != j6_status_ok)
-		j6_thread_exit(result);
+		_syscall_thread_exit(result);
 
-	j6_system_log("sub thread sent message");
+	_syscall_system_log("sub thread sent message");
 
 	for (int i = 1; i < 5; ++i)
-		j6_thread_sleep(i*10);
+		_syscall_thread_sleep(i*10);
 
-	j6_system_log("sub thread exiting");
-	j6_thread_exit(0);
+	_syscall_system_log("sub thread exiting");
+	_syscall_thread_exit(0);
 }
 
 int
@@ -55,10 +57,10 @@ main(int argc, const char **argv)
 	j6_handle_t child = j6_handle_invalid;
 	j6_signal_t out = 0;
 
-	j6_system_log("main thread starting");
+	_syscall_system_log("main thread starting");
 
 	for (int i = 0; i < argc; ++i)
-		j6_system_log(argv[i]);
+		_syscall_system_log(argv[i]);
 
 	void *base = malloc(0x1000);
 	if (!base)
@@ -68,48 +70,43 @@ main(int argc, const char **argv)
 	for (int i = 0; i < 3; ++i)
 		vma_ptr[i*100] = uint64_t(i);
 
-	j6_system_log("main thread wrote to memory area");
+	_syscall_system_log("main thread wrote to memory area");
 
-	j6_status_t result = j6_endpoint_create(&endp);
+	j6_status_t result = _syscall_endpoint_create(&endp);
 	if (result != j6_status_ok)
 		return result;
 
-	j6_system_log("main thread created endpoint");
+	_syscall_system_log("main thread created endpoint");
 
-	result = j6_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
+	result = _syscall_thread_create(reinterpret_cast<void*>(&thread_proc), &child);
 	if (result != j6_status_ok)
 		return result;
 
-	j6_system_log("main thread created sub thread");
+	_syscall_system_log("main thread created sub thread");
 
 	char message[] = "MAIN THREAD SUCCESSFULLY CALLED SENDRECV IF THIS IS LOWERCASE";
 	size_t size = sizeof(message);
 	j6_tag_t tag = 16;
-	result = j6_endpoint_sendrecv(endp, &tag, &size, (void*)message);
+	result = _syscall_endpoint_sendrecv(endp, &tag, &size, (void*)message);
 	if (result != j6_status_ok)
 		return result;
 
 	if (tag != 17)
-		j6_system_log("GOT WRONG TAG FROM SENDRECV");
+		_syscall_system_log("GOT WRONG TAG FROM SENDRECV");
 
-	result = j6_system_bind_irq(__handle_sys, endp, 3);
+	result = _syscall_system_bind_irq(__handle_sys, endp, 3);
 	if (result != j6_status_ok)
 		return result;
 
-	j6_system_log(message);
+	_syscall_system_log(message);
 
-	j6_system_log("main thread waiting on child");
-	result = j6_object_wait(child, -1ull, &out);
+	_syscall_system_log("main thread waiting on child");
+
+	result = _syscall_object_wait(child, -1ull, &out);
 	if (result != j6_status_ok)
 		return result;
 
-	j6_system_log("main thread creating a new process");
-	j6_handle_t child_proc = j6_handle_invalid;
-	result = j6_process_create(&child_proc);
-	if (result != j6_status_ok)
-		return result;
-
-	j6_system_log("main testing irqs");
+	_syscall_system_log("main testing irqs");
 
 
 	serial_port com2(COM2);
@@ -123,21 +120,21 @@ main(int argc, const char **argv)
 
 	size_t len = 0;
 	while (true) {
-		result = j6_endpoint_receive(endp, &tag, &len, nullptr);
+		result = _syscall_endpoint_receive(endp, &tag, &len, nullptr);
 		if (result != j6_status_ok)
 			return result;
 
 		if (j6_tag_is_irq(tag))
-			j6_system_log("main thread got irq!");
+			_syscall_system_log("main thread got irq!");
 	}
 
-	j6_system_log("main thread closing endpoint");
+	_syscall_system_log("main thread closing endpoint");
 
-	result = j6_object_close(endp);
+	result = _syscall_object_close(endp);
 	if (result != j6_status_ok)
 		return result;
 
-	j6_system_log("main thread done, exiting");
+	_syscall_system_log("main thread done, exiting");
 	return 0;
 }
 

src/include/j6/flags.h

@@ -1,10 +0,0 @@
-#pragma once
-/// \file flags.h
-/// Enums used as flags for syscalls
-
-enum j6_vm_flags {
-#define VM_FLAG(name, v) j6_vm_flag_ ## name = v,
-#include "j6/tables/vm_flags.inc"
-#undef VM_FLAG
-	j6_vm_flags_MAX
-};

src/include/j6/init.h

@@ -3,32 +3,25 @@
 /// Types used in process and thread initialization
 
 #include <stdint.h>
-#include "j6/types.h"
 
 enum j6_init_type {					// `value` is a:
-	j6_init_handle_self,			// Handle to the system
-	j6_init_handle_other,			// Handle to this process
+	j6_init_handle_system,			// Handle to the system
+	j6_init_handle_process,			// Handle to this process
+	j6_init_handle_thread,			// Handle to this thread
+	j6_init_handle_space,			// Handle to this process' address space
 	j6_init_desc_framebuffer		// Pointer to a j6_init_framebuffer descriptor
 };
 
-struct j6_typed_handle {
-	enum j6_object_type type;
-	j6_handle_t handle;
-};
-
 struct j6_init_value {
 	enum j6_init_type type;
-	union {
-		struct j6_typed_handle handle;
-		void *data;
-	};
+	uint64_t value;
 };
 
 /// Structure defining a framebuffer.
 /// `flags` has the following bits:
 ///   0-3:   Pixel layout. 0000: rgb8, 0001: bgr8
 struct j6_init_framebuffer {
-	uintptr_t addr;
+	void* addr;
 	size_t size;
 	uint32_t vertical;
 	uint32_t horizontal;

src/include/j6/signals.h

@@ -8,42 +8,9 @@
 
 // Signals 16-47 are defined per-object-type
 
-// Event signals
-#define j7_signal_event00           (1ull << 16)
-#define j7_signal_event01           (1ull << 17)
-#define j7_signal_event02           (1ull << 18)
-#define j7_signal_event03           (1ull << 19)
-#define j7_signal_event04           (1ull << 20)
-#define j7_signal_event05           (1ull << 21)
-#define j7_signal_event06           (1ull << 22)
-#define j7_signal_event07           (1ull << 23)
-#define j7_signal_event08           (1ull << 24)
-#define j7_signal_event09           (1ull << 25)
-#define j7_signal_event10           (1ull << 26)
-#define j7_signal_event11           (1ull << 27)
-#define j7_signal_event12           (1ull << 28)
-#define j7_signal_event13           (1ull << 29)
-#define j7_signal_event14           (1ull << 30)
-#define j7_signal_event15           (1ull << 31)
-#define j7_signal_event16           (1ull << 32)
-#define j7_signal_event17           (1ull << 33)
-#define j7_signal_event18           (1ull << 34)
-#define j7_signal_event19           (1ull << 35)
-#define j7_signal_event20           (1ull << 36)
-#define j7_signal_event21           (1ull << 37)
-#define j7_signal_event22           (1ull << 38)
-#define j7_signal_event23           (1ull << 39)
-#define j7_signal_event24           (1ull << 40)
-#define j7_signal_event25           (1ull << 41)
-#define j7_signal_event26           (1ull << 42)
-#define j7_signal_event27           (1ull << 43)
-#define j7_signal_event28           (1ull << 44)
-#define j7_signal_event29           (1ull << 45)
-#define j7_signal_event30           (1ull << 46)
-#define j7_signal_event31           (1ull << 47)
+// Process signals
 
-// System signals
-#define j6_signal_system_has_log	(1ull << 16)
+// Thread signals
 
 // Channel signals
 #define j6_signal_channel_can_send	(1ull << 16)

src/include/j6/tables/object_types.inc

@@ -1,12 +0,0 @@
-OBJECT_TYPE( none,     0x00 )
-
-OBJECT_TYPE( system,   0x01 )
-
-OBJECT_TYPE( event,    0x02 )
-OBJECT_TYPE( channel,  0x03 )
-OBJECT_TYPE( endpoint, 0x04 )
-
-OBJECT_TYPE( vma,      0x05 )
-
-OBJECT_TYPE( process,  0x06 )
-OBJECT_TYPE( thread,   0x07 )

src/include/j6/tables/vm_flags.inc

@@ -1,9 +0,0 @@
-VM_FLAG( none,            0x00000000)
-VM_FLAG( write,           0x00000001)
-VM_FLAG( exec,            0x00000002)
-VM_FLAG( zero,            0x00000010)
-VM_FLAG( contiguous,      0x00000020)
-VM_FLAG( large_pages,     0x00000100)
-VM_FLAG( huge_pages,      0x00000200)
-VM_FLAG( write_combine,   0x00001000)
-VM_FLAG( mmio,            0x00010000)

src/include/j6/types.h

@@ -36,10 +36,15 @@ typedef uint64_t j6_handle_t;
 #define j6_handle_id_mask 0xffffffffull
 #define j6_handle_invalid 0xffffffffull
 
-enum j6_object_type {
-#define OBJECT_TYPE( name, val ) j6_object_type_ ## name = val,
-#include "j6/tables/object_types.inc"
-#undef OBJECT_TYPE
-
-	j6_object_type_max
+/// A process' initial data structure for communicating with the system
+struct j6_process_init
+{
+	j6_handle_t process;
+	j6_handle_t handles[3];
+};
+
+/// A thread's initial data structure
+struct j6_thread_init
+{
+	j6_handle_t thread;
 };

src/include/kernel_args.h

@@ -35,26 +35,11 @@ struct framebuffer {
 	fb_type type;
 };
 
-enum class section_flags : uint32_t {
-	none    = 0,
-	execute = 1,
-	write   = 2,
-	read    = 4,
-};
-
-struct program_section {
+struct program {
 	uintptr_t phys_addr;
 	uintptr_t virt_addr;
-	uint32_t size;
-	section_flags type;
-};
-
-struct program {
 	uintptr_t entrypoint;
-	uintptr_t base;
-	size_t total_size;
-	size_t num_sections;
-	program_section sections[3];
+	size_t size;
 };
 
 enum class mem_type : uint32_t {
@@ -75,18 +60,6 @@ struct mem_entry
 	uint32_t attr;
 };
 
-constexpr size_t frames_per_block = 64 * 64 * 64;
-
-struct frame_block
-{
-	uintptr_t base;
-	uint32_t count;
-	uint32_t attrs;
-	uint64_t map1;
-	uint64_t map2[64];
-	uint64_t *bitmap;
-};
-
 enum class boot_flags : uint16_t {
 	none  = 0x0000,
 	debug = 0x0001
@@ -100,7 +73,6 @@ struct header {
 	void *pml4;
 	void *page_tables;
 	size_t table_count;
-	size_t table_pages;
 
 	program *programs;
 	size_t num_programs;
@@ -111,10 +83,6 @@ struct header {
 	mem_entry *mem_map;
 	size_t map_count;
 
-	frame_block *frame_blocks;
-	size_t frame_block_count;
-	size_t frame_block_pages;
-
 	void *runtime_services;
 	void *acpi_table;
 

src/include/kernel_memory.h

@@ -17,7 +17,7 @@ namespace memory {
 	constexpr uintptr_t page_offset = 0xffffc00000000000ull;
 
 	/// Max number of pages for a kernel stack
-	constexpr unsigned kernel_stack_pages = 2;
+	constexpr unsigned kernel_stack_pages = 4;
 
 	/// Max number of pages for a kernel buffer
 	constexpr unsigned kernel_buffer_pages = 16;
@@ -35,17 +35,11 @@ namespace memory {
 	constexpr uintptr_t stacks_start = heap_start - kernel_max_stacks;
 
 	/// Max size of kernel buffers area
-	constexpr size_t kernel_max_buffers = 0x8000000000ull; // 512GiB
+	constexpr size_t kernel_max_buffers = 0x10000000000ull; // 1TiB
 
 	/// Start of kernel buffers
 	constexpr uintptr_t buffers_start = stacks_start - kernel_max_buffers;
 
-	/// Max size of kernel bitmap area
-	constexpr size_t kernel_max_bitmap = 0x8000000000ull; // 512GiB
-
-	/// Start of kernel bitmap
-	constexpr uintptr_t bitmap_start = buffers_start - kernel_max_bitmap;
-
 	/// First kernel space PML4 entry
 	constexpr unsigned pml4e_kernel = 256;
 
@@ -64,11 +58,6 @@ namespace memory {
 		return reinterpret_cast<T*>(a|page_offset);
 	}
 
-	/// Convert a physical address to a virtual one (in the offset-mapped area)
-	template <typename T> T * to_virtual(T *p) {
-		return to_virtual<T>(reinterpret_cast<uintptr_t>(p));
-	}
-
 	/// Get the number of pages needed for a given number of bytes.
 	/// \arg bytes  The number of bytes desired
 	/// \returns    The number of pages needed to contain the desired bytes

src/include/log_areas.inc

@@ -1,14 +1,13 @@
 LOG(apic,   info);
 LOG(device, debug);
-LOG(paging, info);
+LOG(paging, warn);
 LOG(driver, info);
-LOG(memory, debug);
+LOG(memory, info);
 LOG(fs,     info);
-LOG(task,   info);
-LOG(sched,  info);
-LOG(loader, debug);
+LOG(task,   debug);
+LOG(loader, info);
 LOG(boot,   debug);
-LOG(syscall,info);
+LOG(syscall,debug);
 LOG(vmem,   debug);
 LOG(objs,   debug);
 LOG(timer,  debug);

src/include/syscalls.inc

@@ -1,21 +1,17 @@
 SYSCALL(0x00, system_log,        const char *)
 SYSCALL(0x01, system_noop,       void)
-SYSCALL(0x02, system_get_log,    j6_handle_t, void *, size_t *)
+SYSCALL(0x02, system_get_log,    j6_handle_t, char *, size_t *)
 SYSCALL(0x03, system_bind_irq,   j6_handle_t, j6_handle_t, unsigned)
-SYSCALL(0x04, system_map_mmio,   j6_handle_t, j6_handle_t *, uintptr_t, size_t, uint32_t)
 
 SYSCALL(0x08, object_koid,       j6_handle_t, j6_koid_t *)
 SYSCALL(0x09, object_wait,       j6_handle_t, j6_signal_t, j6_signal_t *)
 SYSCALL(0x0a, object_signal,     j6_handle_t, j6_signal_t)
 SYSCALL(0x0b, object_close,      j6_handle_t)
 
-SYSCALL(0x10, process_create,    j6_handle_t *)
-SYSCALL(0x11, process_start,     j6_handle_t, uintptr_t, j6_handle_t *, size_t)
-SYSCALL(0x11, process_kill,      j6_handle_t)
-SYSCALL(0x17, process_exit,      int32_t)
+SYSCALL(0x10, process_exit,      int64_t)
 
 SYSCALL(0x18, thread_create,     void *, j6_handle_t *)
-SYSCALL(0x19, thread_exit,       int32_t)
+SYSCALL(0x19, thread_exit,       int64_t)
 SYSCALL(0x1a, thread_pause,      void)
 SYSCALL(0x1b, thread_sleep,      uint64_t)
 
@@ -30,6 +26,6 @@ SYSCALL(0x2b, endpoint_sendrecv, j6_handle_t, j6_tag_t *, size_t *, void *)
 
 SYSCALL(0x30, vma_create,        j6_handle_t *, size_t, uint32_t)
 SYSCALL(0x31, vma_create_map,    j6_handle_t *, size_t, uintptr_t, uint32_t)
-SYSCALL(0x32, vma_map,           j6_handle_t, j6_handle_t, uintptr_t)
-SYSCALL(0x33, vma_unmap,         j6_handle_t, j6_handle_t)
+SYSCALL(0x32, vma_map,           j6_handle_t, uintptr_t)
+SYSCALL(0x33, vma_unmap,         j6_handle_t)
 SYSCALL(0x34, vma_resize,        j6_handle_t, size_t *)

src/kernel/acpi_tables.h

@@ -23,7 +23,7 @@ struct acpi_table_header
 } __attribute__ ((packed));
 
 #define TABLE_HEADER(signature) \
-	static constexpr uint32_t type_id = kutil::byteswap(signature); \
+	static const uint32_t type_id = kutil::byteswap(signature); \
 	acpi_table_header header;
 
 
@@ -198,14 +198,3 @@ struct acpi_hpet
 	uint8_t attributes;
 } __attribute__ ((packed));
 
-struct acpi_bgrt
-{
-	TABLE_HEADER('BGRT');
-	uint16_t version;
-	uint8_t status;
-	uint8_t type;
-	uintptr_t address;
-	uint32_t offset_x;
-	uint32_t offset_y;
-} __attribute__ ((packed));
-

src/kernel/ap_startup.s

@@ -1,149 +0,0 @@
-%include "tasking.inc"
-
-section .ap_startup
-
-BASE equ 0x8000 ; Where the kernel will map this at runtime
-
-CR0_PE  equ (1 << 0)
-CR0_MP  equ (1 << 1)
-CR0_ET  equ (1 << 4)
-CR0_NE  equ (1 << 5)
-CR0_WP  equ (1 << 16)
-CR0_PG  equ (1 << 31)
-CR0_VAL equ CR0_PE|CR0_MP|CR0_ET|CR0_NE|CR0_WP|CR0_PG
-
-CR4_DE         equ (1 << 3)
-CR4_PAE        equ (1 << 5)
-CR4_MCE        equ (1 << 6)
-CR4_PGE        equ (1 << 7)
-CR4_OSFXSR     equ (1 << 9)
-CR4_OSCMMEXCPT equ (1 << 10)
-CR4_FSGSBASE   equ (1 << 16)
-CR4_PCIDE      equ (1 << 17)
-CR4_INIT equ CR4_PAE|CR4_PGE
-CR4_VAL equ CR4_DE|CR4_PAE|CR4_MCE|CR4_PGE|CR4_OSFXSR|CR4_OSCMMEXCPT|CR4_FSGSBASE|CR4_PCIDE
-
-EFER_MSR  equ 0xC0000080
-EFER_SCE  equ (1 << 0)
-EFER_LME  equ (1 << 8)
-EFER_NXE  equ (1 << 11)
-EFER_VAL equ EFER_SCE|EFER_LME|EFER_NXE
-
-bits 16
-default rel
-align 8
-
-global ap_startup
-ap_startup:
-	jmp .start_real
-
-align 8
-	.pml4:  dq 0
-	.cpu:   dq 0
-	.ret:   dq 0
-
-align 16
-.gdt:
-	dq 0x0        ; Null GDT entry
-
-	dq 0x00209A0000000000 ; Code
-	dq 0x0000920000000000 ; Data
-
-align 4
-.gdtd:
-	dw ($ - .gdt)
-	dd BASE + (.gdt - ap_startup)
-
-align 4
-.idtd:
-	dw 0 ; zero-length IDT descriptor
-	dd 0
-
-.start_real:
-	cli
-	cld
-
-	xor ax, ax
-	mov ds, ax
-
-	; set the temporary null IDT
-	lidt [BASE + (.idtd - ap_startup)]
-
-	; Enter long mode
-	mov eax, cr4
-	or eax, CR4_INIT
-	mov cr4, eax
-
-	mov eax, [BASE + (.pml4 - ap_startup)]
-	mov cr3, eax
-
-	mov ecx, EFER_MSR
-	rdmsr
-	or eax, EFER_VAL
-	wrmsr
-
-	mov eax, CR0_VAL
-	mov cr0, eax
-
-	; Set the temporary minimal GDT
-	lgdt [BASE + (.gdtd - ap_startup)]
-
-	jmp (1 << 3):(BASE + (.start_long - ap_startup))
-
-bits 64
-default abs
-align 8
-.start_long:
-	; set data segments
-	mov ax, (2 << 3)
-	mov ds, ax
-	mov es, ax
-	mov fs, ax
-	mov gs, ax
-	mov ss, ax
-
-	mov eax, CR4_VAL
-
-	mov rdi, [BASE + (.cpu - ap_startup)]
-	mov rax, [rdi + CPU_DATA.rsp0]
-	mov rsp, rax
-
-	mov rax, [BASE + (.ret - ap_startup)]
-	jmp rax
-
-
-global ap_startup_code_size
-ap_startup_code_size:
-	dq ($ - ap_startup)
-
-
-section .text
-global init_ap_trampoline
-init_ap_trampoline:
-	push rbp
-	mov rbp, rsp
-
-	; rdi is the kernel pml4
-	mov [BASE + (ap_startup.pml4 - ap_startup)], rdi
-
-	; rsi is the cpu data for this AP
-	mov [BASE + (ap_startup.cpu - ap_startup)], rsi
-
-	; rdx is the address to jump to
-	mov [BASE + (ap_startup.ret - ap_startup)], rdx
-
-	; rcx is the processor id
-	mov rdi, rdx
-
-	pop rbp
-	ret
-
-extern long_ap_startup
-global ap_idle
-ap_idle:
-	call long_ap_startup
-	sti
-.hang:
-	hlt
-	jmp .hang
-

src/kernel/apic.cpp

@@ -6,18 +6,11 @@
 #include "kernel_memory.h"
 #include "log.h"
 
-uint64_t lapic::s_ticks_per_us = 0;
-
-static constexpr uint16_t lapic_id         = 0x0020;
 static constexpr uint16_t lapic_spurious   = 0x00f0;
 
-static constexpr uint16_t lapic_icr_low    = 0x0300;
-static constexpr uint16_t lapic_icr_high   = 0x0310;
-
 static constexpr uint16_t lapic_lvt_timer  = 0x0320;
 static constexpr uint16_t lapic_lvt_lint0  = 0x0350;
 static constexpr uint16_t lapic_lvt_lint1  = 0x0360;
-static constexpr uint16_t lapic_lvt_error  = 0x0370;
 
 static constexpr uint16_t lapic_timer_init = 0x0380;
 static constexpr uint16_t lapic_timer_cur  = 0x0390;
@@ -32,7 +25,6 @@ apic_read(uint32_t volatile *apic, uint16_t offset)
 static void
 apic_write(uint32_t volatile *apic, uint16_t offset, uint32_t value)
 {
-	log::debug(logs::apic, "LAPIC write: %x = %08lx", offset, value);
 	*(apic + offset/sizeof(uint32_t)) = value;
 }
 
@@ -56,58 +48,14 @@ apic::apic(uintptr_t base) :
 }
 
 
-lapic::lapic(uintptr_t base) :
+lapic::lapic(uintptr_t base, isr spurious) :
 	apic(base),
 	m_divisor(0)
 {
-	apic_write(m_base, lapic_lvt_error, static_cast<uint32_t>(isr::isrAPICError));
-	apic_write(m_base, lapic_spurious, static_cast<uint32_t>(isr::isrSpurious));
+	apic_write(m_base, lapic_spurious, static_cast<uint32_t>(spurious));
 	log::info(logs::apic, "LAPIC created, base %lx", m_base);
 }
 
-uint8_t
-lapic::get_id()
-{
-	return static_cast<uint8_t>(apic_read(m_base, lapic_id) >> 24);
-}
-
-void
-lapic::send_ipi(ipi mode, uint8_t vector, uint8_t dest)
-{
-	// Wait until the APIC is ready to send
-	ipi_wait();
-
-	uint32_t command =
-		static_cast<uint32_t>(vector) |
-		static_cast<uint32_t>(mode);
-
-	apic_write(m_base, lapic_icr_high, static_cast<uint32_t>(dest) << 24);
-	apic_write(m_base, lapic_icr_low, command);
-}
-
-void
-lapic::send_ipi_broadcast(ipi mode, bool self, uint8_t vector)
-{
-	// Wait until the APIC is ready to send
-	ipi_wait();
-
-	uint32_t command =
-		static_cast<uint32_t>(vector) |
-		static_cast<uint32_t>(mode) |
-		(self ? 0 : (1 << 18)) |
-		(1 << 19);
-
-	apic_write(m_base, lapic_icr_high, 0);
-	apic_write(m_base, lapic_icr_low, command);
-}
-
-void
-lapic::ipi_wait()
-{
-	while (apic_read(m_base, lapic_icr_low) & (1<<12))
-		asm volatile ("pause" : : : "memory");
-}
-
 void
 lapic::calibrate_timer()
 {
@@ -120,14 +68,14 @@ lapic::calibrate_timer()
 	set_divisor(1);
 	apic_write(m_base, lapic_timer_init, initial);
 
-	uint64_t us = 20000;
+	uint64_t us = 200000;
 	clock::get().spinwait(us);
 
 	uint32_t remaining = apic_read(m_base, lapic_timer_cur);
-	uint64_t ticks_total = initial - remaining;
-	s_ticks_per_us = ticks_total / us;
+	uint32_t ticks_total = initial - remaining;
+	m_ticks_per_us = ticks_total / us;
 
-	log::info(logs::apic, "APIC timer ticks %d times per microsecond.", s_ticks_per_us);
+	log::info(logs::apic, "APIC timer ticks %d times per microsecond.", m_ticks_per_us);
 
 	interrupts_enable();
 }
@@ -147,7 +95,7 @@ lapic::set_divisor(uint8_t divisor)
 	case  64: divbits = 0x9; break;
 	case 128: divbits = 0xa; break;
 	default:
-		kassert(0, "Invalid divisor passed to lapic::set_divisor");
+		kassert(0, "Invalid divisor passed to lapic::enable_timer");
 	}
 
 	apic_write(m_base, lapic_timer_div, divbits);

src/kernel/apic.h

@@ -3,7 +3,6 @@
 /// Classes to control both local and I/O APICs.
 
 #include <stdint.h>
-#include "kutil/enum_bitfields.h"
 
 enum class isr : uint8_t;
 
@@ -19,22 +18,6 @@ protected:
 	uint32_t *m_base;
 };
 
-enum class ipi : uint32_t
-{
-	// Delivery modes
-	fixed    = 0x0000,
-	smi      = 0x0200,
-	nmi      = 0x0400,
-	init     = 0x0500,
-	startup  = 0x0600,
-
-	// Flags
-	deassert = 0x0000,
-	assert   = 0x4000,
-	edge     = 0x0000, ///< edge-triggered
-	level    = 0x8000, ///< level-triggered
-};
-IS_BITFIELD(ipi);
 
 /// Controller for processor-local APICs
 class lapic :
@@ -43,26 +26,8 @@ class lapic :
 public:
 	/// Constructor
 	/// \arg base      Physicl base address of the APIC's MMIO registers
-	lapic(uintptr_t base);
-
-	/// Get the local APIC's ID
-	uint8_t get_id();
-
-	/// Send an inter-processor interrupt.
-	/// \arg mode   The sending mode
-	/// \arg vector The interrupt vector
-	/// \arg dest   The APIC ID of the destination
-	void send_ipi(ipi mode, uint8_t vector, uint8_t dest);
-
-	/// Send an inter-processor broadcast interrupt to all other CPUs
-	/// \arg mode   The sending mode
-	/// \arg self   If true, include this CPU in the broadcast
-	/// \arg vector The interrupt vector
-	void send_ipi_broadcast(ipi mode, bool self, uint8_t vector);
-
-	/// Wait for an IPI to finish sending. This is done automatically
-	/// before sending another IPI with send_ipi().
-	void ipi_wait();
+	/// \arg spurious  Vector of the spurious interrupt handler
+	lapic(uintptr_t base, isr spurious);
 
 	/// Enable interrupts for the LAPIC timer.
 	/// \arg vector   Interrupt vector the timer should use
@@ -92,14 +57,19 @@ public:
 	void calibrate_timer();
 
 private:
-	inline static uint64_t ticks_to_us(uint64_t ticks)    { return ticks / s_ticks_per_us; }
-	inline static uint64_t us_to_ticks(uint64_t interval) { return interval * s_ticks_per_us; }
+	inline uint64_t ticks_to_us(uint32_t ticks) const {
+		return static_cast<uint64_t>(ticks) / m_ticks_per_us;
+	}
+
+	inline uint64_t us_to_ticks(uint64_t interval) const {
+		return interval * m_ticks_per_us;
+	}
 
 	void set_divisor(uint8_t divisor);
 	void set_repeat(bool repeat);
 
 	uint32_t m_divisor;
-	static uint64_t s_ticks_per_us;
+	uint32_t m_ticks_per_us;
 };
 
 

src/kernel/clock.cpp

@@ -16,7 +16,7 @@ clock::clock(uint64_t rate, clock::source source_func, void *data) :
 void
 clock::spinwait(uint64_t us) const
 {
-	uint64_t when = value() + us;
-	while (value() < when) asm ("pause");
+	uint64_t when = m_source(m_data) + us;
+	while (value() < when);
 }
 

src/kernel/cpu.cpp

@@ -2,18 +2,10 @@
 #include "kutil/assert.h"
 #include "kutil/memory.h"
 #include "cpu.h"
-#include "cpu/cpu_id.h"
-#include "device_manager.h"
-#include "gdt.h"
-#include "idt.h"
-#include "kernel_memory.h"
+#include "cpu/cpu.h"
 #include "log.h"
-#include "msr.h"
-#include "objects/vm_area.h"
-#include "syscall.h"
-#include "tss.h"
 
-cpu_data g_bsp_cpu_data;
+cpu_data bsp_cpu_data;
 
 void
 cpu_validate()
@@ -37,30 +29,3 @@ cpu_validate()
 #undef CPU_FEATURE_OPT
 #undef CPU_FEATURE_REQ
 }
-
-void
-cpu_early_init(cpu_data *cpu)
-{
-	IDT::get().install();
-	cpu->gdt->install();
-
-	// Install the GS base pointint to the cpu_data
-	wrmsr(msr::ia32_gs_base, reinterpret_cast<uintptr_t>(cpu));
-}
-
-void
-cpu_init(cpu_data *cpu, bool bsp)
-{
-	if (!bsp) {
-		// The BSP already called cpu_early_init
-		cpu_early_init(cpu);
-	}
-
-	// Set up the syscall MSRs
-	syscall_enable();
-
-	// Set up the page attributes table
-	uint64_t pat = rdmsr(msr::ia32_pat);
-	pat = (pat & 0x00ffffffffffffffull) | (0x01ull << 56); // set PAT 7 to WC
-	wrmsr(msr::ia32_pat, pat);
-}

src/kernel/cpu.h

@@ -2,12 +2,9 @@
 
 #include <stdint.h>
 
-class GDT;
-class lapic;
-class process;
 struct TCB;
 class thread;
-class TSS;
+class process;
 
 struct cpu_state
 {
@@ -21,39 +18,15 @@ struct cpu_state
 /// version in 'tasking.inc'
 struct cpu_data
 {
-	cpu_data *self;
-	uint16_t id;
-	uint16_t index;
-	uint32_t reserved;
 	uintptr_t rsp0;
 	uintptr_t rsp3;
 	TCB *tcb;
-	thread *thread;
-	process *process;
-	TSS *tss;
-	GDT *gdt;
-
-	// Members beyond this point do not appear in
-	// the assembly version
-	lapic *apic;
+	thread *t;
+	process *p;
 };
 
-extern "C" cpu_data * _current_gsbase();
+extern cpu_data bsp_cpu_data;
 
-/// Set up the running CPU. This sets GDT, IDT, and necessary MSRs as well as creating
-/// the cpu_data structure for this processor.
-/// \arg cpu  The cpu_data structure for this CPU
-/// \arg bsp  True if this CPU is the BSP
-void cpu_init(cpu_data *cpu, bool bsp);
-
-/// Do early (before cpu_init) initialization work. Only needs to be called manually for
-/// the BSP, otherwise cpu_init will call it.
-/// \arg cpu  The cpu_data structure for this CPU
-void cpu_early_init(cpu_data *cpu);
-
-/// Get the cpu_data struct for the current executing CPU
-inline cpu_data & current_cpu() { return *_current_gsbase(); }
-
-/// Validate the required CPU features are present. Really, the bootloader already
-/// validated the required features, but still iterate the options and log about them.
+// We already validated the required options in the bootloader,
+// but iterate the options and log about them.
 void cpu_validate();

src/kernel/debug.cpp

@@ -13,7 +13,6 @@ void
 print_regs(const cpu_state &regs)
 {
 	console *cons = console::get();
-	cpu_data &cpu = current_cpu();
 
 	uint64_t cr2 = 0;
 	__asm__ __volatile__ ("mov %%cr2, %0" : "=r"(cr2));
@@ -21,8 +20,8 @@ print_regs(const cpu_state &regs)
 	uintptr_t cr3 = 0;
 	__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (cr3) );
 
-	cons->printf("       process: %llx", cpu.process->koid());
-	cons->printf("   thread: %llx\n", cpu.thread->koid());
+	cons->printf("       process: %llx", bsp_cpu_data.p->koid());
+	cons->printf("   thread: %llx\n", bsp_cpu_data.t->koid());
 
 	print_regL("rax", regs.rax);
 	print_regM("rbx", regs.rbx);
@@ -44,7 +43,7 @@ print_regs(const cpu_state &regs)
 	cons->puts("\n\n");
 	print_regL("rbp", regs.rbp);
 	print_regM("rsp", regs.user_rsp);
-	print_regR("sp0", cpu.rsp0);
+	print_regR("sp0", bsp_cpu_data.rsp0);
 
 	print_regL("rip", regs.rip);
 	print_regM("cr3", cr3);

src/kernel/debug.h

@@ -4,8 +4,6 @@
 
 #include <stdint.h>
 
-struct cpu_state;
-
 extern "C" {
 	uintptr_t get_rsp();
 	uintptr_t get_rip();

src/kernel/device_manager.cpp

@@ -38,7 +38,7 @@ struct acpi2_rsdp
 	uint32_t rsdt_address;
 
 	uint32_t length;
-	acpi_table_header *xsdt_address;
+	uint64_t xsdt_address;
 	uint8_t checksum20;
 	uint8_t reserved[3];
 } __attribute__ ((packed));
@@ -63,7 +63,7 @@ void irq4_callback(void *)
 
 
 device_manager::device_manager() :
-	m_lapic_base(0)
+	m_lapic(0)
 {
 	m_irqs.ensure_capacity(32);
 	m_irqs.set_size(16);
@@ -73,20 +73,13 @@ device_manager::device_manager() :
 	m_irqs[2] = ignore_endpoint;
 }
 
-template <typename T> static const T *
-check_get_table(const acpi_table_header *header)
-{
-	kassert(header && header->validate(T::type_id), "Invalid ACPI table.");
-	return reinterpret_cast<const T *>(header);
-}
-
 void
 device_manager::parse_acpi(const void *root_table)
 {
 	kassert(root_table != 0, "ACPI root table pointer is null.");
 
-	const acpi1_rsdp *acpi1 = memory::to_virtual(
-		reinterpret_cast<const acpi1_rsdp *>(root_table));
+	const acpi1_rsdp *acpi1 =
+		reinterpret_cast<const acpi1_rsdp *>(root_table);
 
 	for (int i = 0; i < sizeof(acpi1->signature); ++i)
 		kassert(acpi1->signature[i] == expected_signature[i],
@@ -103,27 +96,7 @@ device_manager::parse_acpi(const void *root_table)
 	sum = kutil::checksum(acpi2, sizeof(acpi2_rsdp), sizeof(acpi1_rsdp));
 	kassert(sum == 0, "ACPI 2.0 RSDP checksum mismatch.");
 
-	load_xsdt(memory::to_virtual(acpi2->xsdt_address));
-}
-
-const device_manager::apic_nmi *
-device_manager::get_lapic_nmi(uint8_t id) const
-{
-	for (const auto &nmi : m_nmis) {
-		if (nmi.cpu == 0xff || nmi.cpu == id)
-			return &nmi;
-	}
-
-	return nullptr;
-}
-
-const device_manager::irq_override *
-device_manager::get_irq_override(uint8_t irq) const
-{
-	for (const auto &o : m_overrides)
-		if (o.source == irq) return &o;
-
-	return nullptr;
+	load_xsdt(reinterpret_cast<const acpi_xsdt *>(acpi2->xsdt_address));
 }
 
 ioapic *
@@ -139,9 +112,9 @@ put_sig(char *into, uint32_t type)
 }
 
 void
-device_manager::load_xsdt(const acpi_table_header *header)
+device_manager::load_xsdt(const acpi_xsdt *xsdt)
 {
-	const auto *xsdt = check_get_table<acpi_xsdt>(header);
+	kassert(xsdt && acpi_validate(xsdt), "Invalid ACPI XSDT.");
 
 	char sig[5] = {0,0,0,0,0};
 	log::info(logs::device, "ACPI 2.0+ tables loading");
@@ -151,8 +124,7 @@ device_manager::load_xsdt(const acpi_table_header *header)
 
 	size_t num_tables = acpi_table_entries(xsdt, sizeof(void*));
 	for (size_t i = 0; i < num_tables; ++i) {
-		const acpi_table_header *header =
-			memory::to_virtual(xsdt->headers[i]);
+		const acpi_table_header *header = xsdt->headers[i];
 
 		put_sig(sig, header->type);
 		log::debug(logs::device, "  Found table %s", sig);
@@ -161,15 +133,15 @@ device_manager::load_xsdt(const acpi_table_header *header)
 
 		switch (header->type) {
 		case acpi_apic::type_id:
-			load_apic(header);
+			load_apic(reinterpret_cast<const acpi_apic *>(header));
 			break;
 
 		case acpi_mcfg::type_id:
-			load_mcfg(header);
+			load_mcfg(reinterpret_cast<const acpi_mcfg *>(header));
 			break;
 
 		case acpi_hpet::type_id:
-			load_hpet(header);
+			load_hpet(reinterpret_cast<const acpi_hpet *>(header));
 			break;
 
 		default:
@@ -179,42 +151,40 @@ device_manager::load_xsdt(const acpi_table_header *header)
 }
 
 void
-device_manager::load_apic(const acpi_table_header *header)
+device_manager::load_apic(const acpi_apic *apic)
 {
-	const auto *apic = check_get_table<acpi_apic>(header);
-
-	m_lapic_base = apic->local_address;
+	uintptr_t local = apic->local_address;
+	m_lapic = new lapic(local, isr::isrSpurious);
 
 	size_t count = acpi_table_entries(apic, 1);
 	uint8_t const *p = apic->controller_data;
 	uint8_t const *end = p + count;
 
-	// Pass one: count objcts
-	unsigned num_lapics = 0;
-	unsigned num_ioapics = 0;
-	unsigned num_overrides = 0;
-	unsigned num_nmis = 0;
+	// Pass one: count IOAPIC objcts
+	int num_ioapics = 0;
 	while (p < end) {
 		const uint8_t type = p[0];
 		const uint8_t length = p[1];
-
-		switch (type) {
-		case 0: ++num_lapics; break;
-		case 1: ++num_ioapics; break;
-		case 2: ++num_overrides; break;
-		case 4: ++num_nmis; break;
-		default: break;
-		}
-
+		if (type == 1) num_ioapics++;
 		p += length;
 	}
 
-	m_apic_ids.set_capacity(num_lapics);
 	m_ioapics.set_capacity(num_ioapics);
-	m_overrides.set_capacity(num_overrides);
-	m_nmis.set_capacity(num_nmis);
 
-	// Pass two: configure objects
+	// Pass two: set up IOAPIC objcts
+	p = apic->controller_data;
+	while (p < end) {
+		const uint8_t type = p[0];
+		const uint8_t length = p[1];
+		if (type == 1) {
+			uintptr_t base = kutil::read_from<uint32_t>(p+4);
+			uint32_t base_gsr = kutil::read_from<uint32_t>(p+8);
+			m_ioapics.emplace(base, base_gsr);
+		}
+		p += length;
+	}
+
+	// Pass three: configure APIC objects
 	p = apic->controller_data;
 	while (p < end) {
 		const uint8_t type = p[0];
@@ -224,42 +194,38 @@ device_manager::load_apic(const acpi_table_header *header)
 		case 0: { // Local APIC
 				uint8_t uid = kutil::read_from<uint8_t>(p+2);
 				uint8_t id = kutil::read_from<uint8_t>(p+3);
-				m_apic_ids.append(id);
-
-				log::debug(logs::device, "    Local APIC uid %x id %x", uid, id);
+				log::debug(logs::device, "    Local APIC uid %x id %x", id);
 			}
 			break;
 
-		case 1: { // I/O APIC
-				uintptr_t base = kutil::read_from<uint32_t>(p+4);
-				uint32_t base_gsi = kutil::read_from<uint32_t>(p+8);
-				m_ioapics.emplace(base, base_gsi);
-
-				log::debug(logs::device, "    IO APIC gsi %x base %x", base_gsi, base);
-			}
+		case 1: // I/O APIC
 			break;
 
 		case 2: { // Interrupt source override
-				irq_override o;
-				o.source = kutil::read_from<uint8_t>(p+3);
-				o.gsi = kutil::read_from<uint32_t>(p+4);
-				o.flags = kutil::read_from<uint16_t>(p+8);
-				m_overrides.append(o);
+				uint8_t source = kutil::read_from<uint8_t>(p+3);
+				isr gsi = isr::irq00 + kutil::read_from<uint32_t>(p+4);
+				uint16_t flags = kutil::read_from<uint16_t>(p+8);
 
 				log::debug(logs::device, "    Intr source override IRQ %d -> %d Pol %d Tri %d",
-						o.source, o.gsi, (o.flags & 0x3), ((o.flags >> 2) & 0x3));
+						source, gsi, (flags & 0x3), ((flags >> 2) & 0x3));
+
+				// TODO: in a multiple-IOAPIC system this might be elsewhere
+				m_ioapics[0].redirect(source, static_cast<isr>(gsi), flags, true);
 			}
 			break;
 
 		case 4: {// LAPIC NMI
-			apic_nmi nmi;
-			nmi.cpu = kutil::read_from<uint8_t>(p + 2);
-			nmi.lint = kutil::read_from<uint8_t>(p + 5);
-			nmi.flags = kutil::read_from<uint16_t>(p + 3);
-			m_nmis.append(nmi);
+			uint8_t cpu = kutil::read_from<uint8_t>(p + 2);
+			uint8_t num = kutil::read_from<uint8_t>(p + 5);
+			uint16_t flags = kutil::read_from<uint16_t>(p + 3);
 
-			log::debug(logs::device, "    LAPIC NMI Proc %02x LINT%d Pol %d Tri %d",
-					nmi.cpu, nmi.lint, nmi.flags & 0x3, (nmi.flags >> 2) & 0x3);
+			log::debug(logs::device, "    LAPIC NMI Proc %d LINT%d Pol %d Tri %d",
+					kutil::read_from<uint8_t>(p+2),
+					kutil::read_from<uint8_t>(p+5),
+					kutil::read_from<uint16_t>(p+3) & 0x3,
+					(kutil::read_from<uint16_t>(p+3) >> 2) & 0x3);
+
+			m_lapic->enable_lint(num, num == 0 ? isr::isrLINT0 : isr::isrLINT1, true, flags);
 			}
 			break;
 
@@ -269,13 +235,20 @@ device_manager::load_apic(const acpi_table_header *header)
 
 		p += length;
 	}
+
+	for (uint8_t i = 0; i < m_ioapics[0].get_num_gsi(); ++i) {
+		switch (i) {
+			case 2: break;
+			default: m_ioapics[0].mask(i, false);
+		}
+	}
+
+	m_lapic->enable();
 }
 
 void
-device_manager::load_mcfg(const acpi_table_header *header)
+device_manager::load_mcfg(const acpi_mcfg *mcfg)
 {
-	const auto *mcfg = check_get_table<acpi_mcfg>(header);
-
 	size_t count = acpi_table_entries(mcfg, sizeof(acpi_mcfg_entry));
 	m_pci.set_size(count);
 	m_devices.set_capacity(16);
@@ -296,10 +269,8 @@ device_manager::load_mcfg(const acpi_table_header *header)
 }
 
 void
-device_manager::load_hpet(const acpi_table_header *header)
+device_manager::load_hpet(const acpi_hpet *hpet)
 {
-	const auto *hpet = check_get_table<acpi_hpet>(header);
-
 	log::debug(logs::device, "  Found HPET device #%3d: base %016lx  pmin %d  attr %02x",
 			hpet->index, hpet->base_address.address, hpet->periodic_min, hpet->attributes);
 
@@ -340,13 +311,6 @@ device_manager::probe_pci()
 	}
 }
 
-static uint64_t
-fake_clock_source(void*)
-{
-	static uint64_t value = 0;
-	return value++;
-}
-
 void
 device_manager::init_drivers()
 {
@@ -365,20 +329,18 @@ device_manager::init_drivers()
 		ahcid.register_device(&device);
 	}
 	*/
-	clock *master_clock = nullptr;
 	if (m_hpets.count() > 0) {
 		hpet &h = m_hpets[0];
 		h.enable();
 
 		// becomes the singleton
-		master_clock = new clock(h.rate(), hpet_clock_source, &h);
+		clock *master_clock = new clock(h.rate(), hpet_clock_source, &h);
+		kassert(master_clock, "Failed to allocate master clock");
 		log::info(logs::clock, "Created master clock using HPET 0: Rate %d", h.rate());
 	} else {
-		//TODO: Other clocks, APIC clock?
-		master_clock = new clock(5000, fake_clock_source, nullptr);
+		//TODO: APIC clock?
+		kassert(0, "No HPET master clock");
 	}
-
-	kassert(master_clock, "Failed to allocate master clock");
 }
 
 bool

src/kernel/device_manager.h

@@ -6,7 +6,10 @@
 #include "hpet.h"
 #include "pci.h"
 
-struct acpi_table_header;
+struct acpi_xsdt;
+struct acpi_apic;
+struct acpi_mcfg;
+struct acpi_hpet;
 class block_device;
 class endpoint;
 
@@ -24,6 +27,10 @@ public:
 	/// \returns  A reference to the system device manager
 	static device_manager & get() { return s_instance; }
 
+	/// Get the LAPIC
+	/// \returns An object representing the local APIC
+	lapic * get_lapic() { return m_lapic; }
+
 	/// Get an IOAPIC
 	/// \arg i   Index of the requested IOAPIC
 	/// \returns An object representing the given IOAPIC if it exists,
@@ -64,39 +71,6 @@ public:
 	/// \returns  True if the interrupt was handled
 	bool dispatch_irq(unsigned irq);
 
-	struct apic_nmi
-	{
-		uint8_t cpu;
-		uint8_t lint;
-		uint16_t flags;
-	};
-
-	struct irq_override
-	{
-		uint8_t source;
-		uint16_t flags;
-		uint32_t gsi;
-	};
-
-	/// Get the list of APIC ids for other CPUs
-	inline const kutil::vector<uint8_t> & get_apic_ids() const { return m_apic_ids; }
-
-	/// Get the LAPIC base address
-	/// \returns The physical base address of the local apic registers
-	uintptr_t get_lapic_base() const { return m_lapic_base; }
-
-	/// Get the NMI mapping for the given local APIC
-	/// \arg id  ID of the local APIC
-	/// \returns apic_nmi structure describing the NMI configuration,
-	///          or null if no configuration was provided
-	const apic_nmi * get_lapic_nmi(uint8_t id) const;
-
-	/// Get the IRQ source override for the given IRQ
-	/// \arg irq  IRQ number (not isr vector)
-	/// \returns  irq_override structure describing that IRQ's
-	///           configuration, or null if no configuration was provided
-	const irq_override * get_irq_override(uint8_t irq) const;
-
 	/// Register the existance of a block device.
 	/// \arg blockdev  Pointer to the block device
 	void register_block_device(block_device *blockdev);
@@ -126,19 +100,19 @@ public:
 private:
 	/// Parse the ACPI XSDT and load relevant sub-tables.
 	/// \arg xsdt  Pointer to the XSDT from the firmware
-	void load_xsdt(const acpi_table_header *xsdt);
+	void load_xsdt(const acpi_xsdt *xsdt);
 
 	/// Parse the ACPI MADT and initialize APICs from it.
 	/// \arg apic  Pointer to the MADT from the XSDT
-	void load_apic(const acpi_table_header *apic);
+	void load_apic(const acpi_apic *apic);
 
 	/// Parse the ACPI MCFG and initialize PCIe from it.
 	/// \arg mcfg  Pointer to the MCFG from the XSDT
-	void load_mcfg(const acpi_table_header *mcfg);
+	void load_mcfg(const acpi_mcfg *mcfg);
 
 	/// Parse the ACPI HPET and initialize an HPET from it.
 	/// \arg hpet  Pointer to the HPET from the XSDT
-	void load_hpet(const acpi_table_header *hpet);
+	void load_hpet(const acpi_hpet *hpet);
 
 	/// Probe the PCIe busses and add found devices to our
 	/// device list. The device list is destroyed and rebuilt.
@@ -148,13 +122,9 @@ private:
 	/// that has no callback.
 	void bad_irq(uint8_t irq);
 
-	uintptr_t m_lapic_base;
-
+	lapic *m_lapic;
 	kutil::vector<ioapic> m_ioapics;
 	kutil::vector<hpet> m_hpets;
-	kutil::vector<uint8_t> m_apic_ids;
-	kutil::vector<apic_nmi> m_nmis;
-	kutil::vector<irq_override> m_overrides;
 
 	kutil::vector<pci_group> m_pci;
 	kutil::vector<pci_device> m_devices;

src/kernel/frame_allocator.cpp

@@ -1,12 +1,21 @@
+#include "kernel_memory.h"
 #include "kutil/assert.h"
 #include "kutil/memory.h"
-
 #include "frame_allocator.h"
-#include "kernel_args.h"
-#include "kernel_memory.h"
-#include "log.h"
 
 using memory::frame_size;
+using memory::page_offset;
+using frame_block_node = kutil::list_node<frame_block>;
+
+int
+frame_block::compare(const frame_block &rhs) const
+{
+	if (address < rhs.address)
+		return -1;
+	else if (address > rhs.address)
+		return 1;
+	return 0;
+}
 
 
 frame_allocator &
@@ -16,148 +25,54 @@ frame_allocator::get()
 	return g_frame_allocator;
 }
 
-frame_allocator::frame_allocator(kernel::args::frame_block *frames, size_t count) :
-	m_blocks {frames},
-	m_count {count}
-{
-}
-
-inline unsigned
-bsf(uint64_t v)
-{
-	asm ("tzcntq %q0, %q1" : "=r"(v) : "0"(v) : "cc");
-	return v;
-}
+frame_allocator::frame_allocator() {}
 
 size_t
 frame_allocator::allocate(size_t count, uintptr_t *address)
 {
-	kutil::scoped_lock lock {m_lock};
-
-	for (long i = m_count - 1; i >= 0; --i) {
-		frame_block &block = m_blocks[i];
-
-		if (!block.map1)
-			continue;
-
-		// Tree walk to find the first available page
-		unsigned o1 = bsf(block.map1);
-
-		uint64_t m2 = block.map2[o1];
-		unsigned o2 = bsf(m2);
-
-		uint64_t m3 = block.bitmap[(o1 << 6) + o2];
-		unsigned o3 = bsf(m3);
-
-		unsigned frame = (o1 << 12) + (o2 << 6) + o3;
-
-		// See how many contiguous pages are here
-		unsigned n = bsf(~m3 >> o3);
-		if (n > count)
-			n = count;
-
-		*address = block.base + frame * frame_size;
-
-		// Clear the bits to mark these pages allocated
-		m3 &= ~(((1 << n) - 1) << o3);
-		block.bitmap[(o1 << 6) + o2] = m3;
-		if (!m3) {
-			// if that was it for this group, clear the next level bit
-			m2 &= ~(1 << o2);
-			block.map2[o1] = m2;
-
-			if (!m2) {
-				// if that was cleared too, update the top level
-				block.map1 &= ~(1 << o1);
-			}
-		}
-
-		return n;
-	}
-
-	kassert(false, "frame_allocator ran out of free frames!");
+	kassert(!m_free.empty(), "frame_allocator::pop_frames ran out of free frames!");
+	if (m_free.empty())
 		return 0;
+
+	auto *first = m_free.front();
+
+	if (count >= first->count) {
+		*address = first->address;
+		m_free.remove(first);
+		return first->count;
+	} else {
+		first->count -= count;
+		*address = first->address + (first->count * frame_size);
+		return count;
 	}
+}
+
+inline uintptr_t end(frame_block *node) { return node->address + node->count * frame_size; }
 
 void
 frame_allocator::free(uintptr_t address, size_t count)
 {
-	kutil::scoped_lock lock {m_lock};
-
 	kassert(address % frame_size == 0, "Trying to free a non page-aligned frame!");
 
-	if (!count)
-		return;
+	frame_block_node *node =
+		reinterpret_cast<frame_block_node*>(address + page_offset);
 
-	for (long i = 0; i < m_count; ++i) {
-		frame_block &block = m_blocks[i];
-		uintptr_t end = block.base + block.count * frame_size;
+	kutil::memset(node, 0, sizeof(frame_block_node));
+	node->address = address;
+	node->count = count;
 
-		if (address < block.base || address >= end)
-			continue;
+	m_free.sorted_insert(node);
 
-		uint64_t frame = (address - block.base) >> 12;
-		unsigned o1 = (frame >> 12) & 0x3f;
-		unsigned o2 = (frame >> 6) & 0x3f;
-		unsigned o3 = frame & 0x3f;
-
-		while (count--) {
-			block.map1 |= (1 << o1);
-			block.map2[o1] |= (1 << o2);
-			block.bitmap[o2] |= (1 << o3);
-			if (++o3 == 64) {
-				o3 = 0;
-				if (++o2 == 64) {
-					o2 = 0;
-					++o1;
-					kassert(o1 < 64, "Tried to free pages past the end of a block");
-				}
-			}
-		}
-	}
+	frame_block_node *next = node->next();
+	if (next && end(node) == next->address) {
+		node->count += next->count;
+		m_free.remove(next);
 	}
 
-void
-frame_allocator::used(uintptr_t address, size_t count)
-{
-	kutil::scoped_lock lock {m_lock};
-
-	kassert(address % frame_size == 0, "Trying to mark a non page-aligned frame!");
-
-	if (!count)
-		return;
-
-	for (long i = 0; i < m_count; ++i) {
-		frame_block &block = m_blocks[i];
-		uintptr_t end = block.base + block.count * frame_size;
-
-		if (address < block.base || address >= end)
-			continue;
-
-		uint64_t frame = (address - block.base) >> 12;
-		unsigned o1 = (frame >> 12) & 0x3f;
-		unsigned o2 = (frame >> 6) & 0x3f;
-		unsigned o3 = frame & 0x3f;
-
-		while (count--) {
-			block.bitmap[o2] &= ~(1 << o3);
-			if (!block.bitmap[o2]) {
-				block.map2[o1] &= ~(1 << o2);
-
-				if (!block.map2[o1]) {
-					block.map1 &= ~(1 << o1);
-				}
-			}
-
-			if (++o3 == 64) {
-				o3 = 0;
-				if (++o2 == 64) {
-					o2 = 0;
-					++o1;
-					kassert(o1 < 64, "Tried to mark pages past the end of a block");
-				}
-			}
-		}
+	frame_block_node *prev = node->prev();
+	if (prev && end(prev) == address) {
+		prev->count += node->count;
+		m_free.remove(node);
 	}
 }
 

src/kernel/frame_allocator.h

@@ -3,23 +3,18 @@
 /// Allocator for physical memory frames
 
 #include <stdint.h>
-#include "kutil/spinlock.h"
 
-namespace kernel {
-namespace args {
+#include "kutil/linked_list.h"
+
 struct frame_block;
-}}
+using frame_block_list = kutil::linked_list<frame_block>;
 
 /// Allocator for physical memory frames
 class frame_allocator
 {
 public:
-	using frame_block = kernel::args::frame_block;
-
-	/// Constructor
-	/// \arg blocks The bootloader-supplied frame bitmap block list
-	/// \arg count  Number of entries in the block list
-	frame_allocator(frame_block *frames, size_t count);
+	/// Default constructor
+	frame_allocator();
 
 	/// Get free frames from the free list. Only frames from the first free block
 	/// are returned, so the number may be less than requested, but they will
@@ -34,20 +29,26 @@ public:
 	/// \arg count    The number of frames to be freed
 	void free(uintptr_t address, size_t count);
 
-	/// Mark frames as used
-	/// \arg address  The physical address of the first frame to free
-	/// \arg count    The number of frames to be freed
-	void used(uintptr_t address, size_t count);
-
 	/// Get the global frame allocator
 	static frame_allocator & get();
 
 private:
-	frame_block *m_blocks;
-	size_t m_count;
+	frame_block_list m_free; ///< Free frames list
 
-	kutil::spinlock m_lock;
-
-	frame_allocator() = delete;
 	frame_allocator(const frame_allocator &) = delete;
 };
+
+
+/// A block of contiguous frames. Each `frame_block` represents contiguous
+/// physical frames with the same attributes.
+struct frame_block
+{
+	uintptr_t address;
+	uint32_t count;
+
+	/// Compare two blocks by address.
+	/// \arg rhs   The right-hand comparator
+	/// \returns   <0 if this is sorts earlier, >0 if this sorts later, 0 for equal
+	int compare(const frame_block &rhs) const;
+};
+

src/kernel/gdt.cpp

@@ -1,80 +1,35 @@
 #include <stdint.h>
 
 #include "kutil/assert.h"
+#include "kutil/enum_bitfields.h"
 #include "kutil/memory.h"
-#include "kutil/no_construct.h"
 #include "console.h"
-#include "cpu.h"
-#include "gdt.h"
 #include "log.h"
-#include "tss.h"
-
-extern "C" void gdt_write(const void *gdt_ptr, uint16_t cs, uint16_t ds, uint16_t tr);
-
-static constexpr uint8_t kern_cs_index   = 1;
-static constexpr uint8_t kern_ss_index   = 2;
-static constexpr uint8_t user_cs32_index = 3;
-static constexpr uint8_t user_ss_index   = 4;
-static constexpr uint8_t user_cs64_index = 5;
-static constexpr uint8_t tss_index       = 6; // Note that this takes TWO GDT entries
-
-// The BSP's GDT is initialized _before_ global constructors are called,
-// so we don't want it to have a global constructor, lest it overwrite
-// the previous initialization.
-static kutil::no_construct<GDT> __g_bsp_gdt_storage;
-GDT &g_bsp_gdt = __g_bsp_gdt_storage.value;
 
 
-GDT::GDT(TSS *tss) :
-	m_tss(tss)
+enum class gdt_type : uint8_t
 {
-	kutil::memset(this, 0, sizeof(GDT));
+	accessed	= 0x01,
+	read_write	= 0x02,
+	conforming	= 0x04,
+	execute		= 0x08,
+	system		= 0x10,
+	ring1		= 0x20,
+	ring2		= 0x40,
+	ring3		= 0x60,
+	present		= 0x80
+};
+IS_BITFIELD(gdt_type);
 
-	m_ptr.limit = sizeof(m_entries) - 1;
-	m_ptr.base = &m_entries[0];
-
-	// Kernel CS/SS - always 64bit
-	set(kern_cs_index, 0, 0xfffff, true, gdt_type::read_write | gdt_type::execute);
-	set(kern_ss_index, 0, 0xfffff, true, gdt_type::read_write);
-
-	// User CS32/SS/CS64 - layout expected by SYSRET
-	set(user_cs32_index, 0, 0xfffff, false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
-	set(user_ss_index,   0, 0xfffff, true,  gdt_type::ring3 | gdt_type::read_write);
-	set(user_cs64_index, 0, 0xfffff, true,  gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
-
-	set_tss(tss);
-}
-
-GDT &
-GDT::current()
+struct gdt_descriptor
 {
-	cpu_data &cpu = current_cpu();
-	return *cpu.gdt;
-}
-
-void
-GDT::install() const
-{
-	gdt_write(
-		static_cast<const void*>(&m_ptr),
-		kern_cs_index << 3,
-		kern_ss_index << 3,
-		tss_index << 3);
-}
-
-void
-GDT::set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
-{
-	m_entries[i].limit_low = limit & 0xffff;
-	m_entries[i].size = (limit >> 16) & 0xf;
-	m_entries[i].size |= (is64 ? 0xa0 : 0xc0);
-
-	m_entries[i].base_low = base & 0xffff;
-	m_entries[i].base_mid = (base >> 16) & 0xff;
-	m_entries[i].base_high = (base >> 24) & 0xff;
-
-	m_entries[i].type = type | gdt_type::system | gdt_type::present;
-}
+	uint16_t limit_low;
+	uint16_t base_low;
+	uint8_t base_mid;
+	gdt_type type;
+	uint8_t size;
+	uint8_t base_high;
+} __attribute__ ((packed));
 
 struct tss_descriptor
 {
@@ -88,16 +43,72 @@ struct tss_descriptor
 	uint32_t reserved;
 } __attribute__ ((packed));
 
+struct tss_entry
+{
+	uint32_t reserved0;
+
+	uint64_t rsp[3]; // stack pointers for CPL 0-2
+	uint64_t ist[8]; // ist[0] is reserved
+
+	uint64_t reserved1;
+	uint16_t reserved2;
+	uint16_t iomap_offset;
+} __attribute__ ((packed));
+
+struct idt_descriptor
+{
+	uint16_t base_low;
+	uint16_t selector;
+	uint8_t ist;
+	uint8_t flags;
+	uint16_t base_mid;
+	uint32_t base_high;
+	uint32_t reserved;		// must be zero
+} __attribute__ ((packed));
+
+struct table_ptr
+{
+	uint16_t limit;
+	uint64_t base;
+} __attribute__ ((packed));
+
+
+gdt_descriptor g_gdt_table[10];
+idt_descriptor g_idt_table[256];
+table_ptr g_gdtr;
+table_ptr g_idtr;
+tss_entry g_tss;
+
+
+extern "C" {
+	void idt_write();
+	void idt_load();
+
+	void gdt_write(uint16_t cs, uint16_t ds, uint16_t tr);
+	void gdt_load();
+}
+
 void
-GDT::set_tss(TSS *tss)
+gdt_set_entry(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
+{
+	g_gdt_table[i].limit_low = limit & 0xffff;
+	g_gdt_table[i].size = (limit >> 16) & 0xf;
+	g_gdt_table[i].size |= (is64 ? 0xa0 : 0xc0);
+
+	g_gdt_table[i].base_low = base & 0xffff;
+	g_gdt_table[i].base_mid = (base >> 16) & 0xff;
+	g_gdt_table[i].base_high = (base >> 24) & 0xff;
+
+	g_gdt_table[i].type = type | gdt_type::system | gdt_type::present;
+}
+
+void
+tss_set_entry(uint8_t i, uint64_t base, uint64_t limit)
 {
 	tss_descriptor tssd;
-
-	size_t limit = sizeof(TSS);
 	tssd.limit_low = limit & 0xffff;
 	tssd.size = (limit >> 16) & 0xf;
 
-	uintptr_t base = reinterpret_cast<uintptr_t>(tss);
 	tssd.base_00 = base & 0xffff;
 	tssd.base_16 = (base >> 16) & 0xff;
 	tssd.base_24 = (base >> 24) & 0xff;
@@ -109,26 +120,87 @@ GDT::set_tss(TSS *tss)
 		gdt_type::execute |
 		gdt_type::ring3 |
 		gdt_type::present;
-
-	kutil::memcpy(&m_entries[tss_index], &tssd, sizeof(tss_descriptor));
+	kutil::memcpy(&g_gdt_table[i], &tssd, sizeof(tss_descriptor));
 }
 
 void
-GDT::dump(unsigned index) const
+idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags)
 {
+	g_idt_table[i].base_low = addr & 0xffff;
+	g_idt_table[i].base_mid = (addr >> 16) & 0xffff;
+	g_idt_table[i].base_high = (addr >> 32) & 0xffffffff;
+	g_idt_table[i].selector = selector;
+	g_idt_table[i].flags = flags;
+	g_idt_table[i].ist = 0;
+	g_idt_table[i].reserved = 0;
+}
+
+void
+tss_set_stack(int ring, uintptr_t rsp)
+{
+	kassert(ring < 3, "Bad ring passed to tss_set_stack.");
+	g_tss.rsp[ring] = rsp;
+}
+
+uintptr_t
+tss_get_stack(int ring)
+{
+	kassert(ring < 3, "Bad ring passed to tss_get_stack.");
+	return g_tss.rsp[ring];
+}
+
+void
+gdt_init()
+{
+	kutil::memset(&g_gdt_table, 0, sizeof(g_gdt_table));
+	kutil::memset(&g_idt_table, 0, sizeof(g_idt_table));
+
+	g_gdtr.limit = sizeof(g_gdt_table) - 1;
+	g_gdtr.base = reinterpret_cast<uint64_t>(&g_gdt_table);
+
+	// Kernel CS/SS - always 64bit
+	gdt_set_entry(1, 0, 0xfffff,  true, gdt_type::read_write | gdt_type::execute);
+	gdt_set_entry(2, 0, 0xfffff,  true, gdt_type::read_write);
+
+	// User CS32/SS/CS64 - layout expected by SYSRET
+	gdt_set_entry(3, 0, 0xfffff,  false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+	gdt_set_entry(4, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write);
+	gdt_set_entry(5, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+
+	kutil::memset(&g_tss, 0, sizeof(tss_entry));
+	g_tss.iomap_offset = sizeof(tss_entry);
+
+	uintptr_t tss_base = reinterpret_cast<uintptr_t>(&g_tss);
+
+	// Note that this takes TWO GDT entries
+	tss_set_entry(6, tss_base, sizeof(tss_entry));
+
+	gdt_write(1 << 3, 2 << 3, 6 << 3);
+
+	g_idtr.limit = sizeof(g_idt_table) - 1;
+	g_idtr.base = reinterpret_cast<uint64_t>(&g_idt_table);
+
+	idt_write();
+}
+
+void
+gdt_dump(int index)
+{
+	const table_ptr &table = g_gdtr;
+
 	console *cons = console::get();
 
-	unsigned start = 0;
-	unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
+	int start = 0;
+	int count = (table.limit + 1) / sizeof(gdt_descriptor);
 	if (index != -1) {
 		start = index;
 		count = 1;
 	} else {
-		cons->printf("         GDT: loc:%lx size:%d\n", m_ptr.base, m_ptr.limit+1);
+		cons->printf("         GDT: loc:%lx size:%d\n", table.base, table.limit+1);
 	}
 
-	const descriptor *gdt =
-		reinterpret_cast<const descriptor *>(m_ptr.base);
+	const gdt_descriptor *gdt =
+		reinterpret_cast<const gdt_descriptor *>(table.base);
 
 	for (int i = start; i < start+count; ++i) {
 		uint32_t base =
@@ -166,3 +238,51 @@ GDT::dump(unsigned index) const
 				(gdt[i].size & 0x60) == 0x40 ? "32" : "16");
 	}
 }
+
+void
+idt_dump(int index)
+{
+	const table_ptr &table = g_idtr;
+
+
+	int start = 0;
+	int count = (table.limit + 1) / sizeof(idt_descriptor);
+	if (index != -1) {
+		start = index;
+		count = 1;
+		log::info(logs::boot, "IDT FOR INDEX %02x", index);
+	} else {
+		log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", table.base, table.limit+1);
+	}
+
+	const idt_descriptor *idt =
+		reinterpret_cast<const idt_descriptor *>(table.base);
+
+	for (int i = start; i < start+count; ++i) {
+		uint64_t base =
+			(static_cast<uint64_t>(idt[i].base_high) << 32) |
+			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
+			idt[i].base_low;
+
+		char const *type;
+		switch (idt[i].flags & 0xf) {
+			case 0x5: type = " 32tsk "; break;
+			case 0x6: type = " 16int "; break;
+			case 0x7: type = " 16trp "; break;
+			case 0xe: type = " 32int "; break;
+			case 0xf: type = " 32trp "; break;
+			default:  type = " ????? "; break;
+		}
+
+		if (idt[i].flags & 0x80) {
+			log::debug(logs::boot,
+					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
+					(idt[i].selector & 0x3),
+					((idt[i].selector & 0x4) >> 2),
+					(idt[i].selector >> 3),
+					idt[i].ist,
+					type,
+					((idt[i].flags >> 5) & 0x3));
+		}
+	}
+}

src/kernel/gdt.h

@@ -1,66 +1,33 @@
 #pragma once
 /// \file gdt.h
-/// Definitions relating to a CPU's GDT table
+/// Definitions relating to system descriptor tables: GDT, IDT, TSS
 #include <stdint.h>
 
-#include "kutil/enum_bitfields.h"
+/// Set up the GDT and TSS, and switch segment registers to point
+/// to them.
+void gdt_init();
 
-class TSS;
+/// Set an entry in the IDT
+/// \arg i         Index in the IDT (vector of the interrupt this handles)
+/// \arg addr      Address of the handler
+/// \arg selector  GDT selector to set when invoking this handler
+/// \arg flags     Descriptor flags to set
+void idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags);
 
-enum class gdt_type : uint8_t
-{
-	accessed	= 0x01,
-	read_write	= 0x02,
-	conforming	= 0x04,
-	execute		= 0x08,
-	system		= 0x10,
-	ring1		= 0x20,
-	ring2		= 0x40,
-	ring3		= 0x60,
-	present		= 0x80
-};
-IS_BITFIELD(gdt_type);
+/// Set the stack pointer for a given ring in the TSS
+/// \arg ring  Ring to set for (0-2)
+/// \arg rsp   Stack pointer to set
+void tss_set_stack(int ring, uintptr_t rsp);
 
-class GDT
-{
-public:
-	GDT(TSS *tss);
+/// Get the stack pointer for a given ring in the TSS
+/// \arg ring  Ring to get (0-2)
+/// \returns   Stack pointers for that ring
+uintptr_t tss_get_stack(int ring);
 
-	/// Get the currently running CPU's GDT
-	static GDT & current();
-
-	/// Install this GDT to the current CPU
-	void install() const;
-
-	/// Get the addrss of the pointer
-	inline const void * pointer() const { return static_cast<const void*>(&m_ptr); }
-
-	/// Dump debug information about the GDT to the console.
+/// Dump information about the current GDT to the screen
 /// \arg index  Which entry to print, or -1 for all entries
-	void dump(unsigned index = -1) const;
+void gdt_dump(int index = -1);
 
-private:
-	void set(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type);
-	void set_tss(TSS *tss);
-
-	struct descriptor
-	{
-		uint16_t limit_low;
-		uint16_t base_low;
-		uint8_t base_mid;
-		gdt_type type;
-		uint8_t size;
-		uint8_t base_high;
-	} __attribute__ ((packed, align(8)));
-
-	struct ptr
-	{
-		uint16_t limit;
-		descriptor *base;
-	} __attribute__ ((packed, align(4)));
-
-	descriptor m_entries[8];
-	TSS *m_tss;
-
-	ptr m_ptr;
-};
+/// Dump information about the current IDT to the screen
+/// \arg index  Which entry to print, or -1 for all entries
+void idt_dump(int index = -1);

src/kernel/gdt.s

@@ -0,0 +1,35 @@
+extern g_idtr
+extern g_gdtr
+
+global idt_write
+idt_write:
+	lidt [rel g_idtr]
+	ret
+
+global idt_load
+idt_load:
+	sidt [rel g_idtr]
+	ret
+
+global gdt_write
+gdt_write:
+	lgdt [rel g_gdtr]
+	mov ax, si ; second arg is data segment
+	mov ds, ax
+	mov es, ax
+	mov fs, ax
+	mov gs, ax
+	mov ss, ax
+	push qword rdi ; first arg is code segment
+	lea rax, [rel .next]
+	push rax
+	o64 retf
+.next:
+	ltr dx ; third arg is the TSS
+	ret
+
+global gdt_load
+gdt_load:
+	sgdt [rel g_gdtr]
+	ret
+

src/kernel/gdtidt.s

@@ -1,35 +0,0 @@
-
-global idt_write
-idt_write:
-	lidt [rdi] ; first arg is the IDT pointer location
-	ret
-
-global idt_load
-idt_load:
-	sidt [rdi] ; first arg is where to write the idtr value
-	ret
-
-global gdt_write
-gdt_write:
-	lgdt [rdi] ; first arg is the GDT pointer location
-
-	mov ax, dx ; third arg is data segment
-	mov ds, ax
-	mov es, ax
-	mov fs, ax
-	mov gs, ax
-	mov ss, ax
-
-	push qword rsi ; second arg is code segment
-	lea rax, [rel .next]
-	push rax
-	o64 retf
-.next:
-	ltr cx ; fourth arg is the TSS
-	ret
-
-global gdt_load
-gdt_load:
-	sgdt [rdi] ; first arg is where to write the gdtr value
-	ret
-

src/kernel/idt.cpp

@@ -1,137 +0,0 @@
-#include "kutil/memory.h"
-#include "kutil/no_construct.h"
-#include "idt.h"
-#include "log.h"
-
-extern "C" {
-	void idt_write(const void *idt_ptr);
-
-#define ISR(i, s, name)  extern void name ();
-#define EISR(i, s, name) extern void name ();
-#define IRQ(i, q, name)  extern void name ();
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef EISR
-#undef ISR
-}
-
-// The IDT is initialized _before_ global constructors are called,
-// so we don't want it to have a global constructor, lest it overwrite
-// the previous initialization.
-static kutil::no_construct<IDT> __g_idt_storage;
-IDT &g_idt = __g_idt_storage.value;
-
-
-IDT::IDT()
-{
-	kutil::memset(this, 0, sizeof(IDT));
-	m_ptr.limit = sizeof(m_entries) - 1;
-	m_ptr.base = &m_entries[0];
-
-#define ISR(i, s, name)  set(i, & name, 0x08, 0x8e);
-#define EISR(i, s, name) set(i, & name, 0x08, 0x8e);
-#define IRQ(i, q, name)  set(i, & name, 0x08, 0x8e);
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef EISR
-#undef ISR
-}
-
-IDT &
-IDT::get()
-{
-	return g_idt;
-}
-
-void
-IDT::install() const
-{
-	idt_write(static_cast<const void*>(&m_ptr));
-}
-
-void
-IDT::add_ist_entries()
-{
-#define ISR(i, s, name)   if (s) { set_ist(i, s); }
-#define EISR(i, s, name)  if (s) { set_ist(i, s); }
-#define IRQ(i, q, name)
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef EISR
-#undef ISR
-}
-
-uint8_t
-IDT::used_ist_entries() const
-{
-	uint8_t entries = 0;
-
-#define ISR(i, s, name)   if (s) { entries |= (1 << s); }
-#define EISR(i, s, name)  if (s) { entries |= (1 << s); }
-#define IRQ(i, q, name)
-#include "interrupt_isrs.inc"
-#undef IRQ
-#undef EISR
-#undef ISR
-
-	return entries;
-}
-
-void
-IDT::set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags)
-{
-	uintptr_t addr = reinterpret_cast<uintptr_t>(handler);
-
-	m_entries[i].base_low = addr & 0xffff;
-	m_entries[i].base_mid = (addr >> 16) & 0xffff;
-	m_entries[i].base_high = (addr >> 32) & 0xffffffff;
-	m_entries[i].selector = selector;
-	m_entries[i].flags = flags;
-	m_entries[i].ist = 0;
-	m_entries[i].reserved = 0;
-}
-
-void
-IDT::dump(unsigned index) const
-{
-	unsigned start = 0;
-	unsigned count = (m_ptr.limit + 1) / sizeof(descriptor);
-	if (index != -1) {
-		start = index;
-		count = 1;
-		log::info(logs::boot, "IDT FOR INDEX %02x", index);
-	} else {
-		log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", m_ptr.base, m_ptr.limit+1);
-	}
-
-	const descriptor *idt =
-		reinterpret_cast<const descriptor *>(m_ptr.base);
-
-	for (int i = start; i < start+count; ++i) {
-		uint64_t base =
-			(static_cast<uint64_t>(idt[i].base_high) << 32) |
-			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
-			idt[i].base_low;
-
-		char const *type;
-		switch (idt[i].flags & 0xf) {
-			case 0x5: type = " 32tsk "; break;
-			case 0x6: type = " 16int "; break;
-			case 0x7: type = " 16trp "; break;
-			case 0xe: type = " 32int "; break;
-			case 0xf: type = " 32trp "; break;
-			default:  type = " ????? "; break;
-		}
-
-		if (idt[i].flags & 0x80) {
-			log::debug(logs::boot,
-					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
-					(idt[i].selector & 0x3),
-					((idt[i].selector & 0x4) >> 2),
-					(idt[i].selector >> 3),
-					idt[i].ist,
-					type,
-					((idt[i].flags >> 5) & 0x3));
-		}
-	}
-}

src/kernel/idt.h

@@ -1,63 +0,0 @@
-#pragma once
-/// \file idt.h
-/// Definitions relating to a CPU's IDT table
-#include <stdint.h>
-
-class IDT
-{
-public:
-	IDT();
-
-	/// Install this IDT to the current CPU
-	void install() const;
-
-	/// Add the IST entries listed in the ISR table into the IDT.
-	/// This can't be done until after memory is set up so the
-	/// stacks can be created.
-	void add_ist_entries();
-
-	/// Get the IST entry used by an entry.
-	/// \arg i   Which IDT entry to look in
-	/// \returns The IST index used by entry i, or 0 for none
-	inline uint8_t get_ist(uint8_t i) const {
-		return m_entries[i].ist;
-	}
-
-	/// Set the IST entry used by an entry.
-	/// \arg i   Which IDT entry to set
-	/// \arg ist The IST index for entry i, or 0 for none
-	void set_ist(uint8_t i, uint8_t ist) { m_entries[i].ist = ist; }
-
-	/// Get the IST entries that are used by this table, as a bitmap
-	uint8_t used_ist_entries() const;
-
-	/// Dump debug information about the IDT to the console.
-	/// \arg index  Which entry to print, or -1 for all entries
-	void dump(unsigned index = -1) const;
-
-	/// Get the global IDT
-	static IDT & get();
-
-private:
-	void set(uint8_t i, void (*handler)(), uint16_t selector, uint8_t flags);
-
-	struct descriptor
-	{
-		uint16_t base_low;
-		uint16_t selector;
-		uint8_t ist;
-		uint8_t flags;
-		uint16_t base_mid;
-		uint32_t base_high;
-		uint32_t reserved;		// must be zero
-	} __attribute__ ((packed, aligned(16)));
-
-	struct ptr
-	{
-		uint16_t limit;
-		descriptor *base;
-	} __attribute__ ((packed, aligned(4)));
-
-	descriptor m_entries[256];
-	ptr m_ptr;
-};

src/kernel/interrupt_isrs.inc

@@ -1,36 +1,36 @@
-ISR (0x00, 0, isrDivideByZero)
-ISR (0x01, 0, isrDebug)
-ISR (0x02, 1, isrNMI)
-ISR (0x03, 0, isrBreakpoint)
-ISR (0x04, 0, isrOverflow)
-ISR (0x05, 0, isrBRE)
-ISR (0x06, 0, isrInvalidOp)
-ISR (0x07, 0, isrDNA)
-EISR(0x08, 2, isrDoubleFault)
-ISR (0x09, 0, isrCoprocessor)
-EISR(0x0a, 0, isrInvalidTSS)
-EISR(0x0b, 0, isrSegmentNP)
-EISR(0x0c, 0, isrSSFault)
-EISR(0x0d, 0, isrGPFault)
-EISR(0x0e, 3, isrPageFault)
-ISR (0x0f, 0, isr15)
+ISR (0x00, isrDivideByZero)
+ISR (0x01, isrDebug)
+ISR (0x02, isrNMI)
+ISR (0x03, isrBreakpoint)
+ISR (0x04, isrOverflow)
+ISR (0x05, isrBRE)
+ISR (0x06, isrInvalidOp)
+ISR (0x07, isrDNA)
+EISR(0x08, isrDoubleFault)
+ISR (0x09, isrCoprocessor)
+EISR(0x0a, isrInvalidTSS)
+EISR(0x0b, isrSegmentNP)
+EISR(0x0c, isrSSFault)
+EISR(0x0d, isrGPFault)
+EISR(0x0e, isrPageFault)
+ISR (0x0f, isr15)
 
-ISR (0x10, 0, isrX87FPE)
-ISR (0x11, 0, isrAlignmentChk)
-ISR (0x12, 0, isrMachineChk)
-ISR (0x13, 0, isrSIMDFPE)
-ISR (0x14, 0, isrVirt)
-ISR (0x15, 0, isr21)
-ISR (0x16, 0, isr22)
-ISR (0x17, 0, isr23)
-ISR (0x18, 0, isr24)
-ISR (0x19, 0, isr25)
-ISR (0x1a, 0, isr26)
-ISR (0x1b, 0, isr27)
-ISR (0x1c, 0, isr28)
-ISR (0x1d, 0, isr29)
-ISR (0x1e, 0, isrSecurity)
-ISR (0x1f, 0, isr31)
+ISR (0x10, isrX87FPE)
+ISR (0x11, isrAlignmentChk)
+ISR (0x12, isrMachineChk)
+ISR (0x13, isrSIMDFPE)
+ISR (0x14, isrVirt)
+ISR (0x15, isr21)
+ISR (0x16, isr22)
+ISR (0x17, isr23)
+ISR (0x18, isr24)
+ISR (0x19, isr25)
+ISR (0x1a, isr26)
+ISR (0x1b, isr27)
+ISR (0x1c, isr28)
+ISR (0x1d, isr29)
+ISR (0x1e, isrSecurity)
+ISR (0x1f, isr31)
 
 IRQ (0x20, 0x00, irq00)
 IRQ (0x21, 0x01, irq01)
@@ -237,28 +237,28 @@ IRQ (0xde, 0xbe, irqBE)
 IRQ (0xdf, 0xbf, irqBF)
 
 
-ISR (0xe0, 0, isrTimer)
-ISR (0xe1, 0, isrLINT0)
-ISR (0xe2, 0, isrLINT1)
-ISR (0xe3, 0, isrAPICError)
-ISR (0xe4, 0, isrAssert)
+ISR (0xe0, isrTimer)
+ISR (0xe1, isrLINT0)
+ISR (0xe2, isrLINT1)
+ISR (0xe4, isrAssert)
 
-ISR (0xef, 0, isrSpurious)
+UISR(0xee, isrSyscall)
+ISR (0xef, isrSpurious)
 
-ISR (0xf0, 0, isrIgnore0)
-ISR (0xf1, 0, isrIgnore1)
-ISR (0xf2, 0, isrIgnore2)
-ISR (0xf3, 0, isrIgnore3)
-ISR (0xf4, 0, isrIgnore4)
-ISR (0xf5, 0, isrIgnore5)
-ISR (0xf6, 0, isrIgnore6)
-ISR (0xf7, 0, isrIgnore7)
+ISR (0xf0, isrIgnore0)
+ISR (0xf1, isrIgnore1)
+ISR (0xf2, isrIgnore2)
+ISR (0xf3, isrIgnore3)
+ISR (0xf4, isrIgnore4)
+ISR (0xf5, isrIgnore5)
+ISR (0xf6, isrIgnore6)
+ISR (0xf7, isrIgnore7)
 
-ISR (0xf8, 0, isrIgnore8)
-ISR (0xf9, 0, isrIgnore9)
-ISR (0xfa, 0, isrIgnoreA)
-ISR (0xfb, 0, isrIgnoreB)
-ISR (0xfc, 0, isrIgnoreC)
-ISR (0xfd, 0, isrIgnoreD)
-ISR (0xfe, 0, isrIgnoreE)
-ISR (0xff, 0, isrIgnoreF)
+ISR (0xf8, isrIgnore8)
+ISR (0xf9, isrIgnore9)
+ISR (0xfa, isrIgnoreA)
+ISR (0xfb, isrIgnoreB)
+ISR (0xfc, isrIgnoreC)
+ISR (0xfd, isrIgnoreD)
+ISR (0xfe, isrIgnoreE)
+ISR (0xff, isrIgnoreF)

src/kernel/interrupts.cpp

@@ -8,7 +8,6 @@
 #include "debug.h"
 #include "device_manager.h"
 #include "gdt.h"
-#include "idt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "kernel_memory.h"
@@ -16,7 +15,6 @@
 #include "objects/process.h"
 #include "scheduler.h"
 #include "syscall.h"
-#include "tss.h"
 #include "vm_space.h"
 
 static const uint16_t PIC1 = 0x20;
@@ -24,14 +22,21 @@ static const uint16_t PIC2 = 0xa0;
 
 constexpr uintptr_t apic_eoi_addr = 0xfee000b0 + ::memory::page_offset;
 
-constexpr size_t increment_offset = 0x1000;
-
 extern "C" {
 	void _halt();
 
 	void isr_handler(cpu_state*);
 	void irq_handler(cpu_state*);
 
+#define ISR(i, name)     extern void name ();
+#define EISR(i, name)    extern void name ();
+#define UISR(i, name)    extern void name ();
+#define IRQ(i, q, name)  extern void name ();
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef UISR
+#undef EISR
+#undef ISR
 }
 
 isr
@@ -45,11 +50,13 @@ uint8_t
 get_irq(unsigned vector)
 {
 	switch (vector) {
-#define ISR(i, s, name)
-#define EISR(i, s, name)
+#define ISR(i, name)
+#define EISR(i, name)
+#define UISR(i, name)
 #define IRQ(i, q, name) case i : return q;
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 
@@ -57,7 +64,7 @@ get_irq(unsigned vector)
 	}
 }
 
-void
+static void
 disable_legacy_pic()
 {
 	// Mask all interrupts
@@ -77,19 +84,30 @@ disable_legacy_pic()
 	outb(PIC2+1, 0x02); io_wait();
 }
 
+void
+interrupts_init()
+{
+#define ISR(i, name)     idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
+#define EISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
+#define UISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0xee);
+#define IRQ(i, q, name)  idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
+#include "interrupt_isrs.inc"
+#undef IRQ
+#undef UISR
+#undef EISR
+#undef ISR
+
+	disable_legacy_pic();
+
+	log::info(logs::boot, "Interrupts enabled.");
+}
+
 void
 isr_handler(cpu_state *regs)
 {
 	console *cons = console::get();
-	uint8_t vector = regs->interrupt & 0xff;
 
-	// Clear out the IST for this vector so we just keep using
-	// this stack
-	uint8_t old_ist = IDT::get().get_ist(vector);
-	if (old_ist)
-		IDT::get().set_ist(vector, 0);
-
-	switch (static_cast<isr>(vector)) {
+	switch (static_cast<isr>(regs->interrupt & 0xff)) {
 
 	case isr::isrDebug: {
 			cons->set_color(11);
@@ -123,16 +141,6 @@ isr_handler(cpu_state *regs)
 		}
 		break;
 
-	case isr::isrDoubleFault:
-		cons->set_color(9);
-		cons->printf("\nDouble Fault:\n");
-
-		cons->set_color();
-		print_regs(*regs);
-		print_stacktrace(2);
-		_halt();
-		break;
-
 	case isr::isrGPFault: {
 			cons->set_color(9);
 			cons->puts("\nGeneral Protection Fault:\n");
@@ -146,13 +154,13 @@ isr_handler(cpu_state *regs)
 				switch ((regs->errorcode & 0x07) >> 1) {
 				case 0:
 					cons->printf(" GDT[%x]\n", index);
-					GDT::current().dump(index);
+					gdt_dump(index);
 					break;
 
 				case 1:
 				case 3:
 					cons->printf(" IDT[%x]\n", index);
-					IDT::get().dump(index);
+					idt_dump(index);
 					break;
 
 				default:
@@ -271,10 +279,6 @@ isr_handler(cpu_state *regs)
 		print_stacktrace(2);
 		_halt();
 	}
-
-	// Return the IST for this vector to what it was
-	if (old_ist)
-		IDT::get().set_ist(vector, old_ist);
 	*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;
 }
 
@@ -288,8 +292,8 @@ irq_handler(cpu_state *regs)
 		cons->printf("\nReceived unknown IRQ: %d (vec %d)\n",
 				irq, regs->interrupt);
 		cons->set_color();
-
 		print_regs(*regs);
+		_halt();
 	}
 
 	*reinterpret_cast<uint32_t *>(apic_eoi_addr) = 0;

src/kernel/interrupts.h

@@ -7,11 +7,13 @@
 /// Enum of all defined ISR/IRQ vectors
 enum class isr : uint8_t
 {
-#define ISR(i, s, name)     name = i,
-#define EISR(i, s, name)    name = i,
+#define ISR(i, name)     name = i,
+#define EISR(i, name)    name = i,
+#define UISR(i, name)    name = i,
 #define IRQ(i, q, name)  name = i,
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 
@@ -29,5 +31,6 @@ extern "C" {
 	void interrupts_disable();
 }
 
-/// Disable the legacy PIC
-void disable_legacy_pic();
+/// Fill the IDT with our ISRs, and disable the legacy
+/// PIC interrupts.
+void interrupts_init();

src/kernel/interrupts.s

@@ -1,14 +1,8 @@
 %include "push_all.inc"
 
-section .text
-
 extern isr_handler
-global isr_handler_prelude:function (isr_handler_prelude.end - isr_handler_prelude)
+global isr_handler_prelude
 isr_handler_prelude:
-	push rbp     ; Never executed, fake function prelude
-	mov rbp, rsp ; to calm down gdb
-
-.real:
 	push_all
 	check_swap_gs
 
@@ -16,15 +10,10 @@ isr_handler_prelude:
 	mov rsi, rsp
 	call isr_handler
 	jmp isr_handler_return
-.end:
 
 extern irq_handler
-global irq_handler_prelude:function (irq_handler_prelude.end - irq_handler_prelude)
+global irq_handler_prelude
 irq_handler_prelude:
-	push rbp     ; Never executed, fake function prelude
-	mov rbp, rsp ; to calm down gdb
-
-.real:
 	push_all
 	check_swap_gs
 
@@ -32,45 +21,41 @@ irq_handler_prelude:
 	mov rsi, rsp
 	call irq_handler
 	; fall through to isr_handler_return
-.end:
 
-global isr_handler_return:function (isr_handler_return.end - isr_handler_return)
+global isr_handler_return
 isr_handler_return:
 	check_swap_gs
 	pop_all
 
 	add rsp, 16		; because the ISRs added err/num
 	iretq
-.end:
 
 %macro EMIT_ISR 2
-	global %1:function (%1.end - %1)
+	global %1
 	%1:
 		push 0
 		push %2
-		jmp isr_handler_prelude.real
-	.end:
+		jmp isr_handler_prelude
 %endmacro
 
 %macro EMIT_EISR 2
-	global %1:function (%1.end - %1)
+	global %1
 	%1:
 		push %2
-		jmp isr_handler_prelude.real
-	.end:
+		jmp isr_handler_prelude
 %endmacro
 
 %macro EMIT_IRQ 2
-	global %1:function (%1.end - %1)
+	global %1
 	%1:
 		push 0
 		push %2
-		jmp irq_handler_prelude.real
-	.end:
+		jmp irq_handler_prelude
 %endmacro
 
-%define EISR(i, s, name)     EMIT_EISR name, i   ; ISR with error code
-%define  ISR(i, s, name)     EMIT_ISR name, i
+%define EISR(i, name)     EMIT_EISR name, i   ; ISR with error code
+%define UISR(i, name)     EMIT_ISR name, i    ; ISR callable from user space
+%define  ISR(i, name)     EMIT_ISR name, i
 %define  IRQ(i, q, name)  EMIT_IRQ name, i
 
 section .isrs

src/kernel/loader.s

@@ -0,0 +1,24 @@
+%include "push_all.inc"
+
+extern load_process_image
+
+global preloaded_process_init
+preloaded_process_init:
+
+	; create_process already pushed the arguments for load_process_image and
+	; the following iretq onto the stack for us
+
+	pop rdi ; the physical address of the program image
+	pop rsi ; the virtual address of the program image
+	pop rdx ; the size in bytes of the program image
+	pop rcx ; the address of this thread's TCB
+
+	call load_process_image
+
+	; user rsp is now in rax, put it in the right place for iret
+	mov [rsp + 0x18], rax
+
+	; the entrypoint should already be on the stack
+	swapgs
+	iretq
+

src/kernel/log.cpp

@@ -1,10 +1,8 @@
-#include "j6/signals.h"
 #include "kutil/memory.h"
 #include "kutil/no_construct.h"
 #include "console.h"
 #include "log.h"
-#include "objects/system.h"
-#include "objects/thread.h"
+#include "scheduler.h"
 
 static uint8_t log_buffer[0x10000];
 
@@ -28,54 +26,29 @@ output_log(log::area_t area, log::level severity, const char *message)
 	cons->set_color();
 }
 
-static void
-log_flush()
-{
-	system &sys = system::get();
-	sys.assert_signal(j6_signal_system_has_log);
-}
-
 void
 logger_task()
 {
+	uint8_t buffer[257];
+	auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
 	auto *cons = console::get();
 
 	log::info(logs::task, "Starting kernel logger task");
 	g_logger.set_immediate(nullptr);
-	g_logger.set_flush(log_flush);
 
-	thread &self = thread::current();
-	system &sys = system::get();
-
-	size_t buffer_size = 1;
-	uint8_t *buffer = nullptr;
+	scheduler &s = scheduler::get();
 
 	while (true) {
-		size_t size = g_logger.get_entry(buffer, buffer_size);
-		if (size > buffer_size) {
-			while (size > buffer_size) buffer_size *= 2;
-			kutil::kfree(buffer);
-			buffer = reinterpret_cast<uint8_t*>(kutil::kalloc(buffer_size));
-			kassert(buffer, "Could not allocate logger task buffer");
-			continue;
-		}
-
-		if(size) {
-			auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
+		if(g_logger.get_entry(buffer, sizeof(buffer))) {
 			buffer[ent->bytes] = 0;
-
 			cons->set_color(level_colors[static_cast<int>(ent->severity)]);
 			cons->printf("%7s %5s: %s\n",
 				g_logger.area_name(ent->area),
 				g_logger.level_name(ent->severity),
 				ent->message);
 			cons->set_color();
-		}
-
-		if (!g_logger.has_log()) {
-			sys.deassert_signal(j6_signal_system_has_log);
-			sys.add_blocked_thread(&self);
-			self.wait_on_signals(&sys, j6_signal_system_has_log);
+		} else {
+			s.schedule();
 		}
 	}
 }

src/kernel/main.cpp

@@ -6,28 +6,22 @@
 #include "kutil/assert.h"
 #include "apic.h"
 #include "block_device.h"
-#include "clock.h"
 #include "console.h"
 #include "cpu.h"
 #include "device_manager.h"
 #include "gdt.h"
-#include "idt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "kernel_args.h"
 #include "kernel_memory.h"
 #include "log.h"
-#include "msr.h"
 #include "objects/channel.h"
 #include "objects/event.h"
 #include "objects/thread.h"
-#include "objects/vm_area.h"
 #include "scheduler.h"
 #include "serial.h"
 #include "symbol_table.h"
 #include "syscall.h"
-#include "tss.h"
-#include "vm_space.h"
 
 #ifndef GIT_VERSION
 #define GIT_VERSION
@@ -37,25 +31,16 @@ extern "C" {
 	void kernel_main(kernel::args::header *header);
 	void (*__ctors)(void);
 	void (*__ctors_end)(void);
-	void long_ap_startup(cpu_data *cpu);
-	void ap_startup();
-	void ap_idle();
-	void init_ap_trampoline(void*, cpu_data *, void (*)());
 }
 
 extern void __kernel_assert(const char *, unsigned, const char *);
 
-using namespace kernel;
-
-volatile size_t ap_startup_count;
-static bool scheduler_ready = false;
-
 /// Bootstrap the memory managers.
-void memory_initialize_pre_ctors(args::header &kargs);
-void memory_initialize_post_ctors(args::header &kargs);
-process * load_simple_process(args::program &program);
+void setup_pat();
+void memory_initialize_pre_ctors(kernel::args::header *kargs);
+void memory_initialize_post_ctors(kernel::args::header *kargs);
 
-unsigned start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4);
+using namespace kernel;
 
 /// TODO: not this. this is awful.
 args::framebuffer *fb = nullptr;
@@ -82,6 +67,46 @@ run_constructors()
 	}
 }
 
+channel *std_out = nullptr;
+
+void
+stdout_task()
+{
+	uint8_t buffer[257];
+	auto *ent = reinterpret_cast<log::logger::entry *>(buffer);
+	auto *cons = console::get();
+
+	log::info(logs::task, "Starting kernel stdout task");
+
+	scheduler &s = scheduler::get();
+	thread *th = thread::from_tcb(s.current());
+
+	while (true) {
+		j6_signal_t current = std_out->signals();
+		if (!(current & j6_signal_channel_can_recv)) {
+			th->wait_on_signals(std_out, j6_signal_channel_can_recv);
+			s.schedule();
+		}
+
+		size_t n = 256;
+		j6_status_t status = std_out->dequeue(&n, buffer);
+		if (status != j6_status_ok) {
+			log::warn(logs::task, "Kernel stdout error: %x", status);
+			return;
+		}
+
+		buffer[n] = 0;
+		const char *s = reinterpret_cast<const char *>(buffer);
+
+		while (n) {
+			size_t r = cons->puts(s);
+			n -= r + 1;
+			s += r + 1;
+		}
+	}
+}
+
+
 void
 kernel_main(args::header *header)
 {
@@ -90,64 +115,30 @@ kernel_main(args::header *header)
 	init_console();
 	logger_init();
 
-	cpu_validate();
-
-	log::debug(logs::boot, "    jsix header is at: %016lx", header);
-	log::debug(logs::boot, "     Memory map is at: %016lx", header->mem_map);
-	log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
-	log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
-	log::debug(logs::boot, "    Kernel PML4 is at: %016lx", header->pml4);
-
-	uint64_t cr0, cr4;
-	asm ("mov %%cr0, %0" : "=r"(cr0));
-	asm ("mov %%cr4, %0" : "=r"(cr4));
-	uint64_t efer = rdmsr(msr::ia32_efer);
-	log::debug(logs::boot, "Control regs: cr0:%lx cr4:%lx efer:%lx", cr0, cr4, efer);
+	setup_pat();
 
 	bool has_video = false;
 	if (header->video.size > 0) {
 		has_video = true;
-		fb = &header->video;
+		fb = memory::to_virtual<args::framebuffer>(reinterpret_cast<uintptr_t>(&header->video));
 
 		const args::framebuffer &video = header->video;
-		log::debug(logs::boot, "Framebuffer: %dx%d[%d] type %d @ %llx size %llx",
-			video.horizontal,
-			video.vertical,
-			video.scanline,
-			video.type,
-			video.phys_addr,
-			video.size);
+		log::debug(logs::boot, "Framebuffer: %dx%d[%d] type %s @ %016llx",
+			video.horizontal, video.vertical, video.scanline, video.type, video.phys_addr);
 		logger_clear_immediate();
 	}
 
-	extern IDT &g_idt;
-	extern TSS &g_bsp_tss;
-	extern GDT &g_bsp_gdt;
-	extern cpu_data g_bsp_cpu_data;
-	extern uintptr_t idle_stack_end;
+	gdt_init();
+	interrupts_init();
 
-	IDT *idt = new (&g_idt) IDT;
-
-	cpu_data *cpu = &g_bsp_cpu_data;
-	kutil::memset(cpu, 0, sizeof(cpu_data));
-
-	cpu->self = cpu;
-	cpu->tss = new (&g_bsp_tss) TSS;
-	cpu->gdt = new (&g_bsp_gdt) GDT {cpu->tss};
-	cpu->rsp0 = idle_stack_end;
-	cpu_early_init(cpu);
-
-	disable_legacy_pic();
-
-	memory_initialize_pre_ctors(*header);
+	memory_initialize_pre_ctors(header);
 	run_constructors();
-	memory_initialize_post_ctors(*header);
+	memory_initialize_post_ctors(header);
 
-	cpu->tss->create_ist_stacks(idt->used_ist_entries());
+	cpu_validate();
 
 	for (size_t i = 0; i < header->num_modules; ++i) {
 		args::module &mod = header->modules[i];
-
 		switch (mod.type) {
 		case args::mod_type::symbol_table:
 			new symbol_table {mod.location, mod.size};
@@ -158,30 +149,17 @@ kernel_main(args::header *header)
 		}
 	}
 
-	syscall_initialize();
+	log::debug(logs::boot, "    jsix header is at: %016lx", header);
+	log::debug(logs::boot, "     Memory map is at: %016lx", header->mem_map);
+	log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
+	log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime_services);
 
 	device_manager &devices = device_manager::get();
 	devices.parse_acpi(header->acpi_table);
 
-	// Need the local APIC to get the BSP's id
-	uintptr_t apic_base = devices.get_lapic_base();
-
-	lapic *apic = new lapic(apic_base);
-	apic->enable();
-
-	cpu->id = apic->get_id();
-	cpu->apic = apic;
-
-	cpu_init(cpu, true);
-
-	devices.init_drivers();
-	apic->calibrate_timer();
-
-	const auto &apic_ids = devices.get_apic_ids();
-	unsigned num_cpus = start_aps(*apic, apic_ids, header->pml4);
-
-	idt->add_ist_entries();
 	interrupts_enable();
+	devices.init_drivers();
+	devices.get_lapic()->calibrate_timer();
 
 	/*
 	block_device *disk = devices->get_block_device(0);
@@ -207,138 +185,27 @@ kernel_main(args::header *header)
 	}
 	*/
 
-	scheduler *sched = new scheduler {num_cpus};
-	scheduler_ready = true;
+	syscall_enable();
+	scheduler *sched = new scheduler(devices.get_lapic());
+
+	std_out = new channel;
 
 	// Skip program 0, which is the kernel itself
-	for (unsigned i = 1; i < header->num_programs; ++i)
-		load_simple_process(header->programs[i]);
+	for (size_t i = 1; i < header->num_programs; ++i) {
+		args::program &prog = header->programs[i];
+		thread *th = sched->load_process(prog.phys_addr, prog.virt_addr, prog.size, prog.entrypoint); 
+		if (i == 2) {
+			//th->set_state(thread::state::constant);
+		}
+	}
 
 	if (!has_video)
-		sched->create_kernel_task(logger_task, scheduler::max_priority/2, true);
+		sched->create_kernel_task(logger_task, scheduler::max_priority-1, true);
+	sched->create_kernel_task(stdout_task, scheduler::max_priority-1, true);
+
+	const char stdout_message[] = "Hello on the fake stdout channel\n";
+	size_t message_size = sizeof(stdout_message);
+	std_out->enqueue(&message_size, reinterpret_cast<const void*>(stdout_message));
 
 	sched->start();
 }
-
-unsigned
-start_aps(lapic &apic, const kutil::vector<uint8_t> &ids, void *kpml4)
-{
-	using memory::frame_size;
-	using memory::kernel_stack_pages;
-
-	extern size_t ap_startup_code_size;
-	extern process &g_kernel_process;
-	extern vm_area_guarded &g_kernel_stacks;
-
-	clock &clk = clock::get();
-
-	ap_startup_count = 1; // BSP processor
-	log::info(logs::boot, "Starting %d other CPUs", ids.count() - 1);
-
-	// Since we're using address space outside kernel space, make sure
-	// the kernel's vm_space is used
-	cpu_data &bsp = current_cpu();
-	bsp.process = &g_kernel_process;
-
-	uint16_t index = bsp.index;
-
-	// Copy the startup code somwhere the real mode trampoline can run
-	uintptr_t addr = 0x8000; // TODO: find a valid address, rewrite addresses
-	uint8_t vector = addr >> 12;
-	vm_area *vma = new vm_area_fixed(addr, 0x1000, vm_flags::write);
-	vm_space::kernel_space().add(addr, vma);
-	kutil::memcpy(
-		reinterpret_cast<void*>(addr),
-		reinterpret_cast<void*>(&ap_startup),
-		ap_startup_code_size);
-
-	// AP idle stacks need less room than normal stacks, so pack multiple
-	// into a normal stack area
-	static constexpr size_t idle_stack_bytes = 2048; // 2KiB is generous
-	static constexpr size_t full_stack_bytes = kernel_stack_pages * frame_size;
-	static constexpr size_t idle_stacks_per = full_stack_bytes / idle_stack_bytes;
-
-	uint8_t ist_entries = IDT::get().used_ist_entries();
-
-	size_t free_stack_count = 0;
-	uintptr_t stack_area_start = 0;
-
-	ipi mode = ipi::init | ipi::level | ipi::assert;
-	apic.send_ipi_broadcast(mode, false, 0);
-
-	for (uint8_t id : ids) {
-		if (id == bsp.id) continue;
-
-		// Set up the CPU data structures
-		TSS *tss = new TSS;
-		GDT *gdt = new GDT {tss};
-		cpu_data *cpu = new cpu_data;
-		kutil::memset(cpu, 0, sizeof(cpu_data));
-
-		cpu->self = cpu;
-		cpu->id = id;
-		cpu->index = ++index;
-		cpu->gdt = gdt;
-		cpu->tss = tss;
-
-		tss->create_ist_stacks(ist_entries);
-
-		// Set up the CPU's idle task stack
-		if (free_stack_count == 0) {
-			stack_area_start = g_kernel_stacks.get_section();
-			free_stack_count = idle_stacks_per;
-		}
-
-		uintptr_t stack_end = stack_area_start + free_stack_count-- * idle_stack_bytes;
-		stack_end -= 2 * sizeof(void*); // Null frame
-		*reinterpret_cast<uint64_t*>(stack_end) = 0; // pre-fault the page
-		cpu->rsp0 = stack_end;
-
-		// Set up the trampoline with this CPU's data
-		init_ap_trampoline(kpml4, cpu, ap_idle);
-
-		// Kick it off!
-		size_t current_count = ap_startup_count;
-		log::debug(logs::boot, "Starting AP %d: stack %llx", cpu->index, stack_end);
-
-		ipi startup = ipi::startup | ipi::assert;
-
-		apic.send_ipi(startup, vector, id);
-		for (unsigned i = 0; i < 20; ++i) {
-			if (ap_startup_count > current_count) break;
-			clk.spinwait(20);
-		}
-
-		// If the CPU already incremented ap_startup_count, it's done
-		if (ap_startup_count > current_count)
-			continue;
-
-		// Send the second SIPI (intel recommends this)
-		apic.send_ipi(startup, vector, id);
-		for (unsigned i = 0; i < 100; ++i) {
-			if (ap_startup_count > current_count) break;
-			clk.spinwait(100);
-		}
-
-		log::warn(logs::boot, "No response from AP %d within timeout", id);
-	}
-
-	log::info(logs::boot, "%d CPUs running", ap_startup_count);
-	vm_space::kernel_space().remove(vma);
-	return ap_startup_count;
-}
-
-void
-long_ap_startup(cpu_data *cpu)
-{
-	cpu_init(cpu, false);
-	++ap_startup_count;
-	while (!scheduler_ready) asm ("pause");
-
-	uintptr_t apic_base =
-		device_manager::get().get_lapic_base();
-	cpu->apic = new lapic(apic_base);
-	cpu->apic->enable();
-
-	scheduler::get().start();
-}

src/kernel/memory_bootstrap.cpp

@@ -1,31 +1,31 @@
 #include <utility>
 
 #include "kernel_args.h"
-#include "j6/init.h"
 
 #include "kutil/assert.h"
 #include "kutil/heap_allocator.h"
 #include "kutil/no_construct.h"
 
-#include "device_manager.h"
 #include "frame_allocator.h"
-#include "gdt.h"
 #include "io.h"
 #include "log.h"
 #include "msr.h"
 #include "objects/process.h"
-#include "objects/thread.h"
-#include "objects/system.h"
 #include "objects/vm_area.h"
 #include "vm_space.h"
 
+using memory::frame_size;
 using memory::heap_start;
 using memory::kernel_max_heap;
+using memory::kernel_offset;
+using memory::heap_start;
+using memory::page_offset;
+using memory::pml4e_kernel;
+using memory::pml4e_offset;
+using memory::table_entries;
 
 using namespace kernel;
 
-extern "C" void initialize_main_thread();
-extern "C" uintptr_t initialize_main_user_stack();
 
 // These objects are initialized _before_ global constructors are called,
 // so we don't want them to have global constructors at all, lest they
@@ -36,17 +36,20 @@ kutil::heap_allocator &g_kernel_heap = __g_kernel_heap_storage.value;
 static kutil::no_construct<frame_allocator> __g_frame_allocator_storage;
 frame_allocator &g_frame_allocator = __g_frame_allocator_storage.value;
 
-static kutil::no_construct<vm_area_untracked> __g_kernel_heap_area_storage;
-vm_area_untracked &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
+static kutil::no_construct<vm_area_open> __g_kernel_heap_area_storage;
+vm_area_open &g_kernel_heap_area = __g_kernel_heap_area_storage.value;
 
-static kutil::no_construct<vm_area_guarded> __g_kernel_stacks_storage;
-vm_area_guarded &g_kernel_stacks = __g_kernel_stacks_storage.value;
+vm_area_buffers g_kernel_stacks {
+	memory::kernel_max_stacks,
+	vm_space::kernel_space(),
+	vm_flags::write,
+	memory::kernel_stack_pages};
 
-vm_area_guarded g_kernel_buffers {
-	memory::buffers_start,
-	memory::kernel_buffer_pages,
+vm_area_buffers g_kernel_buffers {
 	memory::kernel_max_buffers,
-	vm_flags::write};
+	vm_space::kernel_space(),
+	vm_flags::write,
+	memory::kernel_buffer_pages};
 
 void * operator new(size_t size)           { return g_kernel_heap.allocate(size); }
 void * operator new [] (size_t size)       { return g_kernel_heap.allocate(size); }
@@ -58,84 +61,45 @@ namespace kutil {
 void kfree(void *p) { return g_kernel_heap.free(p); }
 }
 
-template <typename T>
-uintptr_t
-get_physical_page(T *p) {
-	return memory::page_align_down(reinterpret_cast<uintptr_t>(p));
-}
-
-void
-memory_initialize_pre_ctors(args::header &kargs)
+/*
+void walk_page_table(
+	page_table *table,
+	page_table::level level,
+	uintptr_t &current_start,
+	size_t &current_bytes,
+	vm_area &karea)
 {
-	using kernel::args::frame_block;
+	constexpr size_t huge_page_size = (1ull<<30);
+	constexpr size_t large_page_size = (1ull<<21);
 
-	page_table *kpml4 = static_cast<page_table*>(kargs.pml4);
+	for (unsigned i = 0; i < table_entries; ++i) {
+		page_table *next = table->get(i);
+		if (!next) {
+			if (current_bytes)
+				karea.commit(current_start, current_bytes);
+			current_start = 0;
+			current_bytes = 0;
+			continue;
+		} else if (table->is_page(level, i)) {
+			if (!current_bytes)
+				current_start = reinterpret_cast<uintptr_t>(next);
+			current_bytes +=
+				(level == page_table::level::pt
+					? frame_size
+					: level == page_table::level::pd
+						? large_page_size
+						: huge_page_size);
+		} else {
+			page_table::level deeper =
+				static_cast<page_table::level>(
+					static_cast<unsigned>(level) + 1);
 
-	new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
-
-	frame_block *blocks = reinterpret_cast<frame_block*>(memory::bitmap_start);
-	new (&g_frame_allocator) frame_allocator {blocks, kargs.frame_block_count};
-
-	// Mark all the things the bootloader allocated for us as used
-	g_frame_allocator.used(
-		get_physical_page(&kargs),
-		memory::page_count(sizeof(kargs)));
-
-	g_frame_allocator.used(
-		get_physical_page(kargs.frame_blocks),
-		kargs.frame_block_pages);
-
-	g_frame_allocator.used(
-		get_physical_page(kargs.pml4),
-		kargs.table_pages);
-
-	for (unsigned i = 0; i < kargs.num_modules; ++i) {
-		const kernel::args::module &mod = kargs.modules[i];
-		g_frame_allocator.used(
-			get_physical_page(mod.location),
-			memory::page_count(mod.size));
-	}
-
-	for (unsigned i = 0; i < kargs.num_programs; ++i) {
-		const kernel::args::program &prog = kargs.programs[i];
-		for (auto &sect : prog.sections) {
-			if (!sect.size) continue;
-			g_frame_allocator.used(
-				sect.phys_addr,
-				memory::page_count(sect.size));
+			walk_page_table(
+				next, deeper, current_start, current_bytes, kspace);
 		}
 	}
-
-	process *kp = process::create_kernel_process(kpml4);
-	vm_space &vm = kp->space();
-
-	vm_area *heap = new (&g_kernel_heap_area)
-		vm_area_untracked(kernel_max_heap, vm_flags::write);
-
-	vm.add(heap_start, heap);
-
-	vm_area *stacks = new (&g_kernel_stacks) vm_area_guarded {
-		memory::stacks_start,
-		memory::kernel_stack_pages,
-		memory::kernel_max_stacks,
-		vm_flags::write};
-	vm.add(memory::stacks_start, &g_kernel_stacks);
-
-	// Clean out any remaning bootloader page table entries
-	for (unsigned i = 0; i < memory::pml4e_kernel; ++i)
-		kpml4->entries[i] = 0;
-}
-
-void
-memory_initialize_post_ctors(args::header &kargs)
-{
-	vm_space &vm = vm_space::kernel_space();
-	vm.add(memory::buffers_start, &g_kernel_buffers);
-
-	g_frame_allocator.free(
-		get_physical_page(kargs.page_tables),
-		kargs.table_count);
 }
+*/
 
 static void
 log_mtrrs()
@@ -192,104 +156,70 @@ log_mtrrs()
 		pat_names[(pat >> (6*8)) & 7], pat_names[(pat >> (7*8)) & 7]);
 }
 
-
-process *
-load_simple_process(args::program &program)
+void
+setup_pat()
 {
-	using kernel::args::section_flags;
-
-	process *p = new process;
-	vm_space &space = p->space();
-
-	for (const auto &sect : program.sections) {
-		vm_flags flags =
-			(bitfield_has(sect.type, section_flags::execute) ? vm_flags::exec : vm_flags::none) |
-			(bitfield_has(sect.type, section_flags::write) ? vm_flags::write : vm_flags::none);
-
-		vm_area *vma = new vm_area_fixed(sect.phys_addr, sect.size, flags);
-		space.add(sect.virt_addr, vma);
+	uint64_t pat = rdmsr(msr::ia32_pat);
+	pat = (pat & 0x00ffffffffffffffull) | (0x01ull << 56); // set PAT 7 to WC
+	wrmsr(msr::ia32_pat, pat);
+	log_mtrrs();
 }
 
-	uint64_t iopl = (3ull << 12);
-	uintptr_t trampoline = reinterpret_cast<uintptr_t>(initialize_main_thread);
 
-	thread *main = p->create_thread();
-	main->add_thunk_user(program.entrypoint, trampoline, iopl);
-	main->set_state(thread::state::ready);
-
-	return p;
-}
-
-template <typename T>
-inline T * push(uintptr_t &rsp, size_t size = sizeof(T)) {
-	rsp -= size;
-	T *p = reinterpret_cast<T*>(rsp);
-	rsp &= ~(sizeof(uint64_t)-1); // Align the stack
-	return p;
-}
-
-uintptr_t
-initialize_main_user_stack()
+void
+memory_initialize_pre_ctors(args::header *kargs)
 {
-	process &proc = process::current();
-	thread &th = thread::current();
-	TCB *tcb = th.tcb();
+	new (&g_kernel_heap) kutil::heap_allocator {heap_start, kernel_max_heap};
+	new (&g_frame_allocator) frame_allocator;
 
-	const char message[] = "Hello from the kernel!";
-	char *message_arg = push<char>(tcb->rsp3, sizeof(message));
-	kutil::memcpy(message_arg, message, sizeof(message));
-
-	j6_init_value *initv = nullptr;
-	unsigned n = 0;
-
-	extern args::framebuffer *fb;
-	if (fb) {
-		j6_init_framebuffer *fb_desc = push<j6_init_framebuffer>(tcb->rsp3);
-		kutil::memset(fb_desc, 0, sizeof(j6_init_framebuffer));
-
-		fb_desc->addr = fb->phys_addr;
-		fb_desc->size = fb->size;
-		fb_desc->vertical = fb->vertical;
-		fb_desc->horizontal = fb->horizontal;
-		fb_desc->scanline = fb->scanline;
-
-		if (fb->type == kernel::args::fb_type::bgr8)
-			fb_desc->flags |= 1;
-
-		initv = push<j6_init_value>(tcb->rsp3);
-		initv->type = j6_init_desc_framebuffer;
-		initv->data = fb_desc;
-		++n;
+	args::mem_entry *entries = kargs->mem_map;
+	const size_t count = kargs->map_count;
+	for (unsigned i = 0; i < count; ++i) {
+		// TODO: use entry attributes
+		// TODO: copy anything we need from "pending" memory and free it
+		args::mem_entry &e = entries[i];
+		if (e.type == args::mem_type::free)
+			g_frame_allocator.free(e.start, e.pages);
 	}
 
+	page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
+	process *kp = process::create_kernel_process(kpml4);
+	vm_space &vm = kp->space();
 
-	initv = push<j6_init_value>(tcb->rsp3);
-	initv->type = j6_init_handle_other;
-	initv->handle.type = j6_object_type_system;
-	initv->handle.handle = proc.add_handle(&system::get());
-	++n;
+	vm_area *heap = new (&g_kernel_heap_area)
+		vm_area_open(memory::kernel_max_heap, vm, vm_flags::write);
+
+	vm.add(memory::heap_start, heap);
+}
+
+void
+memory_initialize_post_ctors(args::header *kargs)
+{
+	/*
+	uintptr_t current_start = 0;
+	size_t current_bytes = 0;
+
+	// TODO: Should we exclude the top of this area? (eg, buffers, stacks, etc)
+	page_table *kpml4 = reinterpret_cast<page_table*>(kargs->pml4);
+	for (unsigned i = pml4e_kernel; i < pml4e_offset; ++i) {
+		page_table *pdp = kpml4->get(i);
+		kassert(pdp, "Bootloader did not create all kernelspace PDs");
+
+		walk_page_table(
+			pdp, page_table::level::pdp,
+			current_start, current_bytes,
+			g_kernel_space);
+	}
+
+	if (current_bytes)
+		g_kernel_space.commit(current_start, current_bytes);
+	*/
+	vm_space &vm = vm_space::kernel_space();
+	vm.add(memory::stacks_start, &g_kernel_stacks);
+	vm.add(memory::buffers_start, &g_kernel_buffers);
+
+	g_frame_allocator.free(
+		reinterpret_cast<uintptr_t>(kargs->page_tables),
+		kargs->table_count);
 
-	initv = push<j6_init_value>(tcb->rsp3);
-	initv->type = j6_init_handle_self;
-	initv->handle.type = j6_object_type_process;
-	initv->handle.handle = proc.self_handle();
-	++n;
-
-	initv = push<j6_init_value>(tcb->rsp3);
-	initv->type = j6_init_handle_self;
-	initv->handle.type = j6_object_type_thread;
-	initv->handle.handle = th.self_handle();
-	++n;
-
-	uint64_t *initc = push<uint64_t>(tcb->rsp3);
-	*initc = n;
-
-	char **argv0 = push<char*>(tcb->rsp3);
-	*argv0 = message_arg;
-
-	uint64_t *argc = push<uint64_t>(tcb->rsp3);
-	*argc = 1;
-
-	th.clear_state(thread::state::loading);
-	return tcb->rsp3;
 }

src/kernel/objects/channel.cpp

@@ -4,13 +4,13 @@
 #include "objects/channel.h"
 #include "objects/vm_area.h"
 
-extern vm_area_guarded g_kernel_buffers;
+extern vm_area_buffers g_kernel_buffers;
 
 constexpr size_t buffer_bytes = memory::kernel_buffer_pages * memory::frame_size;
 
 channel::channel() :
 	m_len(0),
-	m_data(g_kernel_buffers.get_section()),
+	m_data(g_kernel_buffers.get_buffer()),
 	m_buffer(reinterpret_cast<uint8_t*>(m_data), buffer_bytes),
 	kobject(kobject::type::channel, j6_signal_channel_can_send)
 {
@@ -79,7 +79,7 @@ void
 channel::close()
 {
 	kobject::close();
-	g_kernel_buffers.return_section(m_data);
+	g_kernel_buffers.return_buffer(m_data);
 }
 
 void

src/kernel/objects/kobject.h

@@ -16,9 +16,17 @@ public:
 	/// Types of kernel objects.
 	enum class type : uint16_t
 	{
-#define OBJECT_TYPE( name, val ) name = val,
-#include "j6/tables/object_types.inc"
-#undef OBJECT_TYPE
+		none,
+		system,
+
+		event,
+		channel,
+		endpoint,
+
+		vma,
+
+		process,
+		thread,
 
 		max
 	};

src/kernel/objects/process.cpp

@@ -13,13 +13,17 @@ static kutil::no_construct<process> __g_kernel_process_storage;
 process &g_kernel_process = __g_kernel_process_storage.value;
 
 
+kutil::vector<process*> process::s_processes;
+
 process::process() :
 	kobject {kobject::type::process},
-	m_next_handle {1},
+	m_next_handle {0},
 	m_state {state::running}
 {
+	s_processes.append(this);
+
 	j6_handle_t self = add_handle(this);
-	kassert(self == self_handle(), "Process self-handle is not 1");
+	kassert(self == self_handle(), "Process self-handle is not 0");
 }
 
 // The "kernel process"-only constructor
@@ -35,9 +39,10 @@ process::~process()
 {
 	for (auto &it : m_handles)
 		if (it.val) it.val->handle_release();
+	s_processes.remove_swap(this);
 }
 
-process & process::current() { return *current_cpu().process; }
+process & process::current() { return *bsp_cpu_data.p; }
 process & process::kernel_process() { return g_kernel_process; }
 
 process *
@@ -47,7 +52,7 @@ process::create_kernel_process(page_table *pml4)
 }
 
 void
-process::exit(int32_t code)
+process::exit(unsigned code)
 {
 	// TODO: make this thread-safe
 	m_state = state::exited;
@@ -58,7 +63,7 @@ process::exit(int32_t code)
 		thread->exit(code);
 	}
 
-	if (this == current_cpu().process)
+	if (this == bsp_cpu_data.p)
 		scheduler::get().schedule();
 }
 
@@ -88,7 +93,7 @@ process::update()
 thread *
 process::create_thread(uint8_t priority, bool user)
 {
-	if (priority == default_priority)
+	if (priority == default_pri)
 		priority = scheduler::default_priority;
 
 	thread *th = new thread(*this, priority);
@@ -97,13 +102,11 @@ process::create_thread(uint8_t priority, bool user)
 	if (user) {
 		uintptr_t stack_top = stacks_top - (m_threads.count() * stack_size);
 
-		vm_flags flags = vm_flags::zero|vm_flags::write;
-		vm_area *vma = new vm_area_open(stack_size, flags);
+		vm_area *vma = new vm_area_open(stack_size, m_space,
+				vm_flags::zero|vm_flags::write);
 		m_space.add(stack_top - stack_size, vma);
 
-		// Space for null frame - because the page gets zeroed on
-		// allocation, just pointing rsp here does the trick
-		th->tcb()->rsp3 = stack_top - 2 * sizeof(uint64_t);
+		th->tcb()->rsp3 = stack_top;
 	}
 
 	m_threads.append(th);
@@ -120,8 +123,6 @@ process::thread_exited(thread *th)
 	remove_handle(th->self_handle());
 	delete th;
 
-	// TODO: delete the thread's stack VMA
-
 	if (m_threads.count() == 0) {
 		exit(status);
 		return true;

src/kernel/objects/process.h

@@ -16,10 +16,10 @@ public:
 	constexpr static uintptr_t stacks_top = 0x0000800000000000;
 
 	/// Size of userspace thread stacks
-	constexpr static size_t stack_size = 0x4000000; // 64MiB
+	constexpr static size_t stack_size = 0x4000;
 
 	/// Value that represents default priority
-	constexpr static uint8_t default_priority = 0xff;
+	constexpr static uint8_t default_pri = 0xff;
 
 	/// Constructor.
 	process();
@@ -34,7 +34,7 @@ public:
 
 	/// Terminate this process.
 	/// \arg code   The return code to exit with.
-	void exit(int32_t code);
+	void exit(unsigned code);
 
 	/// Update internal bookkeeping about threads.
 	void update();
@@ -46,7 +46,7 @@ public:
 	/// \args priority  The new thread's scheduling priority
 	/// \args user      If true, create a userspace stack for this thread
 	/// \returns        The newly created thread object
-	thread * create_thread(uint8_t priorty = default_priority, bool user = true);
+	thread * create_thread(uint8_t priorty = default_pri, bool user = true);
 
 	/// Start tracking an object with a handle.
 	/// \args obj  The object this handle refers to
@@ -69,7 +69,7 @@ public:
 	bool thread_exited(thread *th);
 
 	/// Get the handle for this process to refer to itself
-	inline j6_handle_t self_handle() const { return 1; }
+	inline j6_handle_t self_handle() const { return 0; }
 
 	/// Get the process object that owns kernel threads and the
 	/// kernel address space
@@ -84,7 +84,7 @@ private:
 	// This constructor is called by create_kernel_process
 	process(page_table *kpml4);
 
-	int32_t m_return_code;
+	uint32_t m_return_code;
 
 	vm_space m_space;
 
@@ -94,4 +94,6 @@ private:
 
 	enum class state : uint8_t { running, exited };
 	state m_state;
+
+	static kutil::vector<process*> s_processes;
 };

src/kernel/objects/system.h

@@ -10,7 +10,7 @@ class system :
 public:
 	static constexpr kobject::type type = kobject::type::event;
 
-	inline static system & get() { return s_instance; }
+	inline static system * get() { return &s_instance; }
 
 private:
 	static system s_instance;

src/kernel/objects/thread.cpp

@@ -9,7 +9,7 @@
 extern "C" void kernel_to_user_trampoline();
 static constexpr j6_signal_t thread_default_signals = 0;
 
-extern vm_area_guarded &g_kernel_stacks;
+extern vm_area_buffers g_kernel_stacks;
 
 thread::thread(process &parent, uint8_t pri, uintptr_t rsp0) :
 	kobject(kobject::type::thread, thread_default_signals),
@@ -32,7 +32,7 @@ thread::thread(process &parent, uint8_t pri, uintptr_t rsp0) :
 
 thread::~thread()
 {
-	g_kernel_stacks.return_section(m_tcb.kernel_stack);
+	g_kernel_stacks.return_buffer(m_tcb.kernel_stack);
 }
 
 thread *
@@ -43,9 +43,13 @@ thread::from_tcb(TCB *tcb)
 	return reinterpret_cast<thread*>(kutil::offset_pointer(tcb, offset));
 }
 
-thread & thread::current() { return *current_cpu().thread; }
+thread &
+thread::current()
+{
+	return *bsp_cpu_data.t;
+}
 
-inline void schedule_if_current(thread *t) { if (t == current_cpu().thread) scheduler::get().schedule(); }
+inline void schedule_if_current(thread *t) { if (t == bsp_cpu_data.t) scheduler::get().schedule(); }
 
 void
 thread::wait_on_signals(kobject *obj, j6_signal_t signals)
@@ -132,7 +136,7 @@ thread::wake_on_result(kobject *obj, j6_status_t result)
 }
 
 void
-thread::exit(int32_t code)
+thread::exit(uint32_t code)
 {
 	m_return_code = code;
 	set_state(state::exited);
@@ -145,10 +149,6 @@ thread::exit(int32_t code)
 void
 thread::add_thunk_kernel(uintptr_t rip)
 {
-	// This adds just enough values to the top of the
-	// kernel stack to come out of task_switch correctly
-	// and start executing at rip (still in kernel mode)
-
 	m_tcb.rsp -= sizeof(uintptr_t) * 7;
 	uintptr_t *stack = reinterpret_cast<uintptr_t*>(m_tcb.rsp);
 
@@ -162,24 +162,15 @@ thread::add_thunk_kernel(uintptr_t rip)
 }
 
 void
-thread::add_thunk_user(uintptr_t rip3, uintptr_t rip0, uint64_t flags)
+thread::add_thunk_user(uintptr_t rip)
 {
-	// This sets up the stack to:
-	// a) come out of task_switch and return to rip0 (default is the
-	//    kernel/user trampoline) (via add_thunk_kernel) - if this is
-	//    changed, it needs to end up at the trampoline with the stack
-	//    as it was
-	// b) come out of the kernel/user trampoline and start executing
-	//    in user mode at rip
-
 	m_tcb.rsp -= sizeof(uintptr_t) * 8;
 	uintptr_t *stack = reinterpret_cast<uintptr_t*>(m_tcb.rsp);
-	flags |= 0x200;
 
-	stack[7] = rip3;       // return rip in rcx
+	stack[7] = rip;        // return rip in rcx
 	stack[6] = m_tcb.rsp3; // rbp
 	stack[5] = 0xbbbbbbbb; // rbx
-	stack[4] = flags;      // r11 sets RFLAGS
+	stack[4] = 0x00000200; // r11 sets RFLAGS
 	stack[3] = 0x12121212; // r12
 	stack[2] = 0x13131313; // r13
 	stack[1] = 0x14141414; // r14
@@ -187,7 +178,7 @@ thread::add_thunk_user(uintptr_t rip3, uintptr_t rip0, uint64_t flags)
 
 	static const uintptr_t trampoline =
 		reinterpret_cast<uintptr_t>(kernel_to_user_trampoline);
-	add_thunk_kernel(rip0 ? rip0 : trampoline);
+	add_thunk_kernel(trampoline);
 }
 
 void
@@ -200,7 +191,7 @@ thread::setup_kernel_stack()
 	constexpr unsigned null_frame_entries = 2;
 	constexpr size_t null_frame_size = null_frame_entries * sizeof(uint64_t);
 
-	uintptr_t stack_addr = g_kernel_stacks.get_section();
+	uintptr_t stack_addr = g_kernel_stacks.get_buffer();
 	uintptr_t stack_end = stack_addr + stack_bytes;
 
 	uint64_t *null_frame = reinterpret_cast<uint64_t*>(stack_end - null_frame_size);
@@ -221,5 +212,7 @@ thread::create_idle_thread(process &kernel, uint8_t pri, uintptr_t rsp0)
 	thread *idle = new thread(kernel, pri, rsp0);
 	idle->set_state(state::constant);
 	idle->set_state(state::ready);
+	log::info(logs::task, "Created idle thread as koid %llx", idle->koid());
+
 	return idle;
 }

src/kernel/objects/thread.h

@@ -131,18 +131,15 @@ public:
 
 	/// Terminate this thread.
 	/// \arg code   The return code to exit with.
-	void exit(int32_t code);
+	void exit(unsigned code);
 
 	/// Add a stack header that returns to the given address in kernel space.
 	/// \arg rip  The address to return to, must be kernel space
 	void add_thunk_kernel(uintptr_t rip);
 
-	/// Add a stack header that returns to the given address in user space
-	/// via a function in kernel space.
-	/// \arg rip3  The user space address to return to
-	/// \arg rip0  The kernel function to pass through, optional
-	/// \arg flags Extra RFLAGS values to set, optional
-	void add_thunk_user(uintptr_t rip3, uintptr_t rip0 = 0, uint64_t flags = 0);
+	/// Add a stack header that returns to the given address in user space.
+	/// \arg rip  The address to return to, must be user space
+	void add_thunk_user(uintptr_t rip);
 
 	/// Get the handle representing this thread to its process
 	j6_handle_t self_handle() const { return m_self_handle; }
@@ -176,7 +173,7 @@ private:
 	wait_type m_wait_type;
 	// There should be 1 byte of padding here
 
-	int32_t m_return_code;
+	uint32_t m_return_code;
 
 	uint64_t m_wait_data;
 	j6_status_t m_wait_result;

src/kernel/objects/vm_area.cpp

@@ -1,7 +1,5 @@
-#include "frame_allocator.h"
 #include "kernel_memory.h"
 #include "objects/vm_area.h"
-#include "page_tree.h"
 #include "vm_space.h"
 
 using memory::frame_size;
@@ -9,31 +7,30 @@ using memory::frame_size;
 vm_area::vm_area(size_t size, vm_flags flags) :
 	m_size {size},
 	m_flags {flags},
-	m_spaces {m_vector_static, 0, static_size},
 	kobject {kobject::type::vma}
 {
 }
 
 vm_area::~vm_area() {}
 
-bool
-vm_area::add_to(vm_space *space)
+size_t
+vm_area::resize(size_t size)
 {
-	for (auto *s : m_spaces) {
-		if (s == space)
-			return true;
-	}
-	m_spaces.append(space);
-	return true;
+	if (mapper().can_resize(size))
+		m_size = size;
+	return m_size;
 }
 
-bool
-vm_area::remove_from(vm_space *space)
+void
+vm_area::commit(uintptr_t phys, uintptr_t offset, size_t count)
 {
-	m_spaces.remove_swap(space);
-	return
-		!m_spaces.count() &&
-		!(m_flags && vm_flags::mmio);
+	mapper().map(offset, count, phys);
+}
+
+void
+vm_area::uncommit(uintptr_t offset, size_t count)
+{
+	mapper().unmap(offset, count);
 }
 
 void
@@ -43,99 +40,48 @@ vm_area::on_no_handles()
 	delete this;
 }
 
-size_t
-vm_area::resize(size_t size)
-{
-	if (can_resize(size))
-		m_size = size;
-	return m_size;
-}
 
-bool
-vm_area::can_resize(size_t size)
-{
-	for (auto *space : m_spaces)
-		if (!space->can_resize(*this, size))
-			return false;
-	return true;
-}
 
-vm_area_fixed::vm_area_fixed(uintptr_t start, size_t size, vm_flags flags) :
-	m_start {start},
+vm_area_shared::vm_area_shared(size_t size, vm_flags flags) :
+	m_mapper {*this},
 	vm_area {size, flags}
 {
 }
 
-vm_area_fixed::~vm_area_fixed() {}
-
-size_t vm_area_fixed::resize(size_t size)
-{
-	// Not resizable
-	return m_size;
-}
-
-bool vm_area_fixed::get_page(uintptr_t offset, uintptr_t &phys)
-{
-	if (offset > m_size)
-		return false;
-
-	phys = m_start + offset;
-	return true;
-}
-
-
-vm_area_untracked::vm_area_untracked(size_t size, vm_flags flags) :
-	vm_area {size, flags}
+vm_area_shared::~vm_area_shared()
 {
 }
 
-vm_area_untracked::~vm_area_untracked() {}
 
-bool
-vm_area_untracked::get_page(uintptr_t offset, uintptr_t &phys)
-{
-	if (offset > m_size)
-		return false;
-
-	return frame_allocator::get().allocate(1, &phys);
-}
-
-bool
-vm_area_untracked::add_to(vm_space *space)
-{
-	if (!m_spaces.count())
-		return vm_area::add_to(space);
-	return m_spaces[0] == space;
-}
-
-
-vm_area_open::vm_area_open(size_t size, vm_flags flags) :
-	m_mapped {nullptr},
-	vm_area {size, flags}
+vm_area_open::vm_area_open(size_t size, vm_space &space, vm_flags flags) :
+	m_mapper(*this, space),
+	vm_area(size, flags)
 {
 }
 
-vm_area_open::~vm_area_open() {}
-
-bool
-vm_area_open::get_page(uintptr_t offset, uintptr_t &phys)
+void
+vm_area_open::commit(uintptr_t phys, uintptr_t offset, size_t count)
 {
-	return page_tree::find_or_add(m_mapped, offset, phys);
+	m_mapper.map(offset, count, phys);
+}
+
+void
+vm_area_open::uncommit(uintptr_t offset, size_t count)
+{
+	m_mapper.unmap(offset, count);
 }
 
 
-vm_area_guarded::vm_area_guarded(uintptr_t start, size_t buf_pages, size_t size, vm_flags flags) :
-	m_start {start},
+vm_area_buffers::vm_area_buffers(size_t size, vm_space &space, vm_flags flags, size_t buf_pages) :
+	m_mapper {*this, space},
 	m_pages {buf_pages},
 	m_next {memory::frame_size},
-	vm_area_untracked {size, flags}
+	vm_area {size, flags}
 {
 }
 
-vm_area_guarded::~vm_area_guarded() {}
-
 uintptr_t
-vm_area_guarded::get_section()
+vm_area_buffers::get_buffer()
 {
 	if (m_cache.count() > 0) {
 		return m_cache.pop();
@@ -143,27 +89,33 @@ vm_area_guarded::get_section()
 
 	uintptr_t addr = m_next;
 	m_next += (m_pages + 1) * memory::frame_size;
-	return m_start + addr;
+	return m_mapper.space().lookup(*this, addr);
 }
 
 void
-vm_area_guarded::return_section(uintptr_t addr)
+vm_area_buffers::return_buffer(uintptr_t addr)
 {
 	m_cache.append(addr);
 }
 
 bool
-vm_area_guarded::get_page(uintptr_t offset, uintptr_t &phys)
+vm_area_buffers::allowed(uintptr_t offset) const
 {
-	if (offset > m_next)
-		return false;
+	if (offset >= m_next) return false;
 
-	// make sure this isn't in a guard page. (sections are
-	// m_pages big plus 1 leading guard page, so page 0 is
-	// invalid)
-	if ((offset >> 12) % (m_pages+1) == 0)
-		return false;
-
-	return vm_area_untracked::get_page(offset, phys);
+	// Buffers are m_pages big plus 1 leading guard page
+	return memory::page_align_down(offset) % (m_pages+1);
+}
+
+void
+vm_area_buffers::commit(uintptr_t phys, uintptr_t offset, size_t count)
+{
+	m_mapper.map(offset, count, phys);
+}
+
+void
+vm_area_buffers::uncommit(uintptr_t offset, size_t count)
+{
+	m_mapper.unmap(offset, count);
 }
 

src/kernel/objects/vm_area.h

@@ -11,15 +11,26 @@
 
 #include "kernel_memory.h"
 #include "objects/kobject.h"
+#include "vm_mapper.h"
 
-class page_tree;
 class vm_space;
 
 enum class vm_flags : uint32_t
 {
-#define VM_FLAG(name, v) name = v,
-#include "j6/tables/vm_flags.inc"
-#undef VM_FLAG
+	none            = 0x00000000,
+
+	write           = 0x00000001,
+	exec            = 0x00000002,
+
+	zero            = 0x00000010, 
+	contiguous      = 0x00000020,
+
+	large_pages     = 0x00000100,
+	huge_pages      = 0x00000200,
+
+	mmio            = 0x00010000,
+	write_combine   = 0x00020000,
+
 	user_mask       = 0x0000ffff  ///< flags allowed via syscall
 };
 
@@ -44,134 +55,136 @@ public:
 	/// Get the flags set for this area
 	inline vm_flags flags() const { return m_flags; }
 
-	/// Track that this area was added to a vm_space
-	/// \arg space  The space to add this area to
-	/// \returns    False if this area cannot be added
-	virtual bool add_to(vm_space *space);
-
-	/// Track that this area was removed frm a vm_space
-	/// \arg space  The space that is removing this area
-	/// \returns    True if the removing space should free the pages
-	///             mapped for this area
-	virtual bool remove_from(vm_space *space);
-
 	/// Change the virtual size of the memory area. This may cause
 	/// deallocation if the new size is smaller than the current size.
 	/// Note that if resizing is unsuccessful, the previous size will
 	/// be returned.
 	/// \arg size  The desired new virtual size
 	/// \returns   The new virtual size
-	virtual size_t resize(size_t size);
+	size_t resize(size_t size);
 
-	/// Get the physical page for the given offset
-	/// \arg offset The offset into the VMA
-	/// \arg phys   [out] Receives the physical page address, if any
-	/// \returns    True if there should be a page at the given offset
-	virtual bool get_page(uintptr_t offset, uintptr_t &phys) = 0;
+	/// Get the mapper object that maps this area to address spaces
+	virtual vm_mapper & mapper() = 0;
+	virtual const vm_mapper & mapper() const = 0;
+
+	/// Check whether allocation at the given offset is allowed
+	virtual bool allowed(uintptr_t offset) const { return true; }
+
+	/// Commit contiguous physical pages to this area
+	/// \arg phys   The physical address of the first page
+	/// \arg offset The offset from the start of this area these pages represent
+	/// \arg count  The number of pages
+	virtual void commit(uintptr_t phys, uintptr_t offset, size_t count);
+
+	/// Uncommit physical pages from this area
+	/// \arg offset The offset from the start of this area these pages represent
+	/// \arg count  The number of pages
+	virtual void uncommit(uintptr_t offset, size_t count);
 
 protected:
 	virtual void on_no_handles() override;
-	bool can_resize(size_t size);
 
 	size_t m_size;
 	vm_flags m_flags;
-	kutil::vector<vm_space*> m_spaces;
-
-	// Initial static space for m_spaces - most areas will never grow
-	// beyond this size, so avoid allocations
-	static constexpr size_t static_size = 2;
-	vm_space *m_vector_static[static_size];
 };
 
 
-/// A shareable but non-allocatable memory area of contiguous physical
-/// addresses (like mmio)
-class vm_area_fixed :
+/// The standard, sharable, user-controllable VMA type
+class vm_area_shared :
 	public vm_area
 {
 public:
 	/// Constructor.
-	/// \arg start Starting physical address of this area
-	/// \arg size  Size of the physical memory area
+	/// \arg size  Initial virtual size of the memory area
 	/// \arg flags Flags for this memory area
-	vm_area_fixed(uintptr_t start, size_t size, vm_flags flags = vm_flags::none);
-	virtual ~vm_area_fixed();
+	vm_area_shared(size_t size, vm_flags flags = vm_flags::none);
+	virtual ~vm_area_shared();
 
-	virtual size_t resize(size_t size) override;
-	virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
+	virtual vm_mapper & mapper() override { return m_mapper; }
+	virtual const vm_mapper & mapper() const override { return m_mapper; }
 
 private:
-	uintptr_t m_start;
+	vm_mapper_multi m_mapper;
 };
 
 
-/// Area that allows open allocation
+/// Area that allows open allocation (eg, kernel heap)
 class vm_area_open :
 	public vm_area
 {
 public:
 	/// Constructor.
 	/// \arg size  Initial virtual size of the memory area
+	/// \arg space The address space this area belongs to
 	/// \arg flags Flags for this memory area
-	vm_area_open(size_t size, vm_flags flags);
-	virtual ~vm_area_open();
+	vm_area_open(size_t size, vm_space &space, vm_flags flags);
 
-	virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
+	virtual vm_mapper & mapper() override { return m_mapper; }
+	virtual const vm_mapper & mapper() const override { return m_mapper; }
+
+	virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
+	virtual void uncommit(uintptr_t offset, size_t count) override;
 
 private:
-	page_tree *m_mapped;
+	vm_mapper_single m_mapper;
 };
 
 
-/// Area that does not track its allocations and thus cannot be shared
-class vm_area_untracked :
+/// Area split into standard-sized segments
+class vm_area_buffers :
+	public vm_area
+{
+public:
+	/// Constructor.
+	/// \arg size      Initial virtual size of the memory area
+	/// \arg space     The address space this area belongs to
+	/// \arg flags     Flags for this memory area
+	/// \arg buf_pages Pages in an individual buffer
+	vm_area_buffers(
+		size_t size,
+		vm_space &space,
+		vm_flags flags,
+		size_t buf_pages);
+
+	/// Get an available stack address
+	uintptr_t get_buffer();
+
+	/// Return a buffer address to the available pool
+	void return_buffer(uintptr_t addr);
+
+	virtual vm_mapper & mapper() override { return m_mapper; }
+	virtual const vm_mapper & mapper() const override { return m_mapper; }
+
+	virtual bool allowed(uintptr_t offset) const override;
+	virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
+	virtual void uncommit(uintptr_t offset, size_t count) override;
+
+private:
+	vm_mapper_single m_mapper;
+	kutil::vector<uintptr_t> m_cache;
+	size_t m_pages;
+	uintptr_t m_next;
+};
+
+
+/// Area backed by an external source (like a loaded program)
+class vm_area_backed :
 	public vm_area
 {
 public:
 	/// Constructor.
 	/// \arg size  Initial virtual size of the memory area
 	/// \arg flags Flags for this memory area
-	vm_area_untracked(size_t size, vm_flags flags);
-	virtual ~vm_area_untracked();
+	vm_area_backed(size_t size, vm_flags flags);
 
-	virtual bool add_to(vm_space *space) override;
-	virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
-};
+	virtual vm_mapper & mapper() override { return m_mapper; }
+	virtual const vm_mapper & mapper() const override { return m_mapper; }
 
-
-/// Area split into standard-sized segments, separated by guard pages.
-/// Based on vm_area_untracked, can not be shared.
-class vm_area_guarded :
-	public vm_area_untracked
-{
-public:
-	/// Constructor.
-	/// \arg start     Initial address where this area is mapped
-	/// \arg sec_pages Pages in an individual section
-	/// \arg size      Initial virtual size of the memory area
-	/// \arg flags     Flags for this memory area
-	vm_area_guarded(
-		uintptr_t start,
-		size_t sec_pages,
-		size_t size,
-		vm_flags flags);
-
-	virtual ~vm_area_guarded();
-
-	/// Get an available section in this area
-	uintptr_t get_section();
-
-	/// Return a section address to the available pool
-	void return_section(uintptr_t addr);
-
-	virtual bool get_page(uintptr_t offset, uintptr_t &phys) override;
+	virtual void commit(uintptr_t phys, uintptr_t offset, size_t count) override;
+	virtual void uncommit(uintptr_t offset, size_t count) override;
 
 private:
-	kutil::vector<uintptr_t> m_cache;
-	uintptr_t m_start;
-	size_t m_pages;
-	uintptr_t m_next;
+	vm_mapper_multi m_mapper;
 };
 
-
 IS_BITFIELD(vm_flags);

src/kernel/page_tree.cpp

@@ -1,152 +0,0 @@
-#include "kutil/assert.h"
-#include "kutil/memory.h"
-#include "frame_allocator.h"
-#include "page_tree.h"
-
-// Page tree levels map the following parts of a pagewise offset:
-// (Note that a level 0's entries are physical page addrs, the rest
-// map other page_tree nodes)
-//
-// Level 0: 0000000003f    64 pages / 256 KiB 
-// Level 1: 00000000fc0    4K pages /  16 MiB
-// Level 2: 0000003f000  256K pages /   1 GiB
-// Level 3: 00000fc0000   16M pages /  64 GiB
-// Level 4: 0003f000000    1G pages /   4 TiB
-// Level 5: 00fc0000000   64G pages / 256 TiB
-// Level 6: 3f000000000    4T pages /  16 PiB -- Not supported until 5-level paging
-
-static constexpr unsigned max_level = 5;
-static constexpr unsigned bits_per_level = 6;
-
-inline uint64_t to_word(uint64_t base, uint64_t level, uint64_t flags = 0) {
-	// Clear out the non-appropriate bits for this level
-	base &= (~0x3full << (level*bits_per_level));
-
-	return
-		(base & 0x3ffffffffff) |
-		((level & 0x7) << 42) |
-		((flags & 0x7ffff) << 45);
-}
-
-inline uint64_t to_base(uint64_t word) {
-	return word & 0x3ffffffffff;
-}
-
-inline uint64_t to_level(uint64_t word) {
-	return (word >> 42) & 0x3f;
-}
-
-inline uint64_t to_flags(uint64_t word) {
-	return (word >> 45);
-}
-
-inline bool contains(uint64_t page_off, uint64_t word, uint8_t &index) {
-	uint64_t base = to_base(word);
-	uint64_t bits = to_level(word) * bits_per_level;
-	index = (page_off >> bits) & 0x3f;
-	return (page_off & (~0x3full << bits)) == base;
-}
-
-inline uint64_t index_for(uint64_t page_off, uint8_t level) {
-	return (page_off >> (level*bits_per_level))  & 0x3f;
-}
-
-page_tree::page_tree(uint64_t base, uint8_t level) :
-	m_base {to_word(base, level)}
-{
-	kutil::memset(m_entries, 0, sizeof(m_entries));
-}
-
-bool
-page_tree::find(const page_tree *root, uint64_t offset, uintptr_t &page)
-{
-	uint64_t page_off = offset >> 12; // change to pagewise offset
-	page_tree const *node = root;
-	while (node) {
-		uint8_t level = to_level(node->m_base);
-		uint8_t index = 0;
-		if (!contains(page_off, node->m_base, index))
-			return false;
-
-		if (!level) {
-			uintptr_t entry = node->m_entries[index].entry;
-			page = entry & ~1ull; // bit 0 marks 'present'
-			return (entry & 1);
-		}
-
-		node = node->m_entries[index].child;
-	}
-
-	return false;
-}
-
-bool
-page_tree::find_or_add(page_tree * &root, uint64_t offset, uintptr_t &page)
-{
-	uint64_t page_off = offset >> 12; // change to pagewise offset
-	page_tree *level0 = nullptr;
-
-	if (!root) {
-		// There's no root yet, just make a level0 and make it
-		// the root.
-		level0 = new page_tree(page_off, 0);
-		root = level0;
-	} else {
-		// Find or insert an existing level0
-		page_tree **parent = &root;
-		page_tree *node = root;
-		uint8_t parent_level = max_level + 1;
-
-		while (node) {
-			uint8_t level = to_level(node->m_base);
-			uint8_t index = 0;
-			if (!contains(page_off, node->m_base, index)) {
-				// We found a valid parent but the slot where this node should
-				// go contains another node. Insert an intermediate parent of
-				// this node and a new level0 into the parent.
-				uint64_t other = to_base(node->m_base);
-				uint8_t lcl = parent_level;
-				while (index_for(page_off, lcl) == index_for(other, lcl))
-					--lcl;
-
-				page_tree *inter = new page_tree(page_off, lcl);
-				inter->m_entries[index_for(other, lcl)].child = node;
-				*parent = inter;
-
-				level0 = new page_tree(page_off, 0);
-				inter->m_entries[index_for(page_off, lcl)].child = level0;
-				break;
-			}
-
-			if (!level) {
-				level0 = node;
-				break;
-			}
-
-			parent = &node->m_entries[index].child;
-			node = *parent;
-		}
-
-		kassert( node || parent, "Both node and parent were null in find_or_add");
-
-		if (!node) {
-			// We found a parent with an empty spot where this node should
-			// be. Insert a new level0 there.
-			level0 = new page_tree(page_off, 0);
-			*parent = level0;
-		}
-	}
-
-	kassert(level0, "Got through find_or_add without a level0");
-	uint8_t index = index_for(page_off, 0);
-	uint64_t &ent = level0->m_entries[index].entry;
-	if (!(ent & 1)) {
-		// No entry for this page exists, so make one
-		if (!frame_allocator::get().allocate(1, &ent))
-			return false;
-		ent |= 1;
-	}
-
-	page = ent & ~0xfffull;
-	return true;
-}

src/kernel/page_tree.h

@@ -1,39 +0,0 @@
-#pragma once
-/// \file page_tree.h
-/// Definition of mapped page tracking structure and related definitions
-
-#include <stdint.h>
-
-/// A radix tree node that tracks mapped pages
-class page_tree
-{
-public:
-	/// Get the physical address of the page at the given offset.
-	/// \arg root    The root node of the tree
-	/// \arg offset  Offset into the VMA, in bytes
-	/// \arg page    [out] Receives the page physical address, if found
-	/// \returns     True if a page was found
-	static bool find(const page_tree *root, uint64_t offset, uintptr_t &page);
-
-	/// Get the physical address of the page at the given offset. If one does
-	/// not exist yet, allocate a page, insert it, and return that.
-	/// \arg root    [inout] The root node of the tree. This pointer may be updated.
-	/// \arg offset  Offset into the VMA, in bytes
-	/// \arg page    [out] Receives the page physical address, if found
-	/// \returns     True if a page was found
-	static bool find_or_add(page_tree * &root, uint64_t offset, uintptr_t &page);
-
-private:
-	page_tree(uint64_t base, uint8_t level);
-
-	/// Stores the page offset of the start of this node's pages in bits 0:41
-	/// and the depth of tree this node represents in bits 42:44 (0-7)
-	uint64_t m_base;
-
-	/// For a level 0 node, the entries area all physical page addresses.
-	/// Other nodes contain pointers to child tree nodes.
-	union {
-		uintptr_t entry;
-		page_tree *child;
-	} m_entries[64];
-};

src/kernel/scheduler.cpp

@@ -7,7 +7,6 @@
 #include "console.h"
 #include "cpu.h"
 #include "debug.h"
-#include "device_manager.h"
 #include "gdt.h"
 #include "interrupts.h"
 #include "io.h"
@@ -17,7 +16,6 @@
 #include "objects/channel.h"
 #include "objects/process.h"
 #include "objects/system.h"
-#include "objects/thread.h"
 #include "objects/vm_area.h"
 #include "scheduler.h"
 
@@ -26,37 +24,47 @@
 
 #include "kutil/assert.h"
 
-extern "C" void task_switch(TCB *tcb);
+
 scheduler *scheduler::s_instance = nullptr;
 
-struct run_queue
-{
-	tcb_node *current = nullptr;
-	tcb_list ready[scheduler::num_priorities];
-	tcb_list blocked;
+extern kernel::args::framebuffer *fb;
 
-	uint64_t last_promotion = 0;
-	uint64_t last_steal = 0;
-	kutil::spinlock lock;
+const uint64_t rflags_noint = 0x002;
+const uint64_t rflags_int = 0x202;
+
+extern "C" {
+	void preloaded_process_init();
+	uintptr_t load_process_image(uintptr_t phys, uintptr_t virt, size_t bytes, TCB *tcb);
 };
 
-scheduler::scheduler(unsigned cpus) :
-	m_next_pid {1},
-	m_clock {0}
+extern uint64_t idle_stack_end;
+
+scheduler::scheduler(lapic *apic) :
+	m_apic(apic),
+	m_next_pid(1),
+	m_clock(0),
+	m_last_promotion(0)
 {
-	kassert(!s_instance, "Created multiple schedulers!");
-	if (!s_instance)
+	kassert(!s_instance, "Multiple schedulers created!");
 	s_instance = this;
 
-	m_run_queues.set_size(cpus);
-}
+	process *kp = &process::kernel_process();
 
-scheduler::~scheduler()
-{
-	// Not truly necessary - if the scheduler is going away, the whole
-	// system is probably going down. But let's be clean.
-	if (s_instance == this)
-		s_instance = nullptr;
+	log::debug(logs::task, "Kernel process koid %llx", kp->koid());
+
+	thread *idle = thread::create_idle_thread(*kp, max_priority,
+		reinterpret_cast<uintptr_t>(&idle_stack_end));
+
+	log::debug(logs::task, "Idle thread koid %llx", idle->koid());
+
+	auto *tcb = idle->tcb();
+	m_runlists[max_priority].push_back(tcb);
+	m_current = tcb;
+
+	bsp_cpu_data.rsp0 = tcb->rsp0;
+	bsp_cpu_data.tcb = tcb;
+	bsp_cpu_data.p = kp;
+	bsp_cpu_data.t = idle;
 }
 
 template <typename T>
@@ -67,6 +75,139 @@ inline T * push(uintptr_t &rsp, size_t size = sizeof(T)) {
 	return p;
 }
 
+uintptr_t
+load_process_image(uintptr_t phys, uintptr_t virt, size_t bytes, TCB *tcb)
+{
+	using memory::page_align_down;
+	using memory::page_align_up;
+
+	// We're now in the process space for this process, allocate memory for the
+	// process code and load it
+	process &proc = process::current();
+	thread &th = thread::current();
+	vm_space &space = proc.space();
+
+	vm_area *vma = new vm_area_open(bytes, space, vm_flags::zero|vm_flags::write);
+	space.add(virt, vma);
+	vma->commit(phys, 0, memory::page_count(bytes));
+
+	// double zero stack sentinel
+	*push<uint64_t>(tcb->rsp3) = 0;
+	*push<uint64_t>(tcb->rsp3) = 0;
+
+	const char message[] = "Hello from the kernel!";
+	char *message_arg = push<char>(tcb->rsp3, sizeof(message));
+	kutil::memcpy(message_arg, message, sizeof(message));
+
+	j6_init_framebuffer *fb_desc = push<j6_init_framebuffer>(tcb->rsp3);
+	fb_desc->addr = fb ? reinterpret_cast<void*>(0x100000000) : nullptr;
+	fb_desc->size = fb ? fb->size : 0;
+	fb_desc->vertical = fb ? fb->vertical : 0;
+	fb_desc->horizontal = fb ? fb->horizontal : 0;
+	fb_desc->scanline = fb ? fb->scanline : 0;
+	fb_desc->flags = 0;
+
+	if (fb && fb->type == kernel::args::fb_type::bgr8)
+		fb_desc->flags |= 1;
+
+	j6_init_value *initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_system;
+	initv->value = static_cast<uint64_t>(proc.add_handle(system::get()));
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_process;
+	initv->value = static_cast<uint64_t>(proc.self_handle());
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_thread;
+	initv->value = static_cast<uint64_t>(th.self_handle());
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_handle_space;
+	//initv->value = static_cast<uint64_t>(proc.add_handle(&space));
+
+	initv = push<j6_init_value>(tcb->rsp3);
+	initv->type = j6_init_desc_framebuffer;
+	initv->value = reinterpret_cast<uint64_t>(fb_desc);
+
+	uint64_t *initc = push<uint64_t>(tcb->rsp3);
+	*initc = 5;
+
+	char **argv0 = push<char*>(tcb->rsp3);
+	*argv0 = message_arg;
+
+	uint64_t *argc = push<uint64_t>(tcb->rsp3);
+	*argc = 1;
+
+	// Crazypants framebuffer part
+	if (fb) {
+		vma = new vm_area_open(fb->size, space, vm_flags::write|vm_flags::mmio|vm_flags::write_combine);
+		space.add(0x100000000, vma);
+		vma->commit(fb->phys_addr, 0, memory::page_count(fb->size));
+	}
+
+	th.clear_state(thread::state::loading);
+	return tcb->rsp3;
+}
+
+thread *
+scheduler::create_process(bool user)
+{
+	process *p = new process;
+	thread *th = p->create_thread(default_priority, user);
+
+	auto *tcb = th->tcb();
+	tcb->time_left = quantum(default_priority);
+
+	log::debug(logs::task, "Creating thread %llx, priority %d, time slice %d",
+			th->koid(), tcb->priority, tcb->time_left);
+
+	th->set_state(thread::state::ready);
+	return th;
+}
+
+thread *
+scheduler::load_process(uintptr_t phys, uintptr_t virt, size_t size, uintptr_t entry)
+{
+
+	uint16_t kcs = (1 << 3) | 0; // Kernel CS is GDT entry 1, ring 0
+	uint16_t cs = (5 << 3) | 3;  // User CS is GDT entry 5, ring 3
+
+	uint16_t kss = (2 << 3) | 0; // Kernel SS is GDT entry 2, ring 0
+	uint16_t ss = (4 << 3) | 3;  // User SS is GDT entry 4, ring 3
+
+	thread* th = create_process(true);
+	auto *tcb = th->tcb();
+
+	// Create an initial kernel stack space
+	uintptr_t *stack = reinterpret_cast<uintptr_t *>(tcb->rsp0) - 9;
+
+	// Pass args to preloaded_process_init on the stack
+	stack[0] = reinterpret_cast<uintptr_t>(phys);
+	stack[1] = reinterpret_cast<uintptr_t>(virt);
+	stack[2] = reinterpret_cast<uintptr_t>(size);
+	stack[3] = reinterpret_cast<uintptr_t>(tcb);
+
+	tcb->rsp = reinterpret_cast<uintptr_t>(stack);
+	th->add_thunk_kernel(reinterpret_cast<uintptr_t>(preloaded_process_init));
+
+	// Arguments for iret - rip will be pushed on before these
+	stack[4] = reinterpret_cast<uintptr_t>(entry);
+	stack[5] = cs;
+	stack[6] = rflags_int | (3 << 12);
+	stack[7] = process::stacks_top;
+	stack[8] = ss;
+
+	tcb->rsp3 = process::stacks_top;
+
+	log::debug(logs::task, "Loading thread %llx  pri %d", th->koid(), tcb->priority);
+	log::debug(logs::task, "     RSP  %016lx", tcb->rsp);
+	log::debug(logs::task, "     RSP0 %016lx", tcb->rsp0);
+	log::debug(logs::task, "     PML4 %016lx", tcb->pml4);
+
+	return th;
+}
+
 void
 scheduler::create_kernel_task(void (*task)(), uint8_t priority, bool constant)
 {
@@ -96,42 +237,17 @@ scheduler::quantum(int priority)
 void
 scheduler::start()
 {
-	cpu_data &cpu = current_cpu();
-	run_queue &queue = m_run_queues[cpu.index];
-	kutil::scoped_lock lock {queue.lock};
-
-	process *kp = &process::kernel_process();
-	thread *idle = thread::create_idle_thread(*kp, max_priority, cpu.rsp0);
-	log::debug(logs::task, "CPU%02x idle thread koid %llx", cpu.index, idle->koid());
-
-	auto *tcb = idle->tcb();
-	cpu.process = kp;
-	cpu.thread = idle;
-	cpu.tcb = tcb;
-
-	queue.current = tcb;
-
-	log::info(logs::sched, "CPU%02x starting scheduler", cpu.index);
-	cpu.apic->enable_timer(isr::isrTimer, false);
-	cpu.apic->reset_timer(10);
+	log::info(logs::task, "Starting scheduler.");
+	wrmsr(msr::ia32_gs_base, reinterpret_cast<uintptr_t>(&bsp_cpu_data));
+	m_apic->enable_timer(isr::isrTimer, false);
+	m_apic->reset_timer(10);
 }
 
-void
-scheduler::add_thread(TCB *t)
-{
-	cpu_data &cpu = current_cpu();
-	run_queue &queue = m_run_queues[cpu.index];
-	kutil::scoped_lock lock {queue.lock};
-
-	queue.blocked.push_back(static_cast<tcb_node*>(t));
-	t->time_left = quantum(t->priority);
-}
-
-void scheduler::prune(run_queue &queue, uint64_t now)
+void scheduler::prune(uint64_t now)
 {
 	// Find processes that are ready or have exited and
 	// move them to the appropriate lists.
-	auto *tcb = queue.blocked.front();
+	auto *tcb = m_blocked.front();
 	while (tcb) {
 		thread *th = thread::from_tcb(tcb);
 		uint8_t priority = tcb->priority;
@@ -139,7 +255,7 @@ void scheduler::prune(run_queue &queue, uint64_t now)
 		bool ready = th->has_state(thread::state::ready);
 		bool exited = th->has_state(thread::state::exited);
 		bool constant = th->has_state(thread::state::constant);
-		bool current = tcb == queue.current;
+		bool current = tcb == m_current;
 
 		ready |= th->wake_on_time(now);
 
@@ -154,7 +270,7 @@ void scheduler::prune(run_queue &queue, uint64_t now)
 			// page tables
 			if (current) continue;
 
-			queue.blocked.remove(remove);
+			m_blocked.remove(remove);
 			process &p = th->parent();
 
 			// thread_exited deletes the thread, and returns true if the process
@@ -162,19 +278,19 @@ void scheduler::prune(run_queue &queue, uint64_t now)
 			if(!current && p.thread_exited(th))
 				delete &p;
 		} else {
-			queue.blocked.remove(remove);
-			log::debug(logs::sched, "Prune: readying unblocked thread %llx", th->koid());
-			queue.ready[remove->priority].push_back(remove);
+			m_blocked.remove(remove);
+			log::debug(logs::task, "Prune: readying unblocked thread %llx", th->koid());
+			m_runlists[remove->priority].push_back(remove);
 		}
 	}
 }
 
 void
-scheduler::check_promotions(run_queue &queue, uint64_t now)
+scheduler::check_promotions(uint64_t now)
 {
-	for (auto &pri_list : queue.ready) {
+	for (auto &pri_list : m_runlists) {
 		for (auto *tcb : pri_list) {
-			const thread *th = thread::from_tcb(queue.current);
+			const thread *th = thread::from_tcb(m_current);
 			const bool constant = th->has_state(thread::state::constant);
 			if (constant)
 				continue;
@@ -189,145 +305,80 @@ scheduler::check_promotions(run_queue &queue, uint64_t now)
 
 			if (stale) {
 				// If the thread is stale, promote it
-				queue.ready[priority].remove(tcb);
+				m_runlists[priority].remove(tcb);
 				tcb->priority -= 1;
 				tcb->time_left = quantum(tcb->priority);
-				queue.ready[tcb->priority].push_back(tcb);
-				log::info(logs::sched, "Scheduler promoting thread %llx, priority %d",
+				m_runlists[tcb->priority].push_back(tcb);
+				log::debug(logs::task, "Scheduler promoting thread %llx, priority %d",
 						th->koid(), tcb->priority);
 			}
 		}
 	}
 
-	queue.last_promotion = now;
-}
-
-static size_t
-balance_lists(tcb_list &to, tcb_list &from)
-{
-	size_t to_len = to.length();
-	size_t from_len = from.length();
-
-	// Only steal from the rich, don't be Dennis Moore
-	if (from_len <= to_len)
-		return 0;
-
-	size_t steal = (from_len - to_len) / 2;
-	for (size_t i = 0; i < steal; ++i)
-		to.push_front(from.pop_front());
-	return steal;
-}
-
-void
-scheduler::steal_work(cpu_data &cpu)
-{
-	// First grab a scheduler-wide lock to avoid deadlock
-	kutil::scoped_lock steal_lock {m_steal_lock};
-
-	// Lock this cpu's queue for the whole time while we modify it
-	run_queue &my_queue = m_run_queues[cpu.index];
-	kutil::scoped_lock my_queue_lock {my_queue.lock};
-
-	const unsigned count = m_run_queues.count();
-	for (unsigned i = 0; i < count; ++i) {
-		if (i == cpu.index) continue;
-
-		run_queue &other_queue = m_run_queues[i];
-		kutil::scoped_lock other_queue_lock {other_queue.lock};
-
-		size_t stolen = 0;
-
-		// Don't steal from max_priority, that's the idle thread
-		for (unsigned pri = 0; pri < max_priority; ++pri)
-			stolen += balance_lists(my_queue.ready[pri], other_queue.ready[pri]);
-
-		stolen += balance_lists(my_queue.blocked, other_queue.blocked);
-
-		if (stolen)
-			log::debug(logs::sched, "CPU%02x stole %2d tasks from CPU%02x",
-					cpu.index, stolen, i);
-	}
+	m_last_promotion = now;
 }
 
 void
 scheduler::schedule()
 {
-	cpu_data &cpu = current_cpu();
-	run_queue &queue = m_run_queues[cpu.index];
-	lapic &apic = *cpu.apic;
-	uint32_t remaining = apic.stop_timer();
-
-	if (m_clock - queue.last_steal > steal_frequency) {
-		steal_work(cpu);
-		queue.last_steal = m_clock;
-	}
-
-	// We need to explicitly lock/unlock here instead of
-	// using a scoped lock, because the scope doesn't "end"
-	// for the current thread until it gets scheduled again
-	kutil::spinlock::waiter waiter;
-	queue.lock.acquire(&waiter);
-
-	queue.current->time_left = remaining;
-	thread *th = thread::from_tcb(queue.current);
-	uint8_t priority = queue.current->priority;
+	uint8_t priority = m_current->priority;
+	uint32_t remaining = m_apic->stop_timer();
+	m_current->time_left = remaining;
+	thread *th = thread::from_tcb(m_current);
 	const bool constant = th->has_state(thread::state::constant);
 
 	if (remaining == 0) {
 		if (priority < max_priority && !constant) {
 			// Process used its whole timeslice, demote it
-			++queue.current->priority;
-			log::debug(logs::sched, "Scheduler  demoting thread %llx, priority %d",
-					th->koid(), queue.current->priority);
+			++m_current->priority;
+			log::debug(logs::task, "Scheduler  demoting thread %llx, priority %d",
+					th->koid(), m_current->priority);
 		}
-		queue.current->time_left = quantum(queue.current->priority);
+		m_current->time_left = quantum(m_current->priority);
 	} else if (remaining > 0) {
 		// Process gave up CPU, give it a small bonus to its
 		// remaining timeslice.
 		uint32_t bonus = quantum(priority) >> 4;
-		queue.current->time_left += bonus;
+		m_current->time_left += bonus;
 	}
 
+	m_runlists[priority].remove(m_current);
 	if (th->has_state(thread::state::ready)) {
-		queue.ready[queue.current->priority].push_back(queue.current);
+		m_runlists[m_current->priority].push_back(m_current);
 	} else {
-		queue.blocked.push_back(queue.current);
+		m_blocked.push_back(m_current);
 	}
 
 	clock::get().update();
-	prune(queue, ++m_clock);
-	if (m_clock - queue.last_promotion > promote_frequency)
-		check_promotions(queue, m_clock);
+	prune(++m_clock);
+	if (m_clock - m_last_promotion > promote_frequency)
+		check_promotions(m_clock);
 
 	priority = 0;
-	while (queue.ready[priority].empty()) {
+	while (m_runlists[priority].empty()) {
 		++priority;
 		kassert(priority < num_priorities, "All runlists are empty");
 	}
 
-	queue.current->last_ran = m_clock;
+	m_current->last_ran = m_clock;
 
-	auto *next = queue.ready[priority].pop_front();
+	auto *next = m_runlists[priority].pop_front();
 	next->last_ran = m_clock;
-	apic.reset_timer(next->time_left);
-
-	if (next == queue.current) {
-		queue.lock.release(&waiter);
-		return;
-	}
+	m_apic->reset_timer(next->time_left);
 
+	if (next != m_current) {
 		thread *next_thread = thread::from_tcb(next);
 
-	cpu.thread = next_thread;
-	cpu.process = &next_thread->parent();
-	queue.current = next;
+		bsp_cpu_data.t = next_thread;
+		bsp_cpu_data.p = &next_thread->parent();
+		m_current = next;
 
-	log::debug(logs::sched, "CPU%02x switching threads %llx->%llx",
-			cpu.index, th->koid(), next_thread->koid());
-	log::debug(logs::sched, "    priority %d time left %d @ %lld.",
-			next->priority, next->time_left, m_clock);
-	log::debug(logs::sched, "    PML4 %llx", next->pml4);
+		log::debug(logs::task, "Scheduler switching threads %llx->%llx",
+				th->koid(), next_thread->koid());
+		log::debug(logs::task, "    priority %d time left %d @ %lld.",
+				m_current->priority, m_current->time_left, m_clock);
+		log::debug(logs::task, "    PML4 %llx", m_current->pml4);
 
-	queue.lock.release(&waiter);
-	task_switch(queue.current);
+		task_switch(m_current);
+	}
 }

src/kernel/scheduler.h

@@ -3,19 +3,15 @@
 /// The task scheduler and related definitions
 
 #include <stdint.h>
-#include "kutil/spinlock.h"
-#include "kutil/vector.h"
+#include "objects/thread.h"
 
-namespace kernel {
-namespace args {
-	struct program;
-}}
-
-struct cpu_data;
 class lapic;
 class process;
 struct page_table;
-struct run_queue;
+struct cpu_state;
+
+extern "C" void isr_handler(cpu_state*);
+extern "C" void task_switch(TCB *next);
 
 
 /// The task scheduler
@@ -41,14 +37,16 @@ public:
 	static const uint16_t process_quanta = 10;
 
 	/// Constructor.
-	/// \arg cpus  The number of CPUs to schedule for
-	scheduler(unsigned cpus);
-	~scheduler();
+	/// \arg apic  Pointer to the local APIC object
+	scheduler(lapic *apic);
 
 	/// Create a new process from a program image in memory.
-	/// \arg program  The descriptor of the pogram in memory
+	/// \arg phys  Physical address of the loaded program image
+	/// \arg virt  Virtual address of the loaded program image
+	/// \arg size  Size of the program image, in bytes
+	/// \arg entry Virtual address of the program entrypoint
 	/// \returns   The main thread of the loaded process
-	thread * load_process(kernel::args::program &program);
+	thread * load_process(uintptr_t phys, uintptr_t virt, size_t size, uintptr_t entry);
 
 	/// Create a new kernel task
 	/// \arg proc     Function to run as a kernel task
@@ -60,7 +58,7 @@ public:
 			bool constant = false);
 
 	/// Get the quantum for a given priority.
-	static uint32_t quantum(int priority);
+	uint32_t quantum(int priority);
 
 	/// Start the scheduler working. This may involve starting
 	/// timer interrupts or other preemption methods.
@@ -69,35 +67,45 @@ public:
 	/// Run the scheduler, possibly switching to a new task
 	void schedule();
 
-	/// Start scheduling a new thread.
-	/// \arg t  The new thread's TCB
-	void add_thread(TCB *t);
+	/// Get the current TCB.
+	/// \returns  A pointer to the current thread's TCB
+	inline TCB * current() { return m_current; }
 
-	/// Get a reference to the scheduler
+	inline void add_thread(TCB *t) { m_blocked.push_back(static_cast<tcb_node*>(t)); }
+
+	/// Get a reference to the system scheduler
 	/// \returns  A reference to the global system scheduler
 	static scheduler & get() { return *s_instance; }
 
 private:
+	friend uintptr_t syscall_dispatch(uintptr_t, cpu_state &);
 	friend class process;
 
 	static constexpr uint64_t promote_frequency = 10;
-	static constexpr uint64_t steal_frequency = 10;
 
-	void prune(run_queue &queue, uint64_t now);
-	void check_promotions(run_queue &queue, uint64_t now);
-	void steal_work(cpu_data &cpu);
+	/// Create a new process object. This process will have its pid
+	/// set but nothing else.
+	/// \arg user True if this thread will enter userspace
+	/// \returns  The new process' main thread
+	thread * create_process(bool user);
+
+	void prune(uint64_t now);
+	void check_promotions(uint64_t now);
+
+	lapic *m_apic;
 
 	uint32_t m_next_pid;
 	uint32_t m_tick_count;
 
 	process *m_kernel_process;
-
-	kutil::vector<run_queue> m_run_queues;
+	tcb_node *m_current;
+	tcb_list m_runlists[num_priorities];
+	tcb_list m_blocked;
 
 	// TODO: lol a real clock
 	uint64_t m_clock = 0;
+	uint64_t m_last_promotion;
 
-	kutil::spinlock m_steal_lock;
 	static scheduler *s_instance;
 };
 

src/kernel/syscall.cpp

@@ -1,19 +1,24 @@
 #include <stddef.h>
 
-#include "kutil/memory.h"
-
 #include "console.h"
+#include "cpu.h"
 #include "debug.h"
 #include "log.h"
+#include "msr.h"
+#include "scheduler.h"
 #include "syscall.h"
 
 extern "C" {
 	void syscall_invalid(uint64_t call);
+	void syscall_handler_prelude();
 }
 
-uintptr_t syscall_registry[256] __attribute__((section(".syscall_registry")));
-const char * syscall_names[256] __attribute__((section(".syscall_registry")));
-static constexpr size_t num_syscalls = sizeof(syscall_registry) / sizeof(syscall_registry[0]);
+namespace syscalls {
+
+} // namespace syscalls
+
+uintptr_t syscall_registry[static_cast<unsigned>(syscall::MAX)];
+const char * syscall_names[static_cast<unsigned>(syscall::MAX)];
 
 void
 syscall_invalid(uint64_t call)
@@ -22,10 +27,13 @@ syscall_invalid(uint64_t call)
 	cons->set_color(9);
 	cons->printf("\nReceived unknown syscall: %02x\n", call);
 
+	const unsigned num_calls =
+		static_cast<unsigned>(syscall::MAX);
+
 	cons->printf("  Known syscalls:\n");
 		cons->printf("          invalid %016lx\n", syscall_invalid);
 
-	for (unsigned i = 0; i < num_syscalls; ++i) {
+	for (unsigned i = 0; i < num_calls; ++i) {
 		const char *name = syscall_names[i];
 		uintptr_t handler = syscall_registry[i];
 		if (name)
@@ -36,17 +44,113 @@ syscall_invalid(uint64_t call)
 	_halt();
 }
 
+/*
 void
-syscall_initialize()
+syscall_dispatch(cpu_state *regs)
 {
+	console *cons = console::get();
+	syscall call = static_cast<syscall>(regs->rax);
+
+	auto &s = scheduler::get();
+	auto *p = s.current();
+
+	switch (call) {
+	case syscall::noop:
+		break;
+
+	case syscall::debug:
+		cons->set_color(11);
+		cons->printf("\nProcess %d: Received DEBUG syscall\n", p->pid);
+		cons->set_color();
+		print_regs(*regs);
+		cons->printf("\n         Syscall enters: %8d\n", __counter_syscall_enter);
+		cons->printf("         Syscall sysret: %8d\n", __counter_syscall_sysret);
+		break;
+
+	case syscall::send:
+		{
+			pid_t target = regs->rdi;
+			uintptr_t data = regs->rsi;
+
+			cons->set_color(11);
+			cons->printf("\nProcess %d: Received SEND syscall, target %d, data %016lx\n", p->pid, target, data);
+			cons->set_color();
+
+			if (p->wait_on_send(target))
+				s.schedule();
+		}
+		break;
+
+	case syscall::receive:
+		{
+			pid_t source = regs->rdi;
+			uintptr_t data = regs->rsi;
+
+			cons->set_color(11);
+			cons->printf("\nProcess %d: Received RECEIVE syscall, source %d, dat %016lx\n", p->pid, source, data);
+			cons->set_color();
+
+			if (p->wait_on_receive(source))
+				s.schedule();
+		}
+		break;
+
+	case syscall::fork:
+		{
+			cons->set_color(11);
+			cons->printf("\nProcess %d: Received FORK syscall\n", p->pid);
+			cons->set_color();
+
+			pid_t pid = p->fork(regs);
+			cons->printf("\n   fork returning %d\n", pid);
+			regs->rax = pid;
+		}
+		break;
+
+	default:
+		cons->set_color(9);
+		cons->printf("\nReceived unknown syscall: %02x\n", call);
+		cons->set_color();
+		_halt();
+		break;
+	}
+}
+*/
+
+void
+syscall_enable()
+{
+	// IA32_EFER - set bit 0, syscall enable
+	uint64_t efer = rdmsr(msr::ia32_efer);
+	wrmsr(msr::ia32_efer, efer|1);
+
+	// IA32_STAR - high 32 bits contain k+u CS
+	// Kernel CS: GDT[1] ring 0 bits[47:32]
+	//   User CS: GDT[3] ring 3 bits[63:48]
+	uint64_t star =
+		(((1ull << 3) | 0) << 32) |
+		(((3ull << 3) | 3) << 48);
+	wrmsr(msr::ia32_star, star);
+
+	// IA32_LSTAR - RIP for syscall
+	wrmsr(msr::ia32_lstar,
+		reinterpret_cast<uintptr_t>(&syscall_handler_prelude));
+
+	// IA32_FMASK - FLAGS mask inside syscall
+	wrmsr(msr::ia32_fmask, 0x200);
+
+	static constexpr unsigned num_calls =
+		static_cast<unsigned>(syscall::MAX);
+
 	kutil::memset(&syscall_registry, 0, sizeof(syscall_registry));
 	kutil::memset(&syscall_names, 0, sizeof(syscall_names));
 
 #define SYSCALL(id, name, result, ...) \
 	syscall_registry[id] = reinterpret_cast<uintptr_t>(syscalls::name); \
 	syscall_names[id] = #name; \
+	static_assert( id <= num_calls, "Syscall " #name " has id > syscall::MAX" ); \
 	log::debug(logs::syscall, "Enabling syscall 0x%02x as " #name , id);
-#include "j6/tables/syscalls.inc"
+#include "syscalls.inc"
 #undef SYSCALL
 }
 

src/kernel/syscall.h

@@ -7,17 +7,20 @@ struct cpu_state;
 
 enum class syscall : uint64_t
 {
-#define SYSCALL(id, name, ...) name = id,
-#include "j6/tables/syscalls.inc"
+#define SYSCALL(id, name, result, ...) name = id,
+#include "syscalls.inc"
 #undef SYSCALL
+
+	// Maximum syscall id. If you change this, also change
+	// MAX_SYSCALLS in syscall.s
+	MAX = 0x40
 };
 
-void syscall_initialize();
-extern "C" void syscall_enable();
+void syscall_enable();
 
 namespace syscalls
 {
 #define SYSCALL(id, name, ...) j6_status_t name (__VA_ARGS__);
-#include "j6/tables/syscalls.inc"
+#include "syscalls.inc"
 #undef SYSCALL
 }

src/kernel/syscall.s

@@ -1,32 +1,19 @@
+%include "push_all.inc"
 %include "tasking.inc"
 
-; SYSCALL/SYSRET control MSRs
-MSR_STAR   equ 0xc0000081
-MSR_LSTAR  equ 0xc0000082
-MSR_FMASK  equ 0xc0000084
-
-; IA32_STAR - high 32 bits contain k+u CS
-; Kernel CS: GDT[1] ring 0 bits[47:32]
-;   User CS: GDT[3] ring 3 bits[63:48]
-STAR_HIGH  equ \
-	(((1 << 3) | 0)) | \
-	(((3 << 3) | 3) << 16)
-
-; IA32_FMASK - Mask off interrupts in syscalls
-FMASK_VAL  equ 0x200
+; Make sure to keep MAX_SYSCALLS in sync with
+; syscall::MAX in syscall.h
+MAX_SYSCALLS equ 0x40
 
 extern __counter_syscall_enter
 extern __counter_syscall_sysret
+
 extern syscall_registry
 extern syscall_invalid
 
-
-global syscall_handler_prelude:function (syscall_handler_prelude.end - syscall_handler_prelude)
+global syscall_handler_prelude
+global syscall_handler_prelude.return
 syscall_handler_prelude:
-	push rbp     ; Never executed, fake function prelude
-	mov rbp, rsp ; to calm down gdb
-
-.real:
 	swapgs
 	mov [gs:CPU_DATA.rsp3], rsp
 	mov rsp, [gs:CPU_DATA.rsp0]
@@ -51,7 +38,14 @@ syscall_handler_prelude:
 
 	inc qword [rel __counter_syscall_enter]
 
-	and rax, 0xff ; Only 256 possible syscall values
+	cmp rax, MAX_SYSCALLS
+	jle .ok_syscall
+
+.bad_syscall:
+	mov rdi, rax
+	call syscall_invalid
+
+.ok_syscall:
 	lea r11, [rel syscall_registry]
 	mov r11, [r11 + rax * 8]
 	cmp r11, 0
@@ -60,15 +54,8 @@ syscall_handler_prelude:
 	call r11
 
 	inc qword [rel __counter_syscall_sysret]
-	jmp kernel_to_user_trampoline
 
-.bad_syscall:
-	mov rdi, rax
-	call syscall_invalid
-.end:
-
-global kernel_to_user_trampoline:function (kernel_to_user_trampoline.end - kernel_to_user_trampoline)
-kernel_to_user_trampoline:
+.return:
 	pop r15
 	pop r14
 	pop r13
@@ -84,28 +71,3 @@ kernel_to_user_trampoline:
 
 	swapgs
 	o64 sysret
-.end:
-
-global syscall_enable:function (syscall_enable.end - syscall_enable)
-syscall_enable:
-	push rbp
-	mov rbp, rsp
-
-	mov rcx, MSR_STAR
-	mov rax, 0
-	mov rdx, STAR_HIGH
-	wrmsr
-
-	mov rcx, MSR_LSTAR
-	mov rax, syscall_handler_prelude.real
-	mov rdx, rax
-	shr rdx, 32
-	wrmsr
-
-	mov rcx, MSR_FMASK
-	mov rax, FMASK_VAL
-	wrmsr
-
-	pop rbp
-	ret
-.end:

src/kernel/syscalls/endpoint.cpp

@@ -36,7 +36,7 @@ endpoint_receive(j6_handle_t handle, j6_tag_t *tag, size_t *len, void *data)
 	if (!e) return j6_err_invalid_arg;
 
 	j6_tag_t out_tag = j6_tag_invalid;
-	size_t out_len = *len;
+	size_t out_len = 0;
 	j6_status_t s = e->receive(&out_tag, &out_len, data);
 	*tag = out_tag;
 	*len = out_len;
@@ -57,7 +57,7 @@ endpoint_sendrecv(j6_handle_t handle, j6_tag_t *tag, size_t *len, void *data)
 		return status;
 
 	j6_tag_t out_tag = j6_tag_invalid;
-	size_t out_len = *len;
+	size_t out_len = 0;
 	j6_status_t s = e->receive(&out_tag, &out_len, data);
 	*tag = out_tag;
 	*len = out_len;

src/kernel/syscalls/process.cpp

@@ -3,56 +3,18 @@
 
 #include "log.h"
 #include "objects/process.h"
-#include "syscalls/helpers.h"
 
 namespace syscalls {
 
 j6_status_t
-process_create(j6_handle_t *handle)
-{
-	process *child = construct_handle<process>(handle);
-	log::debug(logs::task, "Process %llx created", child->koid());
-	return j6_status_ok;
-}
-
-j6_status_t
-process_start(j6_handle_t handle, uintptr_t entrypoint, j6_handle_t *handles, size_t handle_count)
+process_exit(int64_t status)
 {
 	process &p = process::current();
-	process *c = get_handle<process>(handle);
-	if (handle_count && !handles)
-		return j6_err_invalid_arg;
-
-	for (size_t i = 0; i < handle_count; ++i) {
-		kobject *o = p.lookup_handle(handles[i]);
-		if (o) c->add_handle(o);
-	}
-
-	return j6_err_nyi;
-}
-
-j6_status_t
-process_kill(j6_handle_t handle)
-{
-	process &p = process::current();
-	process *c = get_handle<process>(handle);
-	if (!c) return j6_err_invalid_arg;
-
-	log::debug(logs::task, "Process %llx killed by process %llx", c->koid(), p.koid());
-	c->exit(-1u);
-
-	return j6_status_ok;
-}
-
-j6_status_t
-process_exit(int32_t status)
-{
-	process &p = process::current();
-	log::debug(logs::task, "Process %llx exiting with code %d", p.koid(), status);
+	log::debug(logs::syscall, "Process %llx exiting with code %d", p.koid(), status);
 
 	p.exit(status);
 
-	log::error(logs::task, "returned to exit syscall");
+	log::error(logs::syscall, "returned to exit syscall");
 	return j6_err_unexpected;
 }
 

src/kernel/syscalls/system.cpp

@@ -5,8 +5,6 @@
 #include "log.h"
 #include "objects/endpoint.h"
 #include "objects/thread.h"
-#include "objects/system.h"
-#include "objects/vm_area.h"
 #include "syscalls/helpers.h"
 
 extern log::logger &g_logger;
@@ -33,17 +31,10 @@ system_noop()
 }
 
 j6_status_t
-system_get_log(j6_handle_t sys, void *buffer, size_t *size)
+system_get_log(j6_handle_t sys, char *buffer, size_t *size)
 {
-	if (!size || (*size && !buffer))
-		return j6_err_invalid_arg;
-
-	size_t orig_size = *size;
 	*size = g_logger.get_entry(buffer, *size);
-	if (!g_logger.has_log())
-		system::get().deassert_signal(j6_signal_system_has_log);
-
-	return (*size > orig_size) ? j6_err_insufficient : j6_status_ok;
+	return j6_status_ok;
 }
 
 j6_status_t
@@ -59,16 +50,4 @@ system_bind_irq(j6_handle_t sys, j6_handle_t endp, unsigned irq)
 	return j6_err_invalid_arg;
 }
 
-j6_status_t
-system_map_mmio(j6_handle_t sys, j6_handle_t *vma_handle, uintptr_t phys_addr, size_t size, uint32_t flags)
-{
-	// TODO: check capabilities on sys handle
-	if (!vma_handle) return j6_err_invalid_arg;
-
-	vm_flags vmf = vm_flags::mmio | (static_cast<vm_flags>(flags) & vm_flags::user_mask);
-	construct_handle<vm_area_fixed>(vma_handle, phys_addr, size, vmf);
-
-	return j6_status_ok;
-}
-
 } // namespace syscalls

src/kernel/syscalls/thread.cpp

@@ -15,24 +15,24 @@ thread_create(void *rip, j6_handle_t *handle)
 
 	thread *child = p.create_thread();
 	child->add_thunk_user(reinterpret_cast<uintptr_t>(rip));
-	*handle = child->self_handle();
+	*handle = p.self_handle();
 	child->clear_state(thread::state::loading);
 	child->set_state(thread::state::ready);
 
-	log::debug(logs::task, "Thread %llx spawned new thread %llx, handle %d",
+	log::debug(logs::syscall, "Thread %llx spawned new thread %llx, handle %d",
 		parent.koid(), child->koid(), *handle);
 
 	return j6_status_ok;
 }
 
 j6_status_t
-thread_exit(int32_t status)
+thread_exit(int64_t status)
 {
 	thread &th = thread::current();
-	log::debug(logs::task, "Thread %llx exiting with code %d", th.koid(), status);
+	log::debug(logs::syscall, "Thread %llx exiting with code %d", th.koid(), status);
 	th.exit(status);
 
-	log::error(logs::task, "returned to exit syscall");
+	log::error(logs::syscall, "returned to exit syscall");
 	return j6_err_unexpected;
 }
 
@@ -48,7 +48,7 @@ j6_status_t
 thread_sleep(uint64_t til)
 {
 	thread &th = thread::current();
-	log::debug(logs::task, "Thread %llx sleeping until %llu", th.koid(), til);
+	log::debug(logs::syscall, "Thread %llx sleeping until %llu", th.koid(), til);
 
 	th.wait_on_time(til);
 	return j6_status_ok;

src/kernel/syscalls/vm_area.cpp

@@ -14,7 +14,7 @@ j6_status_t
 vma_create(j6_handle_t *handle, size_t size, uint32_t flags)
 {
 	vm_flags f = vm_flags::user_mask & flags;
-	construct_handle<vm_area_open>(handle, size, f);
+	construct_handle<vm_area_shared>(handle, size, f);
 	return j6_status_ok;
 }
 
@@ -22,34 +22,28 @@ j6_status_t
 vma_create_map(j6_handle_t *handle, size_t size, uintptr_t base, uint32_t flags)
 {
 	vm_flags f = vm_flags::user_mask & flags;
-	vm_area *a = construct_handle<vm_area_open>(handle, size, f);
+	vm_area *a = construct_handle<vm_area_shared>(handle, size, f);
 	process::current().space().add(base, a);
 	return j6_status_ok;
 }
 
 j6_status_t
-vma_map(j6_handle_t handle, j6_handle_t proc, uintptr_t base)
+vma_map(j6_handle_t handle, uintptr_t base)
 {
 	vm_area *a = get_handle<vm_area>(handle);
 	if (!a) return j6_err_invalid_arg;
 
-	process *p = get_handle<process>(proc);
-	if (!p) return j6_err_invalid_arg;
-
-	p->space().add(base, a);
+	process::current().space().add(base, a);
 	return j6_status_ok;
 }
 
 j6_status_t
-vma_unmap(j6_handle_t handle, j6_handle_t proc)
+vma_unmap(j6_handle_t handle)
 {
 	vm_area *a = get_handle<vm_area>(handle);
 	if (!a) return j6_err_invalid_arg;
 
-	process *p = get_handle<process>(proc);
-	if (!p) return j6_err_invalid_arg;
-
-	p->space().remove(a);
+	process::current().space().remove(a);
 	return j6_status_ok;
 }
 

src/kernel/task.s

@@ -1,5 +1,6 @@
 %include "tasking.inc"
 
+extern g_tss
 global task_switch
 task_switch:
 	push rbp
@@ -17,7 +18,7 @@ task_switch:
 	mov [rax + TCB.rsp], rsp
 
 	; Copy off saved user rsp
-	mov rcx, [gs:CPU_DATA.rsp3]    ; rcx: current task's saved user rsp
+	mov rcx, [gs:CPU_DATA.rsp3]    ; rcx: curretn task's saved user rsp
 	mov [rax + TCB.rsp3], rcx
 
 	; Install next task's TCB
@@ -30,7 +31,7 @@ task_switch:
 	mov rcx, [rdi + TCB.rsp0]      ; rcx: top of next task's kernel stack
 	mov [gs:CPU_DATA.rsp0], rcx
 
-	mov rdx, [gs:CPU_DATA.tss]     ; rdx: address of TSS
+	lea rdx, [rel g_tss]           ; rdx: address of TSS
 	mov [rdx + TSS.rsp0], rcx
 
 	; Update saved user rsp
@@ -54,20 +55,8 @@ task_switch:
 	ret
 
 
-extern initialize_main_user_stack
-extern kernel_to_user_trampoline
-global initialize_main_thread
-initialize_main_thread:
-	call initialize_main_user_stack
+extern syscall_handler_prelude.return
+global kernel_to_user_trampoline
+kernel_to_user_trampoline:
+	jmp syscall_handler_prelude.return
 
-	; user rsp is now in rax, put it in the right place for sysret
-	mov [rsp + 0x30], rax
-	mov [gs:CPU_DATA.rsp3], rax
-
-	; the entrypoint should already be on the stack
-	jmp kernel_to_user_trampoline
-
-global _current_gsbase
-_current_gsbase:
-	mov rax, [gs:CPU_DATA.self]
-	ret