Updating README to reflect new build process

Under KVM we were hitting what look like out-of-order and/or issues
during initialization when writing to the page tables and then
immediately writing to the mapped memory.  Adding a memory barrier and
an io_wait() in memory_bootstrap.cpp fixed it.

.gitignore

@@ -1,7 +1,9 @@
 .lock*
-build
+/build*
 *.bak
 tags
 .gdbinit
 popcorn.log
-.waf-*
+*.o
+*.a
+sysroot

LICENSE.md

@@ -2,7 +2,8 @@
 
 Popcorn itself is released under the terms of the MIT license:
 
-> Copyright © 2018 Justin C. Miller, http://devjustinian.com <justin@devjustinian.com>
+> Copyright © 2018 Justin C. Miller, https://devjustinian.com
+> <justin@devjustinian.com>
 > 
 > Permission is hereby granted, free of charge, to any person obtaining a copy
 > of this software and associated documentation files (the “Software”), to deal
@@ -19,8 +20,8 @@ Popcorn itself is released under the terms of the MIT license:
 > FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 > AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 > LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-> THE SOFTWARE.
+> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+> SOFTWARE.
 
 # Included works
 
@@ -28,7 +29,7 @@ Popcorn includes and/or is derived from a number of other works, listed here.
 
 ## Catch2
 
-Popcorn uses [Catch2](http://github.com/catchorg/Catch2) for testing. Catch2 is
+Popcorn uses [Catch2](https://github.com/catchorg/Catch2) for testing. Catch2 is
 released under the terms of the Boost Software license:
 
 > Boost Software License - Version 1.0 - August 17th, 2003
@@ -55,44 +56,30 @@ released under the terms of the Boost Software license:
 > ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 > DEALINGS IN THE SOFTWARE.
 
-## Intel EFI Application Toolkit
+## cpptoml
 
-Popcorn's UEFI loader uses code from Intel's EFI Application toolkit. Relevant
-code includes license statements at the top of each file.
+Popcorn uses the [cpptoml](https://github.com/skystrife/cpptoml) library for
+parsing TOML configuration files. cpptoml is released under the terms of the
+MIT license:
 
-## Waf
-
-Popcorn's build system uses [Waf](https://waf.io/), which claims to be released
-under the BSD license. I could not find its specific license file, so I am
-reproducing a generic 3-clause BSD license (the most restrictive, so as not to
-assume any extra rights that may not actually be granted) for it here:
-
-> Copyright © 2005-2018 Thomas Nagy
+> Copyright (c) 2014 Chase Geigle
+>
+> Permission is hereby granted, free of charge, to any person obtaining a copy of
+> this software and associated documentation files (the "Software"), to deal in
+> the Software without restriction, including without limitation the rights to
+> use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+> the Software, and to permit persons to whom the Software is furnished to do so,
+> subject to the following conditions:
 > 
-> Redistribution and use in source and binary forms, with or without
-> modification, are permitted provided that the following conditions are met:
+> The above copyright notice and this permission notice shall be included in all
+> copies or substantial portions of the Software.
 > 
-> 1. Redistributions of source code must retain the above copyright notice, this
->    list of conditions and the following disclaimer.
-> 
-> 2. Redistributions in binary form must reproduce the above copyright notice,
->    this list of conditions and the following disclaimer in the documentation
->    and/or other materials provided with the distribution.
-> 
-> 3. Neither the name of the copyright holder nor the names of its contributors
->    may be used to endorse or promote products derived from this software
->    without specific prior written permission.
-> 
-> THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-> ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-> WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-> DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-> FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-> DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-> SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-> CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-> OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-> OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+> FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+> COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+> IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+> CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 ## GNU-EFI
 
@@ -130,3 +117,7 @@ to assume any extra rights that may not actually be granted) for it here:
 > OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 > OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
+## Intel EFI Application Toolkit
+
+Popcorn's UEFI loader uses code from Intel's EFI Application toolkit. Relevant
+code includes license statements at the top of each file.

NOTES.md

@@ -5,20 +5,26 @@
 - Better page-allocation model
 - Allow for more than one IOAPIC in ACPI module
   - The objects get created, but GSI lookup only uses the one at index 0
-- Slab allocator for kernel structures
 - mark kernel memory pages global
-- lock `memory_manager` and `page_manager` structures
 - Serial out based on circular/bip biffer and interrupts, not spinning on
   `write_ready()`
 - Split out more code into kutil for testing
+- AHCI / MSI interrupts on Vbox break?
+- FXSAVE to save XMM registers.
+  - optimization using #NM (0x7) to detect SSE usage
+- Clean up of process memory maps
+- Better stack tracer
+- Bootloader rewrite
+  - C++ and sharing library code for ELF, initrd, etc
+  - Parse initrd and pre-load certain ELF images, eg the process loader process?
+  - Do initial memory bootstrap?
+- Calling global ctors
 
 
 - Device Tree
   - Actual serial driver
   - Disk driver
     - File system
-- Memory map swapping
-  - Multiprocessing
-    - Processes in Ring 3
-- Stack tracer
-- build system upgrade (CMake / Waf / etc)
+- Multiprocessing
+  - Syscalls
+

README.md

@@ -27,14 +27,34 @@ The design goals of the project are:
 
 ## Building
 
-Popcorn uses the `waf` build tool, which is included in the repo. The other
-requirements are:
+Popcorn uses the `ninja` build tool, and generates the build files for it with
+the `generate_build.py` script.  The other requirements are:
 
-* python (to run waf)
+* python 3 for generating the build config
+  * The Jinja2 package is also required
 * clang
-* nasm
+* mtools
+* ninja
+* curl for downloading the toolchain
+
+### Setting up the cross toolchain
+
+If you have `clang` and `curl` installed, runing the `scripts/build_sysroot_clang.sh`
+script will download and build a nasm/binutils/LLVM toolchain configured for building
+Popcorn host binaries.
+
+### Building and running Popcorn
+
+Once the toolchain has been set up, running `generate_build.py` will set up the
+build configuration, and `ninja -C build` will actually run the build.  If you
+have `qemu-system-x86_64` installed, the `qemu.sh` script will to run Popcorn
+in QEMU `-nographic` mode.
+
+I personally run this either from a real debian amd64 testing/buster machine or
+a windows WSL debian testing/buster installation. The following should be
+enough to set up such a system to build the kernel:
+
+    sudo apt install qemu-system-x86 nasm clang-6.0 mtools
+	sudo update-alternatives /usr/bin/clang clang /usr/bin/clang-6.0 1000
+	sudo update-alternatives /usr/bin/clang++ clang++ /usr/bin/clang++-6.0 1000
 
-After cloning, run `waf configure`. Then you can run `waf build` to build the
-project, and `waf test` to run the tests. A floppy disk image will be built in
-`build/popcorn.img`. If you have `qemu-system-x86_64` installed, then you can
-run `waf qemu` to run it in `-nographic` mode.

assets/initrd.toml

@@ -0,0 +1,23 @@
+# This is the manifest for the initial ramdisk, read by the `makerd` tool.
+# The contents should be a table array of files to add to the ramdistk:
+#
+#  [[files]]
+#  dest = "foo.bar"             # Name of the file in the ramdisk
+#  source = "build/foo/foo.bar" # Location of the file from the project root
+#  executable = true            # Optional, default false. Whether this is an
+#                               # initial application for the kernel to execute
+#                               # on startup
+
+[[files]]
+dest = "screenfont.psf"
+source = "../assets/fonts/tamsyn8x16r.psf"
+
+[[files]]
+dest = "nulldrv1"
+source = "host/nulldrv"
+executable = true
+
+[[files]]
+dest = "nulldrv2"
+source = "host/nulldrv"
+executable = true

generate_build.py

@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+
+from collections import namedtuple
+
+library = namedtuple('library', ['path', 'deps'])
+program = namedtuple('program', ['path', 'deps', 'output', 'targets'])
+source = namedtuple('source', ['name', 'input', 'output', 'action'])
+version = namedtuple('version', ['major', 'minor', 'patch', 'sha', 'dirty'])
+
+MODULES = {
+    "elf":       library('src/libraries/elf', ['kutil']),
+    "initrd":    library('src/libraries/initrd', ["kutil"]),
+    "kutil":     library('src/libraries/kutil', []),
+
+    "makerd":    program('src/tools/makerd', ["initrd", "kutil"], "makerd", ["native"]),
+
+    "boot":      program('src/boot', [], "boot.elf", ["boot"]),
+
+    "nulldrv":   program('src/drivers/nulldrv', [], "nulldrv", ["host"]),
+    "kernel":    program('src/kernel', ["elf", "initrd", "kutil"], "popcorn.elf", ["host"]),
+}
+
+
+def get_template(env, typename, name):
+    from jinja2.exceptions import TemplateNotFound
+    try:
+        return env.get_template("{}.{}.ninja.j2".format(typename, name))
+    except TemplateNotFound:
+        return env.get_template("{}.default.ninja.j2".format(typename))
+
+
+def get_sources(path, srcroot):
+    import os
+    from os.path import abspath, join, relpath, splitext
+
+    actions = {'.c': 'cc', '.cpp': 'cxx', '.s': 'nasm'}
+
+    sources = []
+    for root, dirs, files in os.walk(path):
+        for f in files:
+            base, ext = splitext(f)
+            if not ext in actions: continue
+            name = join(root, f)
+            sources.append(
+                source(
+                    relpath(name, srcroot),
+                    abspath(name),
+                    relpath(abspath(name), path) + ".o",
+                    actions[ext]))
+
+    return sources
+
+
+def get_git_version():
+    from subprocess import run
+    cp = run(['git', 'describe', '--dirty', '--abbrev=7'],
+            check=True, capture_output=True)
+    full_version = cp.stdout.decode('utf-8').strip()
+
+    dirty = False
+    parts1 = full_version.split('-')
+    if parts1[-1] == "dirty":
+        dirty = True
+        parts1 = parts1[:-1]
+
+    parts2 = parts1[0].split('.')
+
+    return version(
+        parts2[0],
+        parts2[1],
+        parts2[2],
+        parts1[-1][1:],
+        dirty)
+
+
+def main(buildroot):
+    import os
+    from os.path import abspath, dirname, isdir, join
+
+    generator = abspath(__file__)
+    srcroot = dirname(generator)
+
+    if buildroot is None:
+        buildroot = join(srcroot, "build")
+
+    if not isdir(buildroot):
+        os.mkdir(buildroot)
+
+    git_version = get_git_version()
+    print("Generating build files for Popcorn {}.{}.{}-{}...".format(
+        git_version.major, git_version.minor, git_version.patch, git_version.sha))
+
+    from jinja2 import Environment, FileSystemLoader
+    env = Environment(loader=FileSystemLoader(join(srcroot, "scripts", "templates")))
+
+    targets = {}
+    templates = set()
+    buildfiles = []
+    for name, mod in MODULES.items():
+        if isinstance(mod, program):
+            for target in mod.targets:
+                if not target in targets:
+                    targets[target] = set()
+
+                open_list = [name]
+                while open_list:
+                    depname = open_list.pop()
+                    dep = MODULES[depname]
+                    open_list.extend(dep.deps)
+                    targets[target].add(depname)
+
+        sources = get_sources(join(srcroot, mod.path), join(srcroot, "src"))
+        buildfile = join(buildroot, name + ".ninja")
+        buildfiles.append(buildfile)
+        with open(buildfile, 'w') as out:
+            #print("Generating module", name)
+            template = get_template(env, type(mod).__name__, name)
+            templates.add(template.filename)
+            out.write(template.render(
+                name=name,
+                module=mod,
+                sources=sources,
+                buildfile=buildfile,
+                version=git_version))
+
+    for target, mods in targets.items():
+        root = join(buildroot, target)
+        if not isdir(root):
+            os.mkdir(root)
+
+        buildfile = join(root, "target.ninja")
+        buildfiles.append(buildfile)
+        with open(buildfile, 'w') as out:
+            #print("Generating target", target)
+            template = get_template(env, "target", target)
+            templates.add(template.filename)
+            out.write(template.render(
+                target=target,
+                modules=mods,
+                buildfile=buildfile,
+                version=git_version))
+
+    buildfile = join(buildroot, "build.ninja")
+    buildfiles.append(buildfile)
+    with open(buildfile, 'w') as out:
+        #print("Generating main build.ninja")
+        template = env.get_template('build.ninja.j2')
+        templates.add(template.filename)
+
+        out.write(template.render(
+            targets=targets,
+            buildroot=buildroot,
+            srcroot=srcroot,
+            buildfile=buildfile,
+            buildfiles=buildfiles,
+            templates=[abspath(f) for f in templates],
+            generator=generator,
+            version=git_version))
+
+if __name__ == "__main__":
+    import sys
+    buildroot = None
+    if len(sys.argv) > 1:
+        buildroot = sys.argv[1]
+    main(buildroot)

qemu.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env bash
+
+build=${1:-"$(dirname $0)/build"}
+ninja -C $build
+
+kvm=""
+if [[ -f /dev/kvm ]]; then
+	kvm="--enable-kvm"
+fi
+
+exec qemu-system-x86_64 \
+	-drive "if=pflash,format=raw,file=${build}/flash.img" \
+	-drive "format=raw,file=${build}/popcorn.img" \
+	-smp 1 \
+	-m 512 \
+	-d mmu,int,guest_errors \
+	-D popcorn.log \
+	-cpu Broadwell \
+	-M q35 \
+	-no-reboot \
+	-nographic \
+	$kvm

scripts/build_sysroot_clang.sh

@@ -0,0 +1,220 @@
+#!/usr/bin/env bash
+
+TARGET="x86_64-elf"
+NASM_VERSION="2.13.03"
+BINUTILS_VERSION="2.31.1"
+
+TOOLS="clang" # lld libunwind libcxxabi libcxx"
+PROJECTS="compiler-rt libcxxabi libcxx libunwind"
+#RUNTIMES="compiler-rt libcxxabi libcxx libunwind"
+
+set -e
+
+SYSROOT=$(realpath "$(dirname $0)/../sysroot")
+WORK=$(realpath "$(dirname $0)/sysroot")
+mkdir -p "${SYSROOT}"
+mkdir -p "${WORK}"
+
+export CC=clang
+export CXX=clang++
+
+function build_nasm() {
+	if [[ ! -d "${WORK}/nasm-${NASM_VERSION}" ]]; then
+		echo "Downloading NASM..."
+		tarball="nasm-${NASM_VERSION}.tar.gz"
+		curl -sSOL "https://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/${tarball}"
+		tar xzf "${tarball}" -C "${WORK}" && rm "${tarball}"
+	fi
+
+	mkdir -p "${WORK}/build/nasm"
+	pushd "${WORK}/build/nasm"
+
+	if [[ ! -f "${WORK}/build/nasm/config.cache" ]]; then
+		echo "Configuring NASM..."
+		"${WORK}/nasm-${NASM_VERSION}/configure" \
+			--quiet \
+			--config-cache \
+			--disable-werror \
+			--prefix="${SYSROOT}" \
+			--srcdir="${WORK}/nasm-${NASM_VERSION}"
+	fi
+
+	echo "Building NASM..."
+	(make -j && make install) > "${WORK}/build/nasm_build.log"
+	popd
+}
+
+function build_binutils() {
+	if [[ ! -d "${WORK}/binutils-${BINUTILS_VERSION}" ]]; then
+		echo "Downloading binutils..."
+		tarball="binutils-${BINUTILS_VERSION}.tar.gz"
+		curl -sSOL "https://ftp.gnu.org/gnu/binutils/${tarball}"
+		tar xzf "${tarball}" -C "${WORK}" && rm "${tarball}"
+	fi
+
+	mkdir -p "${WORK}/build/binutils"
+	pushd "${WORK}/build/binutils"
+
+	if [[ ! -f "${WORK}/build/binutils/config.cache" ]]; then
+		echo "Configuring binutils..."
+		"${WORK}/binutils-${BINUTILS_VERSION}/configure" \
+			--quiet \
+			--config-cache \
+			--target="${TARGET}" \
+			--prefix="${SYSROOT}" \
+			--with-sysroot="${SYSROOT}" \
+			--with-lib-path="${SYSROOT}/lib" \
+			--disable-nls \
+			--disable-werror
+	fi
+
+	echo "Building binutils..."
+	(make -j && make install) > "${WORK}/build/binutils_build.log"
+	popd
+}
+
+function build_llvm() {
+	if [[ ! -d "${WORK}/llvm" ]]; then
+		echo "Downloading LLVM..."
+		git clone -q \
+			--branch release_70 \
+			--depth 1 \
+			"https://git.llvm.org/git/llvm.git" "${WORK}/llvm"
+	fi
+
+	for tool in ${TOOLS}; do
+		if [[ ! -d "${WORK}/llvm/tools/${tool}" ]]; then
+			echo "Downloading ${tool}..."
+			git clone -q \
+				--branch release_70 \
+				--depth 1 \
+				"https://git.llvm.org/git/${tool}.git" "${WORK}/llvm/tools/${tool}"
+		fi
+	done
+
+	if [[ ! -d "${WORK}/llvm/tools/clang/tools/extra" ]]; then
+		echo "Downloading clang-tools-extra..."
+		git clone -q \
+			--branch release_70 \
+			--depth 1 \
+			"https://git.llvm.org/git/clang-tools-extra.git" "${WORK}/llvm/tools/clang/tools/extra"
+	fi
+
+	for proj in ${PROJECTS}; do
+		if [[ ! -d "${WORK}/llvm/projects/${proj}" ]]; then
+			echo "Downloading ${proj}..."
+			git clone -q \
+				--branch release_70 \
+				--depth 1 \
+				"https://git.llvm.org/git/${proj}.git" "${WORK}/llvm/projects/${proj}"
+		fi
+	done
+
+	for proj in ${RUNTIMES}; do
+		if [[ ! -d "${WORK}/llvm/runtimes/${proj}" ]]; then
+			echo "Downloading ${proj}..."
+			git clone -q \
+				--branch release_70 \
+				--depth 1 \
+				"https://git.llvm.org/git/${proj}.git" "${WORK}/llvm/runtime/${proj}"
+		fi
+	done
+
+	mkdir -p "${WORK}/build/llvm"
+	pushd "${WORK}/build/llvm"
+
+	echo "Configuring LLVM..."
+
+	cmake -G Ninja \
+		-DCLANG_DEFAULT_RTLIB=compiler-rt \
+		-DCLANG_DEFAULT_STD_C=c11 \
+		-DCLANG_DEFAULT_STD_CXX=cxx14 \
+		-DCMAKE_BUILD_TYPE=Release \
+		-DCMAKE_C_COMPILER="clang" \
+		-DCMAKE_CXX_COMPILER="clang++" \
+		-DCMAKE_CXX_FLAGS="-Wno-unused-parameter -D_LIBCPP_HAS_NO_ALIGNED_ALLOCATION -D_LIBUNWIND_IS_BAREMETAL=1 -U_LIBUNWIND_SUPPORT_DWARF_UNWIND" \
+		-DCMAKE_INSTALL_PREFIX="${SYSROOT}" \
+		-DCMAKE_MAKE_PROGRAM=`which ninja` \
+		-DDEFAULT_SYSROOT="${SYSROOT}" \
+		-DLIBCXX_CXX_ABI=libcxxabi \
+		-DLIBCXX_CXX_ABI_INCLUDE_PATHS="${WORK}/llvm/projects/libcxxabi/include" \
+		-DLIBCXX_CXX_ABI_LIBRARY_PATH=lib \
+		-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=OFF \
+		-DLIBCXX_ENABLE_NEW_DELETE_DEFINITIONS=ON \
+		-DLIBCXX_ENABLE_SHARED=OFF \
+		-DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=ON \
+		-DLIBCXX_ENABLE_THREADS=OFF \
+		-DLIBCXX_INCLUDE_BENCHMARKS=OFF \
+		-DLIBCXX_USE_COMPILER_RT=ON \
+		-DLIBCXXABI_ENABLE_NEW_DELETE_DEFINITIONS=OFF \
+		-DLIBCXXABI_ENABLE_SHARED=OFF \
+		-DLIBCXXABI_ENABLE_STATIC_UNWINDER=ON \
+		-DLIBCXXABI_ENABLE_THREADS=OFF \
+		-DLIBCXXABI_LIBCXX_PATH="${WORK}/llvm/projects/libcxx" \
+		-DLIBCXXABI_USE_COMPILER_RT=ON \
+		-DLIBCXXABI_USE_LLVM_UNWINDER=ON \
+		-DLIBUNWIND_ENABLE_SHARED=OFF \
+		-DLIBUNWIND_ENABLE_THREADS=OFF \
+		-DLIBUNWIND_USE_COMPILER_RT=ON \
+		-DLLVM_CONFIG_PATH="${SYSROOT}/bin/llvm-config" \
+		-DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-unknown-elf" \
+		-DLLVM_ENABLE_LIBCXX=ON \
+		-DLLVM_ENABLE_PIC=OFF \
+		-DLLVM_ENABLE_THREADS=OFF \
+		-DLLVM_INSTALL_BINUTILS_SYMLINKS=ON \
+		-DLLVM_TARGETS_TO_BUILD="X86" \
+		${WORK}/llvm > cmake_configure.log
+
+	#  -DCMAKE_ASM_COMPILER=nasm \
+	#  -DCMAKE_LINKER="${SYSROOT}/bin/ld.lld" \
+	#  -DCOMPILER_RT_ENABLE_LLD=ON \
+	#  -DLIBCXX_ENABLE_LLD=ON \
+	#  -DLIBCXX_ENABLE_STATIC_UNWINDER=ON \
+	#  -DLIBCXXABI_ENABLE_LLD=ON \
+	#  -DLIBUNWIND_ENABLE_LLD=ON \
+	#  -DLLVM_ENABLE_LLD=ON \
+	#  -DLLVM_ENABLE_PROJECTS="libcxx;libcxxabi;libunwind;compiler-rt" \
+	#  -DCOMPILER_RT_BAREMETAL_BUILD=ON \
+	#  -DLIBCXXABI_BAREMETAL=ON \
+
+	echo "Building LLVM..."
+	ninja && ninja install
+	ninja cxx cxxabi compiler-rt
+	ninja install-compiler-rt install-cxx install-cxxabi
+	popd
+}
+
+function build_libc() {
+	if [[ ! -d "${WORK}/poplibc" ]]; then
+		echo "Downloading poplibc..."
+		git clone \
+			"https://github.com/justinian/poplibc.git" \
+			"${WORK}/poplibc"
+	else
+		echo "Updating poplibc..."
+		git -C "${WORK}/poplibc" pull
+	fi
+
+	pushd "${WORK}/poplibc"
+	echo "Building poplibc..."
+	make install PREFIX="${SYSROOT}"
+	popd
+}
+
+function update_links() {
+	for exe in `ls "${SYSROOT}/bin/${TARGET}-"*`; do
+		base=$(echo "$exe" | sed -e "s/${TARGET}-//")
+		ln -fs "${exe}" "${base}"
+	done
+}
+
+build_nasm
+build_binutils
+build_libc
+build_llvm
+update_links
+
+export CC="${SYSROOT}/bin/clang"
+export CXX="${SYSROOT}/bin/clang++"
+export LD="${SYSROOT}/bin/ld.lld"
+build_libc

scripts/build_sysroot_gcc.sh

@@ -0,0 +1,186 @@
+#!/usr/bin/env bash
+
+TARGET="x86_64-elf"
+NASM_VERSION="2.14.02"
+GCC_VERSION="7.4.0"
+BINUTILS_VERSION="2.31.1"
+
+SYSROOT=$(realpath "$(dirname $0)/../sysroot")
+WORK=$(realpath "$(dirname $0)/sysroot")
+
+echo "Not currently supported"
+exit 1
+
+set -e
+mkdir -p "${SYSROOT}"
+mkdir -p "${WORK}"
+
+
+function build_nasm() {
+	if [[ ! -d "${WORK}/nasm-${NASM_VERSION}" ]]; then
+		echo "Downloading NASM..."
+		tarball="nasm-${NASM_VERSION}.tar.gz"
+		curl -sSOL "https://www.nasm.us/pub/nasm/releasebuilds/${NASM_VERSION}/${tarball}"
+		tar xzf "${tarball}" -C "${WORK}" && rm "${tarball}"
+	fi
+
+	mkdir -p "${WORK}/build/nasm"
+	pushd "${WORK}/build/nasm"
+
+	if [[ ! -f "${WORK}/build/nasm/config.cache" ]]; then
+		echo "Configuring NASM..."
+		"${WORK}/nasm-${NASM_VERSION}/configure" \
+			--quiet \
+			--config-cache \
+			--disable-werror \
+			--prefix="${SYSROOT}" \
+			--srcdir="${WORK}/nasm-${NASM_VERSION}"
+	fi
+
+	echo "Building NASM..."
+	(make -j && make install) > "${WORK}/build/nasm_build.log"
+	popd
+}
+
+function build_binutils() {
+	if [[ ! -d "${WORK}/binutils-${BINUTILS_VERSION}" ]]; then
+		echo "Downloading binutils..."
+		tarball="binutils-${BINUTILS_VERSION}.tar.gz"
+		curl -sSOL "https://ftp.gnu.org/gnu/binutils/${tarball}"
+		tar xzf "${tarball}" -C "${WORK}" && rm "${tarball}"
+	fi
+
+	mkdir -p "${WORK}/build/binutils"
+	pushd "${WORK}/build/binutils"
+
+	if [[ ! -f "${WORK}/build/binutils/config.cache" ]]; then
+		echo "Configuring binutils..."
+		"${WORK}/binutils-${BINUTILS_VERSION}/configure" \
+			--quiet \
+			--config-cache \
+			--target="${TARGET}" \
+			--prefix="${SYSROOT}" \
+			--with-sysroot="${SYSROOT}" \
+			--with-lib-path="${SYSROOT}/lib" \
+			--disable-nls \
+			--disable-werror
+	fi
+
+	echo "Building binutils..."
+	(make -j && make install) > "${WORK}/build/binutils_build.log"
+	popd
+}
+
+function build_gcc() {
+	if [[ ! -d "${WORK}/gcc-${GCC_VERSION}" ]]; then
+		echo "Downloading GCC..."
+		tarball="gcc-${GCC_VERSION}.tar.gz"
+		curl -sSOL "https://ftp.gnu.org/gnu/gcc/gcc-${GCC_VERSION}/${tarball}"
+		tar xzf "${tarball}" -C "${WORK}" && rm "${tarball}"
+
+		# no-red-zone support version of libgcc
+		echo "MULTILIB_OPTIONS += mno-red-zone" >  "${WORK}/gcc-${GCC_VERSION}/gcc/config/i386/t-${TARGET}"
+		echo "MULTILIB_DIRNAMES += no-red-zone" >> "${WORK}/gcc-${GCC_VERSION}/gcc/config/i386/t-${TARGET}"
+
+cat <<EOF >> "${WORK}/gcc-${GCC_VERSION}/gcc/config.gcc"
+case \${target} in
+	${TARGET})
+		tmake_file="\${tmake_file} i386/t-${TARGET}"
+		;;
+esac
+EOF
+	fi
+
+	mkdir -p "${WORK}/build/gcc"
+	pushd "${WORK}/build/gcc"
+
+	if [[ ! -f "${WORK}/build/gcc/config.cache" ]]; then
+		echo "Configuring GCC..."
+		"${WORK}/gcc-${GCC_VERSION}/configure" \
+			--quiet \
+			--config-cache \
+			--target="${TARGET}" \
+			--prefix="${SYSROOT}" \
+			--with-sysroot="${SYSROOT}" \
+			--with-native-system-header-dir="${SYSROOT}/include" \
+			--with-newlib \
+			--without-headers \
+			--disable-nls \
+			--enable-languages=c,c++ \
+			--disable-shared \
+			--disable-multilib \
+			--disable-decimal-float \
+			--disable-threads \
+			--disable-libatomic \
+			--disable-libgomp \
+			--disable-libmpx \
+			--disable-libquadmath \
+			--disable-libssp \
+			--disable-libvtv \
+			--disable-libstdcxx
+	fi
+
+	echo "Building GCC..."
+	(make -j all-gcc && make -j all-target-libgcc && \
+		make install-gcc && make install-target-libgcc) > "${WORK}/build/gcc_build.log"
+	popd
+}
+
+function build_libstdcxx() {
+	mkdir -p "${WORK}/build/libstdcxx"
+	pushd "${WORK}/build/libstdcxx"
+
+	if [[ ! -f "${WORK}/build/libstdcxx/config.cache" ]]; then
+		echo "Configuring libstdc++..."
+		CFLAGS="-I${SYSROOT}/include" \
+		CXXFLAGS="-I${SYSROOT}/include" \
+		"${WORK}/gcc-${GCC_VERSION}/libstdc++-v3/configure" \
+			--config-cache \
+			--host="${TARGET}" \
+			--target="${TARGET}" \
+			--prefix="${SYSROOT}" \
+			--disable-nls \
+			--disable-multilib \
+			--with-newlib \
+			--disable-libstdcxx-threads \
+			--disable-libstdcxx-pch \
+			--with-gxx-include-dir="${SYSROOT}/include/c++"
+	fi
+
+	echo "Building libstdc++..."
+	(make -j && make install) > "${WORK}/build/libstdcxx_build.log"
+	popd
+}
+
+function build_libc() {
+	if [[ ! -d "${WORK}/poplibc" ]]; then
+		echo "Downloading poplibc..."
+		git clone \
+			"https://github.com/justinian/poplibc.git" \
+			"${WORK}/poplibc"
+	else
+		echo "Updating poplibc..."
+		git -C "${WORK}/poplibc" pull
+	fi
+
+	pushd "${WORK}/poplibc"
+	echo "Building poplibc..."
+	make install PREFIX="${SYSROOT}"
+	popd
+}
+
+function update_links() {
+	for exe in `ls "${SYSROOT}/bin/${TARGET}-"*`; do
+		base=$(echo "$exe" | sed -e "s/${TARGET}-//")
+		ln -fs "${exe}" "${base}"
+	done
+}
+
+build_nasm
+build_binutils
+build_gcc
+
+update_links
+export PATH="${SYSROOT}/bin:${PATH}"
+build_libc
+build_libstdcxx

scripts/templates/build.ninja.j2

@@ -0,0 +1,142 @@
+ninja_required_version = 1.3
+builddir = {{ buildroot }}
+srcroot = {{ srcroot }}
+
+warnflags = $
+    -Wformat=2 $
+    -Winit-self $
+    -Wfloat-equal $
+    -Winline $
+    -Wmissing-format-attribute $
+    -Wmissing-include-dirs $
+    -Wswitch $
+    -Wundef $
+    -Wdisabled-optimization $
+    -Wpointer-arith $
+    -Wno-attributes $
+    -Wno-sign-compare $
+    -Wno-multichar $
+    -Wno-div-by-zero $
+    -Wno-endif-labels $
+    -Wno-pragmas $
+    -Wno-format-extra-args $
+    -Wno-unused-result $
+    -Wno-deprecated-declarations $
+    -Wno-unused-function $
+    -Werror
+
+ccflags = $
+    -I${srcroot}/src/include $
+    -I${srcroot}/src/include/x86_64 $
+    -DVERSION_MAJOR={{ version.major }} $
+    -DVERSION_MINOR={{ version.minor }} $
+    -DVERSION_PATCH={{ version.patch }} $
+    -DVERSION_GITSHA=0x{% if version.dirty %}1{% else %}0{% endif %}{{ version.sha }} $
+    -DGIT_VERSION=\"{{ version.major }}.{{ version.minor }}.{{ version.patch }}-{{ version.sha }}\" $
+    -DGIT_VERSION_WIDE=L\"{{ version.major }}.{{ version.minor }}.{{ version.patch }}-{{ version.sha }}\" $
+    $warnflags
+
+asflags = $
+    -DVERSION_MAJOR={{ version.major }} $
+    -DVERSION_MINOR={{ version.minor }} $
+    -DVERSION_PATCH={{ version.patch }} $
+    -DVERSION_GITSHA=0x{% if version.dirty %}1{% else %}0{% endif %}{{ version.sha }}
+
+cflags = -std=c11
+cxxflags = -std=c++14
+libs =
+
+rule cc
+    deps = gcc
+    depfile = $out.d
+    description = Compiling $name
+    command = $cc -MMD -MF $out.d $ccflags $cflags -o $out -c $in
+
+rule cxx
+    deps = gcc
+    depfile = $out.d
+    description = Compiling $name
+    command = $cxx -MMD -MF $out.d $cxxflags $ccflags -o $out -c $in
+
+rule nasm
+    deps = gcc
+    depfile = $out.d
+    description = Assembling $name
+    command = $nasm -o $out -felf64 -MD $out.d $asflags $in
+
+rule exe
+    description = Linking $name
+    command = $ld $ldflags -o $out $in $libs
+
+rule lib
+    description = Archiving $name
+    command = $ar qcs $out $in
+
+rule regen
+    generator = true
+    description = Regenrating build files
+    command = {{ generator }} $builddir
+
+rule cp
+    description = Copying $name
+    command = cp $in $out
+
+rule makerd
+    description = Making init ramdisk
+    command = $builddir/native/makerd $in $out
+
+rule makeefi
+    description = Converting $name
+    command = objcopy $
+        -j .text $
+        -j .sdata $
+        -j .data $
+        -j .dynamic $
+        -j .dynsym $
+        -j .rel $
+        -j .rela $
+        -j .reloc $
+        --target=efi-app-x86_64 $
+        $in $out
+
+rule makefat
+    description = Creating $name
+    command = $
+        cp $srcroot/assets/diskbase.img $out; $
+        mcopy -s -D o -i $out@@1024K $builddir/fatroot/* ::/
+
+{% for target in targets %}
+subninja {{ target }}/target.ninja
+{% endfor %}
+
+build $
+    {%- for buildfile in buildfiles %}
+    {{ buildfile }} $
+    {%- endfor %}
+    : regen | $
+    {%- for template in templates %}
+    {{ template }} $
+    {%- endfor %}
+    {{ generator }}
+
+build $builddir/flash.img : cp $srcroot/assets/ovmf/x64/OVMF.fd
+    name = flash.img
+
+build $builddir/fatroot/popcorn.elf : cp $builddir/host/popcorn.elf
+    name = kernel to FAT image
+
+build $builddir/fatroot/efi/boot/bootx64.efi : cp $builddir/boot/boot.efi
+    name = bootloader to FAT image
+
+build $builddir/fatroot/initrd.img : makerd ${srcroot}/assets/initrd.toml | $
+    ${builddir}/native/makerd $
+    ${builddir}/host/nulldrv
+
+build $builddir/popcorn.img : makefat | $
+    $builddir/fatroot/initrd.img $
+    $builddir/fatroot/popcorn.elf $
+    $builddir/fatroot/efi/boot/bootx64.efi
+    name = popcorn.img
+
+# vim: et ts=4 sts=4 sw=4
+

scripts/templates/library.default.ninja.j2

@@ -0,0 +1 @@
+{% extends "module.base.ninja.j2" %}

scripts/templates/module.base.ninja.j2

@@ -0,0 +1,29 @@
+moddir = ${builddir}/{{ name }}.dir
+
+{% block variables %}
+ccflags = $ccflags $
+    {%- for dep in module.deps %}
+    -I${srcroot}/src/libraries/{{ dep }}/include $
+    {%- endfor %}
+    -I${srcroot}/{{ module.path }} $
+    -I${srcroot}/{{ module.path }}/include
+{% endblock %}
+
+{% for source in sources %}
+build ${moddir}/{{ source.output }} : {{ source.action }} {{ source.input }} || {{ buildfile }}
+    name = {{ source.name }}
+{% endfor %}
+
+build {% block artifact %} ${builddir}/lib{{ name }}.a : lib {% endblock %} $
+{%- block extrasources %}{% endblock -%}
+{%- for source in sources %}
+    ${moddir}/{{ source.output }}{% if not loop.last %} ${% endif %}
+{%- endfor -%}
+{%- if module.deps %}| {% for dep in module.deps %} ${builddir}/lib{{ dep }}.a {% endfor %}{% endif %}
+    name = {{ name }}
+
+{% block extra %}
+{% endblock %}
+
+# vim: ft=ninja et ts=4 sts=4 sw=4
+

scripts/templates/program.boot.ninja.j2

@@ -0,0 +1,29 @@
+{% extends "program.default.ninja.j2" %}
+{% block variables %}
+{{ super() }}
+
+ld = ld
+cc = clang
+cxx = clang++
+
+ccflags = $ccflags $
+    -DKERNEL_FILENAME=L\"popcorn.elf\" $
+    -DGNU_EFI_USE_MS_ABI $
+    -DHAVE_USE_MS_ABI $
+    -DEFI_DEBUG=0 $
+    -DEFI_DEBUG_CLEAR_MEMORY=0 $
+    -fPIC $
+    -fshort-wchar
+
+ldflags = $ldflags $
+    -T ${srcroot}/src/arch/x86_64/boot.ld $
+    -shared
+
+{% endblock %}
+
+{% block extra %}
+
+build $builddir/boot.efi : makeefi ${builddir}/{{ module.output }}
+    name = boot.efi
+
+{% endblock %}

scripts/templates/program.default.ninja.j2

@@ -0,0 +1,13 @@
+{% extends "module.base.ninja.j2" %}
+{% block variables %}
+{{ super() }}
+
+libs = $
+    -L${builddir} $
+    {%- for dep in module.deps %}
+    -l{{dep}} $
+    {%- endfor %}
+    $libs
+
+{% endblock %}
+{% block artifact %}${builddir}/{{ module.output }} : exe{% endblock %}

scripts/templates/program.kernel.ninja.j2

@@ -0,0 +1,9 @@
+{% extends "program.default.ninja.j2" %}
+{% block variables %}
+{{ super() }}
+
+asflags = $asflags -I${srcroot}/src/kernel/
+libs = $libs
+ldflags = $ldflags -T ${srcroot}/src/arch/x86_64/kernel.ld
+
+{% endblock %}

scripts/templates/target.boot.ninja.j2

@@ -0,0 +1,35 @@
+{% extends "target.default.ninja.j2" %}
+
+{% block binaries %}
+ld = ld
+cc = clang
+cxx = clang++
+nasm = nasm
+{% endblock %}
+
+{% block variables %}
+
+ccflags = $ccflags $
+    -ggdb $
+    -nostdlib $
+    -ffreestanding $
+    -nodefaultlibs $
+    -fno-builtin $
+    -mno-sse $
+    -fno-omit-frame-pointer $
+    -mno-red-zone
+
+cxxflags = $cxxflags $
+    -nostdlibinc $
+    -isystem${srcroot}/sysroot/include/c++/v1 $
+    -fno-exceptions $
+    -fno-rtti
+
+ldflags = $ldflags $
+    -g $
+    -nostdlib $
+    -znocombreloc $
+	-Bsymbolic $
+    -nostartfiles
+
+{% endblock %}

scripts/templates/target.default.ninja.j2

@@ -0,0 +1,17 @@
+builddir = $builddir/{{ target }}
+
+{% block variables %}
+{% endblock %}
+
+{% block binaries %}
+cc = clang
+cxx = clang++
+ld = clang++
+ar = ar
+nasm = nasm
+objcopy = objcopy
+{% endblock %}
+
+{% for module in modules %}
+subninja {{ module }}.ninja
+{% endfor %}

scripts/templates/target.host.ninja.j2

@@ -0,0 +1,42 @@
+{% extends "target.default.ninja.j2" %}
+{% block binaries %}
+cc = ${srcroot}/sysroot/bin/clang
+cxx = ${srcroot}/sysroot/bin/clang++
+ld = ${srcroot}/sysroot/bin/x86_64-elf-ld
+ar = ${srcroot}/sysroot/bin/x86_64-elf-ar
+nasm = ${srcroot}/sysroot/bin/nasm
+objcopy = ${srcroot}/sysroot/bin/x86_64-elf-objcopy
+{% endblock %}
+
+{% block variables %}
+
+ccflags = $ccflags $
+    -nostdlib $
+    -ffreestanding $
+    -nodefaultlibs $
+    -fno-builtin $
+    -mno-sse $
+    -fno-omit-frame-pointer $
+    -mno-red-zone $
+    -g $
+    -mcmodel=large $
+    -D__ELF__ $
+    -D__POPCORN__ $
+    -isystem${srcroot}/sysroot/include $
+    --sysroot="${srcroot}/sysroot"
+
+cxxflags = $cxxflags $
+    -fno-exceptions $
+    -fno-rtti $
+    -isystem${srcroot}/sysroot/include/c++/v1
+
+ldflags = $ldflags $
+    -g $
+    -nostdlib $
+    -znocombreloc $
+    -Bsymbolic $
+    -nostartfiles $
+    -Bstatic
+
+{% endblock %}
+# vim: et ts=4 sts=4 sw=4

scripts/templates/target.native.ninja.j2

@@ -0,0 +1 @@
+{% extends "target.default.ninja.j2" %}

scripts/vmem_translate.py

@@ -0,0 +1,12 @@
+#!/usr/bin/env python
+
+def translate(i4 = 0, i3 = 0, i2 = 0, i1 = 0, offset = 0):
+    addr = (i4 << 39) + (i3 << 30) + (i2 << 21) + (i1 << 12) + offset
+    if addr & (1 << 47):
+        addr |= 0xffff000000000000
+    return addr
+
+if __name__ == "__main__":
+    import sys
+    print("{:016x}".format(translate(*map(int, sys.argv[1:]))))
+

src/arch/x86_64/kernel.ld

@@ -1,7 +1,7 @@
 ENTRY(_start)
 SECTIONS
 {
-	OFFSET = 0xFFFF800000000000;
+	OFFSET = 0xFFFFFF0000000000;
 	. = OFFSET + 0x100000;
 
 	.header : {
@@ -27,11 +27,24 @@ SECTIONS
 		*(.note.*)
 	}
 
-	.bss ALIGN(0x1000) : {
+	.bss ALIGN(16) : {
 		__bss_start = .;
 		*(.bss)
 		__bss_end = .;
 	}
 
+	.eh_frame : {
+		__eh_frame_start = .;
+		KEEP(*(.eh_frame))
+		__eh_frame_end = .;
+	}
+
+	.eh_frame_hdr : {
+		KEEP(*(.eh_frame_hdr))
+	}
+
+	__eh_frame_hdr_start = SIZEOF(.eh_frame_hdr) > 0 ? ADDR(.eh_frame_hdr) : 0;
+	__eh_frame_hdr_end = SIZEOF(.eh_frame_hdr) > 0 ? . : 0;
+
 	kernel_end = ALIGN(4096);
 }

src/boot/loader.c

@@ -7,7 +7,7 @@
 #define PAGE_SIZE 0x1000
 
 static CHAR16 kernel_name[] = KERNEL_FILENAME;
-static CHAR16 font_name[] = KERNEL_FONT;
+static CHAR16 initrd_name[] = INITRD_FILENAME;
 
 EFI_STATUS
 loader_alloc_pages(
@@ -38,7 +38,7 @@ loader_alloc_pages(
 }
 
 EFI_STATUS
-loader_load_font(
+loader_load_initrd(
 	EFI_BOOT_SERVICES *bootsvc,
 	EFI_FILE_PROTOCOL *root,
 	struct loader_data *data)
@@ -46,13 +46,13 @@ loader_load_font(
 	EFI_STATUS status;
 
 	EFI_FILE_PROTOCOL *file = NULL;
-	status = root->Open(root, &file, (CHAR16 *)font_name, EFI_FILE_MODE_READ,
+	status = root->Open(root, &file, (CHAR16 *)initrd_name, EFI_FILE_MODE_READ,
 						EFI_FILE_READ_ONLY | EFI_FILE_HIDDEN | EFI_FILE_SYSTEM);
 
 	if (status == EFI_NOT_FOUND)
 		return status;
 
-	CHECK_EFI_STATUS_OR_RETURN(status, L"Opening file %s", font_name);
+	CHECK_EFI_STATUS_OR_RETURN(status, L"Opening file %s", initrd_name);
 
 	char info[sizeof(EFI_FILE_INFO) + 100];
 	size_t info_length = sizeof(info);
@@ -60,16 +60,16 @@ loader_load_font(
 	status = file->GetInfo(file, &guid_file_info, &info_length, info);
 	CHECK_EFI_STATUS_OR_RETURN(status, L"Getting file info");
 
-	data->font_length = ((EFI_FILE_INFO *)info)->FileSize;
+	data->initrd_length = ((EFI_FILE_INFO *)info)->FileSize;
 
 	status = loader_alloc_pages(
 			bootsvc,
-			KERNEL_FONT_MEMTYPE,
-			&data->font_length,
-			&data->font);
+			INITRD_MEMTYPE,
+			&data->initrd_length,
+			&data->initrd);
 	CHECK_EFI_STATUS_OR_RETURN(status, L"Allocating pages");
 
-	status = file->Read(file, &data->font_length, data->font);
+	status = file->Read(file, &data->initrd_length, data->initrd);
 	CHECK_EFI_STATUS_OR_RETURN(status, L"Reading file");
 
 	status = file->Close(file);
@@ -87,6 +87,8 @@ loader_load_elf(
 {
 	EFI_STATUS status;
 
+	con_debug(L"Opening kernel file %s\r\n", (CHAR16 *)kernel_name);
+
 	EFI_FILE_PROTOCOL *file = NULL;
 	status = root->Open(root, &file, (CHAR16 *)kernel_name, EFI_FILE_MODE_READ,
 						EFI_FILE_READ_ONLY | EFI_FILE_HIDDEN | EFI_FILE_SYSTEM);
@@ -107,6 +109,8 @@ loader_load_elf(
 	status = file->Read(file, &length, &header);
 	CHECK_EFI_STATUS_OR_RETURN(status, L"Reading ELF header");
 
+	con_debug(L"Read %u bytes of ELF header\r\n", length);
+
 	if (length < sizeof(struct elf_header))
 		CHECK_EFI_STATUS_OR_RETURN(EFI_LOAD_ERROR, L"Incomplete read of ELF header");
 
@@ -131,10 +135,14 @@ loader_load_elf(
 		header.machine != 0x3e)
 		CHECK_EFI_STATUS_OR_RETURN(EFI_LOAD_ERROR, L"ELF load error: wrong machine architecture");
 
+	con_debug(L"ELF is valid, entrypoint %lu\r\n", header.entrypoint);
+
 	data->kernel_entry = (void *)header.entrypoint;
 
 	struct elf_program_header prog_header;
 	for (int i = 0; i < header.ph_num; ++i) {
+		con_debug(L"Reading ELF program header %d\r\n", i);
+
 		status = file->SetPosition(file, header.ph_offset + i * header.ph_entsize);
 		CHECK_EFI_STATUS_OR_RETURN(status, L"Setting ELF file position");
 
@@ -156,6 +164,8 @@ loader_load_elf(
 
 	struct elf_section_header sec_header;
 	for (int i = 0; i < header.sh_num; ++i) {
+		con_debug(L"Reading ELF section header %d ", i);
+
 		status = file->SetPosition(file, header.sh_offset + i * header.sh_entsize);
 		CHECK_EFI_STATUS_OR_RETURN(status, L"Setting ELF file position");
 
@@ -163,11 +173,15 @@ loader_load_elf(
 		status = file->Read(file, &length, &sec_header);
 		CHECK_EFI_STATUS_OR_RETURN(status, L"Reading ELF section header");
 
-		if ((sec_header.flags & ELF_SHF_ALLOC) == 0) continue;
+		if ((sec_header.flags & ELF_SHF_ALLOC) == 0) {
+			con_debug(L"SHF_ALLOC section\r\n");
+			continue;
+		}
 
 		void *addr = (void *)(sec_header.addr - KERNEL_VIRT_ADDRESS);
 
 		if (sec_header.type == ELF_ST_PROGBITS) {
+			con_debug(L"PROGBITS section\r\n");
 			status = file->SetPosition(file, sec_header.offset);
 			CHECK_EFI_STATUS_OR_RETURN(status, L"Setting ELF file position");
 
@@ -175,7 +189,10 @@ loader_load_elf(
 			status = file->Read(file, &length, addr);
 			CHECK_EFI_STATUS_OR_RETURN(status, L"Reading file");
 		} else if (sec_header.type == ELF_ST_NOBITS) {
+			con_debug(L"NOBITS section\r\n");
 			bootsvc->SetMem(addr, sec_header.size, 0);
+		} else {
+			con_debug(L"other section\r\n");
 		}
 	}
 
@@ -215,14 +232,14 @@ loader_load_kernel(
 		if (status == EFI_NOT_FOUND)
 			continue;
 
-		CHECK_EFI_STATUS_OR_RETURN(status, L"loader_load_file: %s", kernel_name);
+		CHECK_EFI_STATUS_OR_RETURN(status, L"loader_load_elf: %s", kernel_name);
 
-		data->font = (void *)((uint64_t)data->kernel + data->kernel_length);
-		status = loader_load_font(bootsvc, root, data);
+		data->initrd = (void *)((uint64_t)data->kernel + data->kernel_length);
+		status = loader_load_initrd(bootsvc, root, data);
 
-		CHECK_EFI_STATUS_OR_RETURN(status, L"loader_load_file: %s", font_name);
+		CHECK_EFI_STATUS_OR_RETURN(status, L"loader_load_file: %s", initrd_name);
 
-		data->data = (void *)((uint64_t)data->font + data->font_length);
+		data->data = (void *)((uint64_t)data->initrd + data->initrd_length);
 		data->data_length += PAGE_SIZE; // extra page for map growth
 		status = loader_alloc_pages(
 				bootsvc,

src/boot/loader.h

@@ -9,35 +9,27 @@
 #endif
 
 #ifndef KERNEL_VIRT_ADDRESS
-#define KERNEL_VIRT_ADDRESS 0xFFFF800000000000
-#endif
-
-#ifndef VIRTUAL_OFFSET
-#define VIRTUAL_OFFSET 0xf00000000
+#define KERNEL_VIRT_ADDRESS 0xFFFFFF0000000000
 #endif
 
 #ifndef KERNEL_MEMTYPE
 #define KERNEL_MEMTYPE 0x80000000
 #endif
 
-#ifndef KERNEL_FONT_MEMTYPE
-#define KERNEL_FONT_MEMTYPE 0x80000001
+#ifndef INITRD_MEMTYPE
+#define INITRD_MEMTYPE 0x80000001
 #endif
 
 #ifndef KERNEL_DATA_MEMTYPE
 #define KERNEL_DATA_MEMTYPE 0x80000002
 #endif
 
-#ifndef KERNEL_PT_MEMTYPE
-#define KERNEL_PT_MEMTYPE 0x80000004
-#endif
-
 #ifndef KERNEL_FILENAME
 #define KERNEL_FILENAME L"kernel.elf"
 #endif
 
-#ifndef KERNEL_FONT
-#define KERNEL_FONT L"screenfont.psf"
+#ifndef INITRD_FILENAME
+#define INITRD_FILENAME L"initrd.img"
 #endif
 
 struct loader_data {
@@ -45,8 +37,8 @@ struct loader_data {
 	void *kernel_entry;
 	size_t kernel_length;
 
-	void *font;
-	size_t font_length;
+	void *initrd;
+	size_t initrd_length;
 
 	void *data;
 	size_t data_length;

src/boot/main.c

@@ -83,7 +83,7 @@ efi_main(EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *system_table)
 	CHECK_EFI_STATUS_OR_FAIL(status);
 
 	con_printf(L"    %u image bytes at 0x%x\r\n", load.kernel_length, load.kernel);
-	con_printf(L"    %u font bytes at 0x%x\r\n", load.font_length, load.font);
+	con_printf(L"    %u initrd bytes at 0x%x\r\n", load.initrd_length, load.initrd);
 	con_printf(L"    %u data bytes at 0x%x\r\n", load.data_length, load.data);
 
 	struct kernel_header *version = (struct kernel_header *)load.kernel;
@@ -111,9 +111,9 @@ efi_main(EFI_HANDLE image_handle, EFI_SYSTEM_TABLE *system_table)
 
 	data_header->flags = 0;
 
-	data_header->font = load.font;
-	data_header->font_length = load.font_length;
-	memory_mark_pointer_fixup((void **)&data_header->font);
+	data_header->initrd = load.initrd;
+	data_header->initrd_length = load.initrd_length;
+	memory_mark_pointer_fixup((void **)&data_header->initrd);
 
 	data_header->data = load.data;
 	data_header->data_length = load.data_length;

src/boot/memory.c

@@ -188,9 +188,10 @@ memory_virtualize(EFI_RUNTIME_SERVICES *runsvc, struct memory_map *map)
 	while (d < end) {
 		switch (d->Type) {
 		case KERNEL_MEMTYPE:
-		case KERNEL_FONT_MEMTYPE:
+		case INITRD_MEMTYPE:
 		case KERNEL_DATA_MEMTYPE:
 			d->Attribute |= EFI_MEMORY_RUNTIME;
+			d->VirtualStart = d->PhysicalStart + KERNEL_VIRT_ADDRESS;
 
 		default:
 			if (d->Attribute & EFI_MEMORY_RUNTIME) {

src/boot/utility.h

@@ -32,3 +32,9 @@ const CHAR16 *util_error_message(EFI_STATUS status);
 			: "r"((uint64_t)s), "r"((uint64_t)d), "r"((uint64_t)__LINE__) \
 			: "rax", "rdx", "r8", "r9", "r10"); \
 	}
+
+#ifdef BOOTLOADER_DEBUG
+#define con_debug(...) con_printf(L"DEBUG: " __VA_ARGS__)
+#else
+#define con_debug(...)
+#endif

src/boot/wscript

@@ -1,61 +0,0 @@
-
-def configure(ctx):
-    from os.path import join
-
-    lds_path = join(ctx.env.ARCH_D, "boot.ld")
-
-    ctx.env.append_value('DEFINES_EFI', [
-        'KERNEL_FILENAME=L"{}"'.format(ctx.env.KERNEL_FILENAME),
-        'GNU_EFI_USE_MS_ABI',
-        'HAVE_USE_MS_ABI',
-        'EFI_DEBUG=0',
-        'EFI_DEBUG_CLEAR_MEMORY=0',
-    ])
-    ctx.env.append_value('CFLAGS_EFI', ['-fPIC', '-fshort-wchar'])
-    ctx.env.append_value('LINKFLAGS_EFI', [
-        '-shared',
-        '-T', lds_path,
-    ])
-    ctx.env.append_value('SECTIONS_EFI', [
-    ])
-
-def build(bld):
-    sources = bld.path.ant_glob("**/*.c")
-    sources += bld.path.ant_glob("**/*.s")
-
-    bld.program(
-        source      = sources,
-        target      = "boot.elf",
-        use         = 'EFI',
-    )
-
-    from waflib.Task import Task
-    class make_efi(Task):
-        color = 'YELLOW'
-        def keyword(self):
-            return "Creating"
-        def __str__(self):
-            node = self.outputs[0]
-            return node.path_from(node.ctx.launch_node())
-        def run(self):
-            from subprocess import check_call as call
-
-            args = self.env.objcopy
-            sections = [".text", ".sdata", ".data", ".dynamic",
-                        ".dynsym", ".rel", ".rela", ".reloc"]
-            for s in sections: args.extend(['-j', s])
-            args.append('--target=efi-app-' + self.env.POPCORN_ARCH)
-            args.append(self.inputs[0].abspath())
-            args.append(self.outputs[0].abspath())
-            call(args)
-
-    src = bld.path
-    out = bld.path.get_bld()
-
-    efi = make_efi(env=bld.env)
-    efi.set_inputs([out.make_node("boot.elf")])
-    efi.set_outputs([out.make_node("boot.efi")])
-    bld.add_to_group(efi)
-
-
-# vim: ft=python et sw=4

src/drivers/ahci/driver.cpp

@@ -3,13 +3,15 @@
 #include "log.h"
 #include "pci.h"
 
+namespace ahci {
 
-ahci_driver::ahci_driver()
+
+driver::driver()
 {
 }
 
 void
-ahci_driver::register_device(pci_device *device)
+driver::register_device(pci_device *device)
 {
 	log::info(logs::driver, "AHCI registering device %d:%d:%d:",
 			device->bus(), device->device(), device->function());
@@ -17,12 +19,4 @@ ahci_driver::register_device(pci_device *device)
 	ahci::hba &hba = m_devices.emplace(device);
 }
 
-ahci::port *
-ahci_driver::find_disk()
-{
-	for (auto &hba : m_devices) {
-		ahci::port *d = hba.find_disk();
-		if (d) return d;
-	}
-	return nullptr;
-}
+} // namespace

src/drivers/ahci/driver.h

@@ -6,13 +6,15 @@
 
 class pci_device;
 
+namespace ahci {
+
 
 /// Basic AHCI driver
-class ahci_driver
+class driver
 {
 public:
 	/// Constructor.
-	ahci_driver();
+	driver();
 
 	/// Register a device with the driver
 	/// \arg device  The PCI device to handle
@@ -22,10 +24,8 @@ public:
 	/// \arg device  The PCI device to remove
 	void unregister_device(pci_device *device);
 
-	/// Debug: find the first disk
-	ahci::port * find_disk();
-
 private:
 	kutil::vector<ahci::hba> m_devices;
 };
 
+} // namespace

src/drivers/ahci/hba.cpp

@@ -1,7 +1,9 @@
 #include <stdint.h>
 #include "ahci/ata.h"
 #include "ahci/hba.h"
+#include "console.h"
 #include "device_manager.h"
+#include "fs/gpt.h"
 #include "log.h"
 #include "page_manager.h"
 #include "pci.h"
@@ -57,10 +59,14 @@ void irq_cb(void *data)
 hba::hba(pci_device *device)
 {
 	page_manager *pm = page_manager::get();
+	device_manager &dm = device_manager::get();
 
 	uint32_t bar5 = device->get_bar(5);
-	m_data = reinterpret_cast<hba_data *>(bar5 & ~0xfffull);
-	pm->map_offset_pointer(reinterpret_cast<void **>(&m_data), 0x2000);
+	log::debug(logs::driver, "HBA raw BAR5 is %08lx", bar5);
+
+	void *data = reinterpret_cast<void *>(bar5 & ~0xfffull);
+	pm->map_offset_pointer(&data, 0x2000);
+	m_data = reinterpret_cast<hba_data volatile *>(data);
 
 	if (! bitfield_has(m_data->cap, hba_cap::ahci_only))
 		m_data->host_control |= 0x80000000; // Enable AHCI mode
@@ -70,17 +76,17 @@ hba::hba(pci_device *device)
 	unsigned ports = (icap & 0xf) + 1;
 	unsigned slots = ((icap >> 8) & 0x1f) + 1;
 
-	log::debug(logs::driver, "  %d ports", ports);
+	log::debug(logs::driver, "  %d ports: %08x", ports, m_data->port_impl);
 	log::debug(logs::driver, "  %d command slots", slots);
 
-	port_data *pd = reinterpret_cast<port_data *>(
+	auto *pd = reinterpret_cast<port_data volatile *>(
 			kutil::offset_pointer(m_data, 0x100));
 
 	bool needs_interrupt = false;
 	m_ports.ensure_capacity(ports);
 	for (unsigned i = 0; i < ports; ++i) {
 		bool impl = ((m_data->port_impl & (1 << i)) != 0);
-		port &p = m_ports.emplace(i, kutil::offset_pointer(pd, 0x80 * i), impl);
+		port &p = m_ports.emplace(this, i, kutil::offset_pointer(pd, 0x80 * i), impl);
 		if (p.get_state() == port::state::active)
 			needs_interrupt = true;
 	}
@@ -89,29 +95,48 @@ hba::hba(pci_device *device)
 		device_manager::get().allocate_msi("AHCI Device", *device, irq_cb, this);
 		m_data->host_control |= 0x02; // enable interrupts
 	}
-}
 
-port *
-hba::find_disk()
-{
-	for (auto &port : m_ports) {
-		if (port.get_state() == port::state::active &&
-			port.get_type() == sata_signature::sata_drive)
-			return &port;
+	for (auto &p : m_ports) {
+		if (!p.active()) continue;
+
+		if (p.get_type() == sata_signature::sata_drive) {
+			p.sata_reconnect();
+			/*
+			if (fs::partition::load(&p) == 0)
+				dm.register_block_device(&p);
+			*/
+		}
 	}
-
-	return nullptr;
 }
 
 void
 hba::handle_interrupt()
 {
+	uint32_t status = m_data->int_status;
 	for (auto &port : m_ports) {
-		if (m_data->int_status & (1 << port.index())) {
+		if (status & (1 << port.index())) {
 			port.handle_interrupt();
 		}
 	}
-	m_data->int_status = 0;
+	// Write 1 to the handled interrupts
+	m_data->int_status = status;
+}
+
+void
+hba::dump()
+{
+	console *cons = console::get();
+	static const char *regs[] = {
+		" CAP", " GHC", "  IS", "  PI", "  VS", " C3C",
+		" C3P", " EML", " EMC", "CAP2", "BOHC"
+	};
+
+	cons->printf("HBA Registers:\n");
+	auto *data = reinterpret_cast<uint32_t volatile *>(m_data);
+	for (int i = 0; i < 11; ++i) {
+		cons->printf("  %s: %08x\n", regs[i], data[i]);
+	}
+	cons->putc('\n');
 }
 
 } // namespace ahci

src/drivers/ahci/hba.h

@@ -25,12 +25,12 @@ public:
 	/// Interrupt handler.
 	void handle_interrupt();
 
-	/// Debug: find the first disk
-	port * find_disk();
+	/// Dump the HBA registers to the console
+	void dump();
 
 private:
 	pci_device *m_device;
-	hba_data *m_data;
+	hba_data volatile *m_data;
 	kutil::vector<port> m_ports;
 };
 

src/drivers/ahci/port.cpp

@@ -2,13 +2,21 @@
 #include "kutil/assert.h"
 #include "kutil/enum_bitfields.h"
 #include "ahci/ata.h"
+#include "ahci/hba.h"
 #include "ahci/fis.h"
 #include "ahci/port.h"
+#include "console.h"
 #include "io.h"
 #include "log.h"
 #include "page_manager.h"
 
+namespace ahci {
+	enum class cmd_list_flags : uint16_t;
+}
+
 IS_BITFIELD(ahci::port_cmd);
+IS_BITFIELD(volatile ahci::port_cmd);
+IS_BITFIELD(ahci::cmd_list_flags);
 
 namespace ahci {
 
@@ -110,10 +118,11 @@ struct port_data
 } __attribute__ ((packed));
 
 
-port::port(uint8_t index, port_data *data, bool impl) :
+port::port(hba *device, uint8_t index, port_data volatile *data, bool impl) :
 	m_index(index),
 	m_type(sata_signature::none),
 	m_state(state::unimpl),
+	m_hba(device),
 	m_data(data),
 	m_fis(nullptr),
 	m_cmd_list(nullptr),
@@ -154,7 +163,13 @@ port::update()
 
 		rebase();
 		m_pending.set_size(32);
-		m_data->interrupt_enable = 1;
+		for (auto &pend : m_pending) {
+			pend.type = command_type::none;
+		}
+
+		// Clear any old pending interrupts and enable interrupts
+		m_data->interrupt_status = m_data->interrupt_status;
+		m_data->interrupt_enable = 0xffffffff;
 	} else {
 		m_state = state::inactive;
 	}
@@ -190,19 +205,27 @@ port::stop_commands()
 }
 
 int
-port::make_command(size_t length)
+port::make_command(size_t length, fis_register_h2d **fis)
 {
 	int slot = -1;
-	uint32_t used_slots = (m_data->serial_active | m_data->cmd_issue);
+	uint32_t used_slots =
+		m_data->serial_active |
+		m_data->cmd_issue |
+		m_data->interrupt_status;
+
 	for (int i = 0; i < 32; ++i) {
-		if ((used_slots & (1 << i)) == 0) {
+		if (used_slots & (1 << i)) continue;
+
+		if (m_pending[i].type == command_type::none) {
 			slot = i;
 			break;
+		} else {
+			log::debug(logs::driver, "Type is %d", m_pending[i].type);
 		}
 	}
 
 	if (slot < 0) {
-		log::info(logs::driver, "AHCI could not get a free command slot.");
+		log::error(logs::driver, "AHCI could not get a free command slot.");
 		return -1;
 	}
 
@@ -214,6 +237,14 @@ port::make_command(size_t length)
 	kutil::memset(&cmdt, 0, sizeof(cmd_table) +
 			max_prd_count * sizeof(prdt_entry));
 
+	ent.flags = cmd_list_fis_size(sizeof(fis_register_h2d));
+
+	fis_register_h2d *cfis = reinterpret_cast<fis_register_h2d *>(&cmdt.cmd_fis);
+	kutil::memset(cfis, 0, sizeof(fis_register_h2d));
+	cfis->type = fis_type::register_h2d;
+	cfis->pm_port = 0x80; // set command register flag
+	*fis = cfis;
+
 	size_t remaining = length;
 	for (int i = 0; i < max_prd_count; ++i) {
 		size_t prd_len = std::min(remaining, 0x200000ul);
@@ -222,7 +253,7 @@ port::make_command(size_t length)
 		void *mem = pm->map_offset_pages(page_count(prd_len));
 		kutil::memset(mem, 0xaf, prd_len);
 
-		addr_t phys = pm->offset_phys(mem);
+		uintptr_t phys = pm->offset_phys(mem);
 		cmdt.entries[i].data_base_low = phys & 0xffffffff;
 		cmdt.entries[i].data_base_high = phys >> 32;
 		cmdt.entries[i].byte_count = prd_len - 1;
@@ -240,21 +271,20 @@ port::make_command(size_t length)
 	return slot;
 }
 
-bool
-port::read(uint64_t sector, size_t length, void *dest)
+int
+port::read_async(uint64_t offset, size_t length, void *dest)
 {
-	int slot = make_command(length);
+	fis_register_h2d *fis;
+	int slot = make_command(length, &fis);
 	if (slot < 0)
-		return false;
+		return 0;
 
 	cmd_table &cmdt = m_cmd_table[slot];
 
-	fis_register_h2d *fis = reinterpret_cast<fis_register_h2d *>(&cmdt.cmd_fis);
-	fis->type = fis_type::register_h2d;
-	fis->pm_port = 0x80; // set command register flag
 	fis->command = ata_cmd::read_dma_ext;
 	fis->device = 0x40; // ATA8-ACS p.175
 
+	uint64_t sector = offset >> 9;
 	fis->lba0 = (sector      ) & 0xff;
 	fis->lba1 = (sector >>  8) & 0xff;
 	fis->lba2 = (sector >> 16) & 0xff;
@@ -270,8 +300,64 @@ port::read(uint64_t sector, size_t length, void *dest)
 			count, sector, sector*512);
 
 	m_pending[slot].type = command_type::read;
+	m_pending[slot].offset = offset % 512;
+	m_pending[slot].count = 0;
 	m_pending[slot].data = dest;
-	return issue_command(slot);
+	if(issue_command(slot))
+		return slot;
+	else
+		return -1;
+}
+
+size_t
+port::read(uint64_t offset, size_t length, void *dest)
+{
+	int slot = read_async(offset, length, dest);
+
+	int timeout = 0;
+	while (m_pending[slot].type == command_type::read) {
+		if (timeout++ > 5) {
+			return 0;
+		}
+		asm("hlt");
+	}
+	kassert(m_pending[slot].type == command_type::finished,
+			"Read got unexpected command type");
+
+	size_t count = m_pending[slot].count;
+	m_pending[slot].type = command_type::none;
+	m_pending[slot].count = 0;
+
+	return count;
+}
+
+int
+port::identify_async()
+{
+	fis_register_h2d *fis;
+	int slot = make_command(512, &fis);
+	if (slot < 0)
+		return 0;
+
+	fis->command = ata_cmd::identify;
+
+	m_pending[slot].type = command_type::identify;
+	m_pending[slot].offset = 0;
+	m_pending[slot].count = 0;
+	m_pending[slot].data = 0;
+
+	if(issue_command(slot))
+		return slot;
+	else
+		return -1;
+}
+
+void
+port::sata_reconnect()
+{
+	m_data->serial_control |= 1;
+	io_wait(1000); // About 1ms
+	m_data->serial_control &= ~1;
 }
 
 bool
@@ -297,14 +383,23 @@ port::issue_command(int slot)
 void
 port::handle_interrupt()
 {
-	log::debug(logs::driver, "AHCI port %d got an interrupt");
+	log::debug(logs::driver, "AHCI port %d got an interrupt", m_index);
 
 	// TODO: handle other states in interrupt_status
 
-	if (m_data->interrupt_status & 0x40000000) {
+	uint32_t is = m_data->interrupt_status;
+
+	if (is & 0x00000040) {
+		// Port connect status change: For now clear the "exchange"
+		// bit in SERR, this should probably kick off diagnostics.
+		m_data->serial_error = 0x04000000;
+		identify_async();
+	}
+
+	if (is & 0x40000000) {
 		log::error(logs::driver, "AHCI task file error");
-		// TODO: clean up!
-		return;
+		dump();
+		kassert(0, "Task file error");
 	}
 
 	log::debug(logs::driver, "AHCI interrupt status: %08lx  %08lx",
@@ -312,19 +407,25 @@ port::handle_interrupt()
 
 	uint32_t ci = m_data->cmd_issue;
 	for (int i = 0; i < 32; ++i) {
+		// Skip commands still listed as "issued"
 		if (ci & (1 << i)) continue;
 
+		// Any commands not still listed as "issued" that are still pending for
+		// the driver are now finished, so handle them.
 		pending &p = m_pending[i];
 		switch (p.type) {
 		case command_type::read:
 			finish_read(i);
 			break;
+		case command_type::identify:
+			finish_identify(i);
+			break;
 		default:
 			break;
 		}
-		p.type = command_type::none;
-		p.data = nullptr;
 	}
+
+	// Clear the whole status register to mark it as handled
 	m_data->interrupt_status = m_data->interrupt_status;
 }
 
@@ -335,21 +436,92 @@ port::finish_read(int slot)
 	cmd_table &cmdt = m_cmd_table[slot];
 	cmd_list_entry &ent = m_cmd_list[slot];
 
+	size_t count = 0;
 	void *p = m_pending[slot].data;
+	uint8_t offset = m_pending[slot].offset;
 	for (int i = 0; i < ent.prd_table_length; ++i) {
 		size_t prd_len = (cmdt.entries[i].byte_count & 0x7fffffff) + 1;
 
-		addr_t phys = 
-			static_cast<addr_t>(cmdt.entries[i].data_base_low) |
-			static_cast<addr_t>(cmdt.entries[i].data_base_high) << 32;
+		uintptr_t phys =
+			static_cast<uintptr_t>(cmdt.entries[i].data_base_low) |
+			static_cast<uintptr_t>(cmdt.entries[i].data_base_high) << 32;
+
+		void *mem = kutil::offset_pointer(pm->offset_virt(phys), offset);
+
+		log::debug(logs::driver, "Reading PRD %2d: %016lx->%016lx [%lxb]", i, mem, p, prd_len);
 
-		void *mem = pm->offset_virt(phys);
 		kutil::memcpy(p, mem, prd_len);
-		p = kutil::offset_pointer(p, prd_len);
+		p = kutil::offset_pointer(p, prd_len - offset);
+		count += (prd_len - offset);
+		offset = 0;
 
 		pm->unmap_pages(mem, page_count(prd_len));
 	}
 
+	m_pending[slot].count = count;
+	m_pending[slot].type = command_type::finished;
+	m_pending[slot].data = nullptr;
+}
+
+static void
+ident_strcpy(uint16_t *from, int words, char *dest)
+{
+	for (int i = 0; i < words; ++i) {
+		*dest++ = *from >> 8;
+		*dest++ = *from & 0xff;
+		from++;
+	}
+	*dest = 0;
+}
+
+void
+port::finish_identify(int slot)
+{
+	page_manager *pm = page_manager::get();
+	cmd_table &cmdt = m_cmd_table[slot];
+	cmd_list_entry &ent = m_cmd_list[slot];
+
+	kassert(ent.prd_table_length == 1, "AHCI identify used multiple PRDs");
+
+	size_t prd_len = (cmdt.entries[0].byte_count & 0x7fffffff) + 1;
+
+	uintptr_t phys =
+		static_cast<uintptr_t>(cmdt.entries[0].data_base_low) |
+		static_cast<uintptr_t>(cmdt.entries[0].data_base_high) << 32;
+
+	log::debug(logs::driver, "Reading ident PRD:");
+
+	uint16_t *mem = reinterpret_cast<uint16_t *>(pm->offset_virt(phys));
+	char string[41];
+
+	ident_strcpy(&mem[10], 10, &string[0]);
+	log::debug(logs::driver, "    Device serial: %s", string);
+
+	ident_strcpy(&mem[23], 4, &string[0]);
+	log::debug(logs::driver, "   Device version: %s", string);
+
+	ident_strcpy(&mem[27], 20, &string[0]);
+	log::debug(logs::driver, "     Device model: %s", string);
+
+	uint32_t sectors = mem[60] | (mem[61] << 16);
+	log::debug(logs::driver, "      Max sectors: %xh", sectors);
+
+	uint16_t lb_size = mem[106];
+	log::debug(logs::driver, " lsects per psect: %d %s %s", 1 << (lb_size & 0xf),
+			lb_size & 0x20 ? "multiple logical per physical" : "",
+			lb_size & 0x10 ? "physical > 512b" : "");
+
+	uint32_t b_per_ls = 2 * (mem[117] | (mem[118] << 16));
+	log::debug(logs::driver, "      b per lsect: %d", b_per_ls);
+
+	/*
+	for (int i=0; i<256; i += 4)
+		log::debug(logs::driver, "  %3d: %04x  %3d: %04x  %3d: %04x  %3d: %04x",
+				i, mem[i], i+1, mem[i+1], i+2, mem[i+2], i+3, mem[i+3]);
+	*/
+
+	pm->unmap_pages(mem, page_count(prd_len));
+	m_pending[slot].type = command_type::none;
 }
 
 void
@@ -363,9 +535,9 @@ port::free_command(int slot)
 	for (int i = 0; i < ent.prd_table_length; ++i) {
 		size_t prd_len = (cmdt.entries[i].byte_count & 0x7fffffff) + 1;
 
-		addr_t phys =
-			static_cast<addr_t>(cmdt.entries[i].data_base_low) |
-			static_cast<addr_t>(cmdt.entries[i].data_base_high) << 32;
+		uintptr_t phys =
+			static_cast<uintptr_t>(cmdt.entries[i].data_base_low) |
+			static_cast<uintptr_t>(cmdt.entries[i].data_base_high) << 32;
 		void *mem = pm->offset_virt(phys);
 		pm->unmap_pages(mem, page_count(prd_len));
 	}
@@ -385,7 +557,7 @@ port::rebase()
 	size_t pages = 1 + page_count(prd_size * 32);
 
 	void *mem = pm->map_offset_pages(pages);
-	addr_t phys = pm->offset_phys(mem);
+	uintptr_t phys = pm->offset_phys(mem);
 
 	log::debug(logs::driver, "Rebasing address for AHCI port %d to %lx [%d]", m_index, mem, pages);
 
@@ -430,4 +602,22 @@ port::rebase()
 	start_commands();
 }
 
+void
+port::dump()
+{
+	console *cons = console::get();
+	static const char *regs[] = {
+		" CLB", "+CLB", "  FB", " +FB", "  IS", "  IE",
+		" CMD", nullptr, " TFD", " SIG", "SSTS", "SCTL", "SERR",
+		"SACT", "  CI", "SNTF", " FBS", "DEVS"
+	};
+
+	cons->printf("Port Registers:\n");
+	auto *data = reinterpret_cast<volatile uint32_t *>(m_data);
+	for (int i = 0; i < 18; ++i) {
+		if (regs[i]) cons->printf("  %s: %08x\n", regs[i], data[i]);
+	}
+	cons->putc('\n');
+}
+
 } // namespace ahci

src/drivers/ahci/port.h

@@ -4,25 +4,30 @@
 #include <stddef.h>
 #include <stdint.h>
 #include "kutil/vector.h"
+#include "block_device.h"
 
 namespace ahci {
 
 struct cmd_list_entry;
 struct cmd_table;
+struct fis_register_h2d;
+class hba;
 enum class sata_signature : uint32_t;
 enum class port_cmd : uint32_t;
 struct port_data;
 
 
 /// A port on an AHCI HBA
-class port
+class port :
+	public block_device
 {
 public:
 	/// Constructor.
-	/// \arg index Index of the port on its HBA
-	/// \arg data  Pointer to the device's registers for this port
-	/// \arg impl  Whether this port is marked as implemented in the HBA
-	port(uint8_t index, port_data *data, bool impl);
+	/// \arg device The HBA device this port belongs to
+	/// \arg index  Index of the port on its HBA
+	/// \arg data   Pointer to the device's registers for this port
+	/// \arg impl   Whether this port is marked as implemented in the HBA
+	port(hba *device, uint8_t index, port_data volatile *data, bool impl);
 
 	/// Destructor
 	~port();
@@ -37,6 +42,10 @@ public:
 	/// \returns  An enum representing the state
 	inline state get_state() const { return m_state; }
 
+	/// Check if this device is active
+	/// \returns  True if the device state is active
+	inline bool active() const { return m_state == state::active; }
+
 	/// Get the type signature of this device
 	/// \returns  An enum representing the type of device
 	inline sata_signature get_type() const { return m_type; }
@@ -53,16 +62,34 @@ public:
 	/// Stop command processing from this port
 	void stop_commands();
 
-	/// Read data from the drive.
-	/// \arg sector  Starting sector to read
+	/// Start a read operation from the drive.
+	/// \arg offset  Offset to start from
 	/// \arg length  Number of bytes to read
 	/// \arg dest    A buffer where the data will be placed
-	/// \returns     True if the command succeeded
-	bool read(uint64_t sector, size_t length, void *dest);
+	/// \returns     A handle to the read operation, or -1 on error
+	int read_async(uint64_t offset, size_t length, void *dest);
+
+	/// Read from the drive, blocking until finished.
+	/// \arg offset  Offset to start from
+	/// \arg length  Number of bytes to read
+	/// \arg dest    A buffer where the data will be placed
+	/// \returns     The number of bytes read
+	virtual size_t read(uint64_t offset, size_t length, void *dest);
+
+	/// Start an identify operation for the drive.
+	/// \returns     A handle to the read operation, or -1 on error
+	int identify_async();
+
+	/// Tell the HBA to reconnect to the SATA device. A successful
+	/// reconnect will kick off an identify command.
+	void sata_reconnect();
 
 	/// Handle an incoming interrupt
 	void handle_interrupt();
 
+	/// Dump the port registers to the console
+	void dump();
+
 private:
 	/// Rebase the port command structures to a new location in system
 	/// memory, to be allocated from the page manager.
@@ -70,8 +97,9 @@ private:
 
 	/// Initialize a command structure
 	/// \arg length  The number of bytes of data needed in the PRDs
+	/// \arg fis     [out] The FIS for this command
 	/// \returns     The index of the command slot, or -1 if none available
-	int make_command(size_t length);
+	int make_command(size_t length, fis_register_h2d **fis);
 
 	/// Send a constructed command to the hardware
 	/// \arg slot   The index of the command slot used
@@ -88,20 +116,29 @@ private:
 	/// \arg slot  The command slot that the read command used
 	void finish_read(int slot);
 
+	/// Finish an identify command started by `identify_async()`.
+	/// This will free the structures allocated, so `free_command()` is
+	/// not necessary.
+	/// \arg slot  The command slot that the read command used
+	void finish_identify(int slot);
+
 	sata_signature m_type;
 	uint8_t m_index;
 	state m_state;
 
-	port_data *m_data;
+	hba *m_hba;
+	port_data volatile *m_data;
 	void *m_fis;
 	cmd_list_entry *m_cmd_list;
 	cmd_table *m_cmd_table;
 
-	enum class command_type : uint8_t { none, read, write };
+	enum class command_type : uint8_t { none, read, write, identify, finished };
 
 	struct pending
 	{
 		command_type type;
+		uint8_t offset;
+		size_t count;
 		void *data;
 	};
 

src/drivers/nulldrv/main.cpp

@@ -0,0 +1 @@
+int main(int argc, const char **argv) { return 0; }

src/drivers/nulldrv/main.s

@@ -0,0 +1,25 @@
+global _start
+_start:
+	xor rbp, rbp	; Sentinel rbp
+	mov r11, 0		; counter
+	mov rbx, 20		; sleep timeout
+
+.loop:
+	mov rax, 1		; DEBUG syscall
+	;mov rax, 0		; NOOP syscall
+	;syscall
+	int 0xee
+
+	inc r11
+	cmp r11, 2
+
+	jle .loop
+
+	mov rax, 4		; SLEEP syscall
+	; syscall
+	int 0xee
+
+	add rbx, 20
+
+	mov r11, 0
+	jmp .loop

src/include/efi/efisetjmp.h

@@ -4,7 +4,16 @@
 #include <efi/eficompiler.h>
 #include <efi/efisetjmp_arch.h>
 
+#ifndef __has_builtin
+#define __has_builtin(x) 0
+#endif
+
+#if ! __has_builtin(setjmp)
 extern UINTN setjmp(jmp_buf *env) __attribute__((returns_twice));
+#endif
+
+#if ! __has_builtin(longjmp)
 extern VOID longjmp(jmp_buf *env, UINTN value) __attribute__((noreturn));
+#endif
 
 #endif /* GNU_EFI_SETJMP_H */

src/include/elf.h

@@ -0,0 +1,48 @@
+#pragma once
+
+/* elf.h - basic defines for external code written assuming <elf.h> works. Only
+ * Elf64 values are included.
+ */
+
+typedef uint16_t Elf64_Half;
+typedef uint32_t Elf64_Word;
+typedef int32_t Elf64_Sword;
+typedef uint64_t Elf64_Xword;
+typedef int64_t Elf64_Sxword;
+typedef uint64_t Elf64_Addr;
+typedef uint64_t Elf64_Off;
+typedef uint16_t Elf64_Section;
+typedef Elf64_Half Elf64_Versym;
+
+typedef struct {
+	Elf64_Addr r_offset;
+	Elf64_Xword r_info;
+} Elf64_Rel;
+
+typedef struct {
+	Elf64_Addr r_offset;
+	Elf64_Word r_info;
+	Elf64_Sword r_addend;
+} Elf64_Rela;
+
+typedef struct {
+	Elf64_Sxword d_tag;
+	union {
+		Elf64_Xword d_val;
+		Elf64_Addr d_ptr;
+	} d_un;
+} Elf64_Dyn;
+
+#define ELF64_R_TYPE(x) ((x) & 0xffffffff)
+
+typedef enum {
+	DT_NULL = 0,
+	DT_RELA = 7,
+	DT_RELASZ = 8,
+	DT_RELAENT = 9
+} ElfDynTag;
+
+typedef enum {
+	R_X86_64_NONE = 0,
+	R_X86_64_RELATIVE = 8
+} Elf_x86_64_RelType;

src/include/kernel_data.h

@@ -16,8 +16,8 @@ struct popcorn_data {
 	uint32_t _reserved0;
 	uint32_t flags;
 
-	void *font;
-	size_t font_length;
+	void *initrd;
+	size_t initrd_length;
 
 	void *data;
 	size_t data_length;

src/kernel/allocator.cpp

@@ -1,9 +0,0 @@
-#include "kutil/memory_manager.h"
-
-kutil::memory_manager g_kernel_memory_manager;
-
-// kutil malloc/free implementation
-namespace kutil {
-	void * malloc(size_t n) { return g_kernel_memory_manager.allocate(n); }
-	void free(void *p) { g_kernel_memory_manager.free(p); }
-}

src/kernel/apic.cpp

@@ -1,6 +1,7 @@
 #include "kutil/assert.h"
 #include "apic.h"
 #include "interrupts.h"
+#include "io.h"
 #include "log.h"
 #include "page_manager.h"
 
@@ -50,30 +51,96 @@ lapic::lapic(uint32_t *base, isr spurious) :
 }
 
 void
-lapic::enable_timer(isr vector, uint8_t divisor, uint32_t count, bool repeat)
+lapic::calibrate_timer()
 {
+	interrupts_disable();
+
+	log::info(logs::apic, "Calibrating APIC timer...");
+
+	// Set up PIT sleep
+	uint8_t command = 0x30; // channel 0, loybyte/highbyte, mode 0
+	outb(0x43, command);
+
+	const uint32_t initial = -1u;
+	enable_timer_internal(isr::isrSpurious, 1, initial, false);
+
+	const int iterations = 5;
+	for (int i=0; i<iterations; ++i) {
+		const uint16_t pit_33ms = 39375;
+		uint16_t pit_count = pit_33ms;
+		outb(0x40, pit_count & 0xff);
+		io_wait();
+		outb(0x40, (pit_count >> 8) & 0xff);
+
+
+		while (pit_count <= pit_33ms) {
+			outb(0x43, 0); // latch counter values
+			pit_count =
+				static_cast<uint16_t>(inb(0x40)) |
+				static_cast<uint16_t>(inb(0x40)) << 8;
+		}
+	}
+
+	uint32_t remain = stop_timer();
+	uint32_t ticks_total = initial - remain;
+	m_ticks_per_us = ticks_total / (iterations * 33000);
+	log::info(logs::apic, "APIC timer ticks %d times per nanosecond.", m_ticks_per_us);
+
+	interrupts_enable();
+}
+
+uint32_t
+lapic::enable_timer_internal(isr vector, uint8_t divisor, uint32_t count, bool repeat)
+{
+	uint32_t divbits = 0;
+
 	switch (divisor) {
-	case   1: divisor = 11; break;
-	case   2: divisor = 0; break;
-	case   4: divisor = 1; break;
-	case   8: divisor = 2; break;
-	case  16: divisor = 3; break;
-	case  32: divisor = 8; break;
-	case  64: divisor = 9; break;
-	case 128: divisor = 10; break;
+	case   1: divbits = 0xb; break;
+	case   2: divbits = 0x0; break;
+	case   4: divbits = 0x1; break;
+	case   8: divbits = 0x2; break;
+	case  16: divbits = 0x3; break;
+	case  32: divbits = 0x8; break;
+	case  64: divbits = 0x9; break;
+	case 128: divbits = 0xa; break;
 	default:
 		kassert(0, "Invalid divisor passed to lapic::enable_timer");
 	}
 
-	apic_write(m_base, 0x3e0, divisor);
-	apic_write(m_base, 0x380, count);
-
 	uint32_t lvte = static_cast<uint8_t>(vector);
 	if (repeat)
 		lvte |= 0x20000;
 
-	log::debug(logs::apic, "Enabling APIC timer with isr %d.", vector);
+	log::debug(logs::apic, "Enabling APIC timer count %ld, divisor %d, isr %02x",
+			count, divisor, vector);
+
 	apic_write(m_base, 0x320, lvte);
+	apic_write(m_base, 0x3e0, divbits);
+
+	reset_timer(count);
+	return count;
+}
+
+uint32_t
+lapic::enable_timer(isr vector, uint64_t interval, bool repeat)
+{
+	uint64_t ticks = interval * m_ticks_per_us;
+
+	int divisor = 1;
+	while (ticks > -1u) {
+		ticks /= 2;
+		divisor *= 2;
+	}
+
+	return enable_timer_internal(vector, divisor, static_cast<uint32_t>(ticks), repeat);
+}
+
+uint32_t
+lapic::reset_timer(uint32_t count)
+{
+	uint32_t remaining = apic_read(m_base, 0x390);
+	apic_write(m_base, 0x380, count);
+	return remaining;
 }
 
 void

src/kernel/apic.h

@@ -32,10 +32,19 @@ public:
 
 	/// Enable interrupts for the LAPIC timer.
 	/// \arg vector   Interrupt vector the timer should use
-	/// \arg divisor  The frequency divisor of the bus Hz (power of 2, <= 128)
-	/// \arg count    The count of ticks before an interrupt
+	/// \arg interval The timer interval, in microseconds
 	/// \arg repeat   If false, this timer is one-off, otherwise repeating
-	void enable_timer(isr vector, uint8_t divisor, uint32_t count, bool repeat = true);
+	/// \returns      The count of ticks the timer is set for
+	uint32_t enable_timer(isr vector, uint64_t interval, bool repeat = true);
+
+	/// Reset the timer countdown.
+	/// \arg count  The count of ticks before an interrupt, or 0 to stop the timer
+	/// \returns    The count of ticks that were remaining before reset
+	uint32_t reset_timer(uint32_t count);
+
+	/// Stop the timer.
+	/// \returns  The count of ticks remaining before an interrupt was to happen
+	inline uint32_t stop_timer() { return reset_timer(0); }
 
 	/// Enable interrupts for the LAPIC LINT0 pin.
 	/// \arg num      Local interrupt number (0 or 1)
@@ -46,6 +55,14 @@ public:
 
 	void enable();  ///< Enable servicing of interrupts
 	void disable(); ///< Disable (temporarily) servicing of interrupts
+
+	/// Calibrate the timer speed against the PIT
+	void calibrate_timer();
+
+private:
+	uint32_t enable_timer_internal(isr vector, uint8_t divisor, uint32_t count, bool repeat);
+
+	uint32_t m_ticks_per_us;
 };
 
 

src/kernel/assert.cpp

@@ -7,7 +7,7 @@ __kernel_assert(const char *file, unsigned line, const char *message)
 	console *cons = console::get();
 	if (cons) {
 		cons->set_color(9 , 0);
-		cons->puts("\n\n  ERROR: ");
+		cons->puts("\n\n       ERROR: ");
 		cons->puts(file);
 		cons->puts(":");
 		cons->put_dec(line);
@@ -15,15 +15,12 @@ __kernel_assert(const char *file, unsigned line, const char *message)
 		cons->puts(message);
 	}
 
-	__asm__ __volatile__( 
-		"movq %0, %%r8;"
-		"movq %1, %%r9;"
-		"movq %2, %%r10;"
-		"movq $0, %%rdx;"
-		"divq %%rdx;"
-		: // no outputs
-		: "r"((uint64_t)line), "r"(file), "r"(message)
-		: "rax", "rdx", "r8", "r9", "r10");
-
-	while (1);
+	__asm__ ( "int $0e7h" );
+	while (1) __asm__ ("hlt");
+}
+
+extern "C" [[noreturn]] void
+__assert_fail(const char *message, const char *file, unsigned int line, const char *function)
+{
+	__kernel_assert(file, line, message);
 }

src/kernel/block_device.h

@@ -0,0 +1,11 @@
+#pragma once
+/// \file block_device.h
+/// Interface definition for block devices
+#include <stddef.h>
+
+/// Interface for block devices
+class block_device
+{
+public:
+	virtual size_t read(size_t offset, size_t length, void *buffer) = 0;
+};

src/kernel/boot.s

@@ -28,8 +28,8 @@ _start:
 	extern kernel_main
 	call kernel_main
 
-	cli
-
+	; Kernel init is over, wait for the scheduler to
+	; take over
 .hang:
 	hlt
 	jmp .hang

src/kernel/console.cpp

@@ -1,8 +1,8 @@
 #include "kutil/coord.h"
+#include "kutil/guid.h"
 #include "kutil/memory.h"
 #include "console.h"
 #include "font.h"
-#include "memory.h"
 #include "screen.h"
 #include "serial.h"
 
@@ -334,6 +334,23 @@ void console::vprintf(const char *fmt, va_list args)
 					done = true;
 					break;
 
+				case 'G': {
+						// Special: GUID type
+						kutil::guid g = va_arg(args, kutil::guid);
+						put_hex<uint32_t>(g.a, 8, '0');
+						putc('-');
+						put_hex<uint16_t>((g.a >> 32) & 0xffff, 4, '0');
+						putc('-');
+						put_hex<uint16_t>((g.a >> 48) & 0xffff, 4, '0');
+						putc('-');
+						put_hex<uint16_t>((kutil::byteswap(g.b) >> 16) & 0xffff, 4, '0');
+						putc('-');
+						put_hex<uint16_t>(kutil::byteswap(g.b) & 0xffff, 4, '0');
+						put_hex<uint32_t>(kutil::byteswap(g.b >> 32), 8, '0');
+					}
+					done = true;
+					break;
+
 				case 'l':
 					switch (*r++) {
 						case 'x': put_hex<uint64_t>(va_arg(args, uint64_t), right ? width : -width, pad); done = true; break;

src/kernel/cpu.cpp

@@ -52,6 +52,7 @@ cpu_id::cpu_id()
 
 }
 
+
 cpu_id::regs
 cpu_id::get(uint32_t leaf, uint32_t sub) const
 {

src/kernel/cpu.h

@@ -1,5 +1,13 @@
 #pragma once
 
+struct cpu_state
+{
+	uint64_t ds;
+	uint64_t r15, r14, r13, r12, r11, r10, r9,  r8;
+	uint64_t rdi, rsi, rbp, rbx, rdx, rcx, rax;
+	uint64_t interrupt, errorcode;
+	uint64_t rip, cs, rflags, user_rsp, ss;
+};
 
 class cpu_id
 {

src/kernel/debug.cpp

@@ -0,0 +1,68 @@
+#include "console.h"
+#include "cpu.h"
+#include "debug.h"
+#include "gdt.h"
+
+#define print_reg(name, value) cons->printf("         %s: %016lx\n", name, (value));
+
+void
+print_regs(const cpu_state &regs)
+{
+	console *cons = console::get();
+
+	print_reg("rax", regs.rax);
+	print_reg("rbx", regs.rbx);
+	print_reg("rcx", regs.rcx);
+	print_reg("rdx", regs.rdx);
+	print_reg("rdi", regs.rdi);
+	print_reg("rsi", regs.rsi);
+
+	cons->puts("\n");
+	print_reg(" r8", regs.r8);
+	print_reg(" r9", regs.r9);
+	print_reg("r10", regs.r10);
+	print_reg("r11", regs.r11);
+	print_reg("r12", regs.r12);
+	print_reg("r13", regs.r13);
+	print_reg("r14", regs.r14);
+	print_reg("r15", regs.r15);
+
+	cons->puts("\n");
+	print_reg("rbp", regs.rbp);
+	print_reg("rsp", regs.user_rsp);
+	print_reg("sp0", tss_get_stack(0));
+
+	cons->puts("\n");
+	print_reg(" ds", regs.ds);
+	print_reg(" cs", regs.cs);
+	print_reg(" ss", regs.ss);
+
+	cons->puts("\n");
+	print_reg("rip", regs.rip);
+}
+
+void
+print_stacktrace(int skip)
+{
+	console *cons = console::get();
+	int frame = 0;
+	uint64_t bp = get_frame(skip);
+	while (bp) {
+		cons->printf("    frame %2d: %lx\n", frame, bp);
+		bp = get_frame(++frame + skip);
+	}
+}
+
+void
+print_stack(const cpu_state &regs)
+{
+	console *cons = console::get();
+
+	cons->puts("\nStack:\n");
+	uint64_t sp = regs.user_rsp;
+	while (sp <= regs.rbp) {
+		cons->printf("%016x: %016x\n", sp, *reinterpret_cast<uint64_t *>(sp));
+		sp += sizeof(uint64_t);
+	}
+}
+

src/kernel/debug.h

@@ -0,0 +1,19 @@
+#pragma once
+/// \file debug.h
+/// Debugging utilities
+
+#include <stdint.h>
+
+extern "C" {
+	uintptr_t get_rsp();
+	uintptr_t get_rip();
+	uintptr_t get_frame(int frame);
+
+}
+
+void print_regs(const cpu_state &regs);
+void print_stack(const cpu_state &regs);
+void print_stacktrace(int skip = 0);
+
+#define print_reg(name, value) cons->printf("         %s: %016lx\n", name, (value));
+

src/kernel/debug.s

@@ -1,10 +1,12 @@
+global get_rsp
+get_rsp:
+	mov rax, rsp
+	ret
 
-section .text
-global do_the_set_registers
-do_the_set_registers:
-	mov rax, 0xdeadbeef0badc0de
-	mov r8, rcx
-	mov r9, rdi
+global get_rip
+get_rip:
+	pop rax ; do the same thing as 'ret', except with 'jmp'
+	jmp rax ; with the return address still in rax
 
 global _halt
 _halt:

src/kernel/device_manager.cpp

@@ -4,20 +4,17 @@
 #include "kutil/assert.h"
 #include "kutil/memory.h"
 #include "acpi_tables.h"
-#include "ahci/driver.h"
 #include "apic.h"
 #include "console.h"
 #include "device_manager.h"
 #include "interrupts.h"
 #include "log.h"
-#include "memory.h"
 #include "page_manager.h"
 
 
 static const char expected_signature[] = "RSD PTR ";
 
 device_manager device_manager::s_instance(nullptr);
-ahci_driver ahcid;
 
 struct acpi1_rsdp
 {
@@ -42,29 +39,16 @@ struct acpi2_rsdp
 	uint8_t reserved[3];
 } __attribute__ ((packed));
 
-uint8_t
-acpi_checksum(const void *p, size_t len, size_t off = 0)
-{
-	uint8_t sum = 0;
-	const uint8_t *c = reinterpret_cast<const uint8_t *>(p);
-	for (int i = off; i < len; ++i) sum += c[i];
-	return sum;
-}
-
 bool
 acpi_table_header::validate(uint32_t expected_type) const
 {
-	if (acpi_checksum(this, length) != 0) return false;
+	if (kutil::checksum(this, length) != 0) return false;
 	return !expected_type || (expected_type == type);
 }
 
 
 void irq2_callback(void *)
 {
-	console *cons = console::get();
-	cons->set_color(11);
-	cons->puts(".");
-	cons->set_color();
 }
 
 void irq4_callback(void *)
@@ -87,7 +71,7 @@ device_manager::device_manager(const void *root_table) :
 		kassert(acpi1->signature[i] == expected_signature[i],
 				"ACPI RSDP table signature mismatch");
 
-	uint8_t sum = acpi_checksum(acpi1, sizeof(acpi1_rsdp), 0);
+	uint8_t sum = kutil::checksum(acpi1, sizeof(acpi1_rsdp), 0);
 	kassert(sum == 0, "ACPI 1.0 RSDP checksum mismatch.");
 
 	kassert(acpi1->revision > 1, "ACPI 1.0 not supported.");
@@ -95,7 +79,7 @@ device_manager::device_manager(const void *root_table) :
 	const acpi2_rsdp *acpi2 =
 		reinterpret_cast<const acpi2_rsdp *>(acpi1);
 
-	sum = acpi_checksum(acpi2, sizeof(acpi2_rsdp), sizeof(acpi1_rsdp));
+	sum = kutil::checksum(acpi2, sizeof(acpi2_rsdp), sizeof(acpi1_rsdp));
 	kassert(sum == 0, "ACPI 2.0 RSDP checksum mismatch.");
 
 	load_xsdt(reinterpret_cast<const acpi_xsdt *>(acpi2->xsdt_address));
@@ -222,8 +206,6 @@ device_manager::load_apic(const acpi_apic *apic)
 		p += length;
 	}
 
-	// m_lapic->enable_timer(isr::isrTimer, 128, 3000000);
-
 	for (uint8_t i = 0; i < m_ioapics[0]->get_num_gsi(); ++i) {
 		switch (i) {
 			case 2: break;
@@ -231,7 +213,6 @@ device_manager::load_apic(const acpi_apic *apic)
 		}
 	}
 
-	m_ioapics[0]->dump_redirs();
 	m_lapic->enable();
 }
 
@@ -292,6 +273,7 @@ device_manager::init_drivers()
 {
 	// Eventually this should be e.g. a lookup into a loadable driver list
 	// for now, just look for AHCI devices
+	/*
 	for (auto &device : m_devices) {
 		if (device.devclass() != 1 || device.subclass() != 6)
 			continue;
@@ -303,6 +285,7 @@ device_manager::init_drivers()
 
 		ahcid.register_device(&device);
 	}
+	*/
 }
 
 bool
@@ -320,3 +303,9 @@ device_manager::allocate_msi(const char *name, pci_device &device, irq_callback
 			static_cast<uint16_t>(vector));
 	return true;
 }
+
+void
+device_manager::register_block_device(block_device *blockdev)
+{
+	m_blockdevs.append(blockdev);
+}

src/kernel/device_manager.h

@@ -7,6 +7,7 @@
 struct acpi_xsdt;
 struct acpi_apic;
 struct acpi_mcfg;
+class block_device;
 class lapic;
 class ioapic;
 
@@ -65,6 +66,23 @@ public:
 		return false;
 	}
 
+	/// Register the existance of a block device.
+	/// \arg blockdev  Pointer to the block device
+	void register_block_device(block_device *blockdev);
+
+	/// Get the number of block devices in the system
+	/// \returns  A count of devices
+	inline unsigned get_num_block_devices() const { return m_blockdevs.count(); }
+
+	/// Get a block device
+	/// \arg i    Index of the device to get
+	/// \returns  A pointer to the requested device, or nullptr
+	inline block_device * get_block_device(unsigned i)
+	{
+		return i < m_blockdevs.count() ?
+			m_blockdevs[i] : nullptr;
+	}
+
 private:
 	/// Parse the ACPI XSDT and load relevant sub-tables.
 	/// \arg xsdt  Pointer to the XSDT from the firmware
@@ -100,6 +118,8 @@ private:
 	};
 	kutil::vector<irq_allocation> m_irqs;
 
+	kutil::vector<block_device *> m_blockdevs;
+
 	static device_manager s_instance;
 
 	device_manager() = delete;

src/kernel/fs/gpt.cpp

@@ -0,0 +1,114 @@
+#include "kutil/assert.h"
+#include "kutil/guid.h"
+#include "kutil/memory.h"
+#include "device_manager.h"
+#include "fs/gpt.h"
+#include "log.h"
+
+namespace fs {
+
+const kutil::guid efi_system_part = kutil::make_guid(0xC12A7328, 0xF81F, 0x11D2, 0xBA4B, 0x00A0C93EC93B);
+const kutil::guid efi_unused_part = kutil::make_guid(0, 0, 0, 0, 0);
+
+const uint64_t gpt_signature = 0x5452415020494645; // "EFI PART"
+const size_t block_size = 512;
+
+struct gpt_header
+{
+	uint64_t signature;
+	uint32_t revision;
+	uint32_t headersize;
+	uint32_t crc32;
+	uint32_t reserved;
+	uint64_t my_lba;
+	uint64_t alt_lba;
+	uint64_t first_usable_lba;
+	uint64_t last_usable_lba;
+	kutil::guid disk_guid;
+	uint64_t table_lba;
+	uint32_t entry_count;
+	uint32_t entry_length;
+	uint32_t table_crc32;
+} __attribute__ ((packed));
+
+struct gpt_entry
+{
+	kutil::guid type;
+	kutil::guid part_guid;
+	uint64_t start_lba;
+	uint64_t end_lba;
+	uint64_t attributes;
+	uint16_t name_wide[36];
+} __attribute__ ((packed));
+
+
+partition::partition(block_device *parent, size_t start, size_t length) :
+	m_parent(parent),
+	m_start(start),
+	m_length(length)
+{
+}
+
+size_t
+partition::read(size_t offset, size_t length, void *buffer)
+{
+	if (offset + length > m_length)
+		offset = m_length - offset;
+	return m_parent->read(m_start + offset, length, buffer);
+}
+
+
+unsigned
+partition::load(block_device *device)
+{
+	// Read LBA 1
+	uint8_t block[block_size];
+	size_t count = device->read(block_size, block_size, &block);
+	kassert(count == block_size, "Short read for GPT header.");
+
+	gpt_header *header = reinterpret_cast<gpt_header *>(&block);
+	if (header->signature != gpt_signature)
+		return 0;
+
+	size_t arraysize = header->entry_length * header->entry_count;
+	log::debug(logs::fs, "Found GPT header: %d paritions, size 0x%lx",
+			header->entry_count, arraysize);
+
+	uint8_t *array = new uint8_t[arraysize];
+	count = device->read(block_size * header->table_lba, arraysize, array);
+	kassert(count == arraysize, "Short read for GPT entry array.");
+
+	auto &dm = device_manager::get();
+
+	unsigned found = 0;
+	gpt_entry *entry0 = reinterpret_cast<gpt_entry *>(array);
+	for (uint32_t i = 0; i < header->entry_count; ++i) {
+		gpt_entry *entry = kutil::offset_pointer(entry0, i * header->entry_length);
+		if (entry->type == efi_unused_part) continue;
+
+		// TODO: real UTF16->UTF8
+		char name[sizeof(gpt_entry::name_wide) / 2];
+		for (int i = 0; i < sizeof(name); ++i)
+			name[i] = entry->name_wide[i];
+
+		log::debug(logs::fs, "Found partition %02x at %lx-%lx", i, entry->start_lba, entry->end_lba);
+		if (entry->type == efi_system_part)
+			log::debug(logs::fs, "   type EFI SYSTEM PARTITION");
+		else
+			log::debug(logs::fs, "   type %G", entry->type);
+		log::debug(logs::fs, "   name %s", name);
+		log::debug(logs::fs, "   attr %016lx", entry->attributes);
+
+		found += 1;
+		partition *part = new partition(
+				device,
+				entry->start_lba * block_size,
+				(entry->end_lba - entry->start_lba) * block_size);
+		dm.register_block_device(part);
+	}
+
+	return found;
+}
+
+
+} // namespace fs

src/kernel/fs/gpt.h

@@ -0,0 +1,37 @@
+#pragma once
+/// \file gpt.h
+/// Definitions for dealing with GUID Partition Tables
+#include "block_device.h"
+
+namespace fs {
+
+
+class partition :
+	public block_device
+{
+public:
+	/// Constructor.
+	/// \arg parent  The block device this partition is a part of
+	/// \arg start   The starting offset in bytes from the start of the parent
+	/// \arg lenght  The length in bytes of this partition
+	partition(block_device *parent, size_t start, size_t length);
+
+	/// Read bytes from the partition.
+	/// \arg offset  The offset in bytes at which to start reading
+	/// \arg length  The number of bytes to read
+	/// \arg buffer  [out] Data is read into this buffer
+	/// \returns     The number of bytes read
+	virtual size_t read(size_t offset, size_t length, void *buffer);
+
+	/// Find partitions on a block device and add them to the device manager
+	/// \arg device  The device to search for partitions
+	/// \returns     The number of partitions found
+	static unsigned load(block_device *device);
+
+private:
+	block_device *m_parent;
+	size_t m_start;
+	size_t m_length;
+};
+
+} // namespace fs

src/kernel/gdt.cpp

@@ -0,0 +1,273 @@
+#include <stdint.h>
+
+#include "kutil/assert.h"
+#include "kutil/enum_bitfields.h"
+#include "kutil/memory.h"
+#include "console.h"
+#include "log.h"
+
+
+enum class gdt_type : uint8_t
+{
+	accessed	= 0x01,
+	read_write	= 0x02,
+	conforming	= 0x04,
+	execute		= 0x08,
+	system		= 0x10,
+	ring1		= 0x20,
+	ring2		= 0x40,
+	ring3		= 0x60,
+	present		= 0x80
+};
+IS_BITFIELD(gdt_type);
+
+struct gdt_descriptor
+{
+	uint16_t limit_low;
+	uint16_t base_low;
+	uint8_t base_mid;
+	gdt_type type;
+	uint8_t size;
+	uint8_t base_high;
+} __attribute__ ((packed));
+
+struct tss_descriptor
+{
+	uint16_t limit_low;
+	uint16_t base_00;
+	uint8_t base_16;
+	gdt_type type;
+	uint8_t size;
+	uint8_t base_24;
+	uint32_t base_32;
+	uint32_t reserved;
+} __attribute__ ((packed));
+
+struct tss_entry
+{
+	uint32_t reserved0;
+
+	uint64_t rsp[3]; // stack pointers for CPL 0-2
+	uint64_t ist[8]; // ist[0] is reserved
+
+	uint64_t reserved1;
+	uint16_t reserved2;
+	uint16_t iomap_offset;
+} __attribute__ ((packed));
+
+struct idt_descriptor
+{
+	uint16_t base_low;
+	uint16_t selector;
+	uint8_t ist;
+	uint8_t flags;
+	uint16_t base_mid;
+	uint32_t base_high;
+	uint32_t reserved;		// must be zero
+} __attribute__ ((packed));
+
+struct table_ptr
+{
+	uint16_t limit;
+	uint64_t base;
+} __attribute__ ((packed));
+
+
+gdt_descriptor g_gdt_table[10];
+idt_descriptor g_idt_table[256];
+table_ptr g_gdtr;
+table_ptr g_idtr;
+tss_entry g_tss;
+
+
+extern "C" {
+	void idt_write();
+	void idt_load();
+
+	void gdt_write(uint16_t cs, uint16_t ds, uint16_t tr);
+	void gdt_load();
+}
+
+void
+gdt_set_entry(uint8_t i, uint32_t base, uint64_t limit, bool is64, gdt_type type)
+{
+	g_gdt_table[i].limit_low = limit & 0xffff;
+	g_gdt_table[i].size = (limit >> 16) & 0xf;
+	g_gdt_table[i].size |= (is64 ? 0xa0 : 0xc0);
+
+	g_gdt_table[i].base_low = base & 0xffff;
+	g_gdt_table[i].base_mid = (base >> 16) & 0xff;
+	g_gdt_table[i].base_high = (base >> 24) & 0xff;
+
+	g_gdt_table[i].type = type | gdt_type::system | gdt_type::present;
+}
+
+void
+tss_set_entry(uint8_t i, uint64_t base, uint64_t limit)
+{
+	tss_descriptor tssd;
+	tssd.limit_low = limit & 0xffff;
+	tssd.size = (limit >> 16) & 0xf;
+
+	tssd.base_00 = base & 0xffff;
+	tssd.base_16 = (base >> 16) & 0xff;
+	tssd.base_24 = (base >> 24) & 0xff;
+	tssd.base_32 = (base >> 32) & 0xffffffff;
+	tssd.reserved = 0;
+
+	tssd.type =
+		gdt_type::accessed |
+		gdt_type::execute |
+		gdt_type::ring3 |
+		gdt_type::present;
+	kutil::memcpy(&g_gdt_table[i], &tssd, sizeof(tss_descriptor));
+}
+
+void
+idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags)
+{
+	g_idt_table[i].base_low = addr & 0xffff;
+	g_idt_table[i].base_mid = (addr >> 16) & 0xffff;
+	g_idt_table[i].base_high = (addr >> 32) & 0xffffffff;
+	g_idt_table[i].selector = selector;
+	g_idt_table[i].flags = flags;
+	g_idt_table[i].ist = 0;
+	g_idt_table[i].reserved = 0;
+}
+
+void
+tss_set_stack(int ring, uintptr_t rsp)
+{
+	kassert(ring < 3, "Bad ring passed to tss_set_stack.");
+	g_tss.rsp[ring] = rsp;
+}
+
+uintptr_t
+tss_get_stack(int ring)
+{
+	kassert(ring < 3, "Bad ring passed to tss_get_stack.");
+	return g_tss.rsp[ring];
+}
+
+void
+gdt_init()
+{
+	kutil::memset(&g_gdt_table, 0, sizeof(g_gdt_table));
+	kutil::memset(&g_idt_table, 0, sizeof(g_idt_table));
+
+	g_gdtr.limit = sizeof(g_gdt_table) - 1;
+	g_gdtr.base = reinterpret_cast<uint64_t>(&g_gdt_table);
+
+	// Kernel CS/SS - always 64bit
+	gdt_set_entry(1, 0, 0xfffff,  true, gdt_type::read_write | gdt_type::execute);
+	gdt_set_entry(2, 0, 0xfffff,  true, gdt_type::read_write);
+
+	// User CS32/SS/CS64 - layout expected by SYSRET
+	gdt_set_entry(3, 0, 0xfffff,  false, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+	gdt_set_entry(4, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write);
+	gdt_set_entry(5, 0, 0xfffff,  true, gdt_type::ring3 | gdt_type::read_write | gdt_type::execute);
+
+	kutil::memset(&g_tss, 0, sizeof(tss_entry));
+	g_tss.iomap_offset = sizeof(tss_entry);
+
+	uintptr_t tss_base = reinterpret_cast<uintptr_t>(&g_tss);
+
+	// Note that this takes TWO GDT entries
+	tss_set_entry(6, tss_base, sizeof(tss_entry));
+
+	gdt_write(1 << 3, 2 << 3, 6 << 3);
+
+	g_idtr.limit = sizeof(g_idt_table) - 1;
+	g_idtr.base = reinterpret_cast<uint64_t>(&g_idt_table);
+
+	idt_write();
+}
+
+void
+gdt_dump()
+{
+	const table_ptr &table = g_gdtr;
+
+	console *cons = console::get();
+	cons->printf("         GDT: loc:%lx size:%d\n", table.base, table.limit+1);
+
+	int count = (table.limit + 1) / sizeof(gdt_descriptor);
+	const gdt_descriptor *gdt =
+		reinterpret_cast<const gdt_descriptor *>(table.base);
+
+	for (int i = 0; i < count; ++i) {
+		uint32_t base =
+			(gdt[i].base_high << 24) |
+			(gdt[i].base_mid << 16) |
+			gdt[i].base_low;
+
+		uint32_t limit =
+			static_cast<uint32_t>(gdt[i].size & 0x0f) << 16 |
+			gdt[i].limit_low;
+
+		cons->printf("          %02d:", i);
+		if (! bitfield_has(gdt[i].type, gdt_type::present)) {
+			cons->puts(" Not Present\n");
+			continue;
+		}
+
+		cons->printf(" Base %08x  limit %05x ", base, limit);
+
+		switch (gdt[i].type & gdt_type::ring3) {
+			case gdt_type::ring3: cons->puts("ring3"); break;
+			case gdt_type::ring2: cons->puts("ring2"); break;
+			case gdt_type::ring1: cons->puts("ring1"); break;
+			default: cons->puts("ring0"); break;
+		}
+
+		cons->printf(" %s %s %s %s %s %s %s\n",
+			bitfield_has(gdt[i].type, gdt_type::accessed) ? "A" : " ",
+			bitfield_has(gdt[i].type, gdt_type::read_write) ? "RW" : "  ",
+			bitfield_has(gdt[i].type, gdt_type::conforming) ? "C" : " ",
+			bitfield_has(gdt[i].type, gdt_type::execute) ? "EX" : "  ",
+			bitfield_has(gdt[i].type, gdt_type::system) ? "S" : " ",
+			(gdt[i].size & 0x80) ? "KB" : " B",
+			(gdt[i].size & 0x60) == 0x20 ? "64" :
+				(gdt[i].size & 0x60) == 0x40 ? "32" : "16");
+	}
+}
+
+void
+idt_dump()
+{
+	const table_ptr &table = g_idtr;
+
+	log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", table.base, table.limit+1);
+
+	int count = (table.limit + 1) / sizeof(idt_descriptor);
+	const idt_descriptor *idt =
+		reinterpret_cast<const idt_descriptor *>(table.base);
+
+	for (int i = 0; i < count; ++i) {
+		uint64_t base =
+			(static_cast<uint64_t>(idt[i].base_high) << 32) |
+			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
+			idt[i].base_low;
+
+		char const *type;
+		switch (idt[i].flags & 0xf) {
+			case 0x5: type = " 32tsk "; break;
+			case 0x6: type = " 16int "; break;
+			case 0x7: type = " 16trp "; break;
+			case 0xe: type = " 32int "; break;
+			case 0xf: type = " 32trp "; break;
+			default:  type = " ????? "; break;
+		}
+
+		if (idt[i].flags & 0x80) {
+			log::debug(logs::boot,
+					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
+					(idt[i].selector & 0x3),
+					((idt[i].selector & 0x4) >> 2),
+					(idt[i].selector >> 3),
+					idt[i].ist,
+					type,
+					((idt[i].flags >> 5) & 0x3));
+		}
+	}
+}

src/kernel/gdt.h

@@ -0,0 +1,31 @@
+#pragma once
+/// \file gdt.h
+/// Definitions relating to system descriptor tables: GDT, IDT, TSS
+#include <stdint.h>
+
+/// Set up the GDT and TSS, and switch segment registers to point
+/// to them.
+void gdt_init();
+
+/// Set an entry in the IDT
+/// \arg i         Index in the IDT (vector of the interrupt this handles)
+/// \arg addr      Address of the handler
+/// \arg selector  GDT selector to set when invoking this handler
+/// \arg flags     Descriptor flags to set
+void idt_set_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags);
+
+/// Set the stack pointer for a given ring in the TSS
+/// \arg ring  Ring to set for (0-2)
+/// \arg rsp   Stack pointer to set
+void tss_set_stack(int ring, uintptr_t rsp);
+
+/// Get the stack pointer for a given ring in the TSS
+/// \arg ring  Ring to get (0-2)
+/// \returns   Stack pointers for that ring
+uintptr_t tss_get_stack(int ring);
+
+/// Dump information about the current GDT to the screen
+void gdt_dump();
+
+/// Dump information about the current IDT to the screen
+void idt_dump();

src/kernel/gdt.s

@@ -0,0 +1,35 @@
+extern g_idtr
+extern g_gdtr
+
+global idt_write
+idt_write:
+	lidt [rel g_idtr]
+	ret
+
+global idt_load
+idt_load:
+	sidt [rel g_idtr]
+	ret
+
+global gdt_write
+gdt_write:
+	lgdt [rel g_gdtr]
+	mov ax, si ; second arg is data segment
+	mov ds, ax
+	mov es, ax
+	mov fs, ax
+	mov gs, ax
+	mov ss, ax
+	push qword rdi ; first arg is code segment
+	lea rax, [rel .next]
+	push rax
+	o64 retf
+.next:
+	ltr dx
+	ret
+
+global gdt_load
+gdt_load:
+	sgdt [rel g_gdtr]
+	ret
+

src/kernel/interrupt_isrs.inc

@@ -134,9 +134,115 @@ IRQ (0x7d, 0x5d, irq5D)
 IRQ (0x7e, 0x5e, irq5E)
 IRQ (0x7f, 0x5f, irq5F)
 
-ISR (0xec, isrTimer)
-ISR (0xed, isrLINT0)
-ISR (0xee, isrLINT1)
+IRQ (0x80, 0x60, irq60)
+IRQ (0x81, 0x61, irq61)
+IRQ (0x82, 0x62, irq62)
+IRQ (0x83, 0x63, irq63)
+IRQ (0x84, 0x64, irq64)
+IRQ (0x85, 0x65, irq65)
+IRQ (0x86, 0x66, irq66)
+IRQ (0x87, 0x67, irq67)
+IRQ (0x88, 0x68, irq68)
+IRQ (0x89, 0x69, irq69)
+IRQ (0x8a, 0x6a, irq6A)
+IRQ (0x8b, 0x6b, irq6B)
+IRQ (0x8c, 0x6c, irq6C)
+IRQ (0x8d, 0x6d, irq6D)
+IRQ (0x8e, 0x6e, irq6E)
+IRQ (0x8f, 0x6f, irq6F)
+
+IRQ (0x90, 0x70, irq70)
+IRQ (0x91, 0x71, irq71)
+IRQ (0x92, 0x72, irq72)
+IRQ (0x93, 0x73, irq73)
+IRQ (0x94, 0x74, irq74)
+IRQ (0x95, 0x75, irq75)
+IRQ (0x96, 0x76, irq76)
+IRQ (0x97, 0x77, irq77)
+IRQ (0x98, 0x78, irq78)
+IRQ (0x99, 0x79, irq79)
+IRQ (0x9a, 0x7a, irq7A)
+IRQ (0x9b, 0x7b, irq7B)
+IRQ (0x9c, 0x7c, irq7C)
+IRQ (0x9d, 0x7d, irq7D)
+IRQ (0x9e, 0x7e, irq7E)
+IRQ (0x9f, 0x7f, irq7F)
+
+IRQ (0xa0, 0x80, irq80)
+IRQ (0xa1, 0x81, irq81)
+IRQ (0xa2, 0x82, irq82)
+IRQ (0xa3, 0x83, irq83)
+IRQ (0xa4, 0x84, irq84)
+IRQ (0xa5, 0x85, irq85)
+IRQ (0xa6, 0x86, irq86)
+IRQ (0xa7, 0x87, irq87)
+IRQ (0xa8, 0x88, irq88)
+IRQ (0xa9, 0x89, irq89)
+IRQ (0xaa, 0x8a, irq8A)
+IRQ (0xab, 0x8b, irq8B)
+IRQ (0xac, 0x8c, irq8C)
+IRQ (0xad, 0x8d, irq8D)
+IRQ (0xae, 0x8e, irq8E)
+IRQ (0xaf, 0x8f, irq8F)
+
+IRQ (0xb0, 0x90, irq90)
+IRQ (0xb1, 0x91, irq91)
+IRQ (0xb2, 0x92, irq92)
+IRQ (0xb3, 0x93, irq93)
+IRQ (0xb4, 0x94, irq94)
+IRQ (0xb5, 0x95, irq95)
+IRQ (0xb6, 0x96, irq96)
+IRQ (0xb7, 0x97, irq97)
+IRQ (0xb8, 0x98, irq98)
+IRQ (0xb9, 0x99, irq99)
+IRQ (0xba, 0x9a, irq9A)
+IRQ (0xbb, 0x9b, irq9B)
+IRQ (0xbc, 0x9c, irq9C)
+IRQ (0xbd, 0x9d, irq9D)
+IRQ (0xbe, 0x9e, irq9E)
+IRQ (0xbf, 0x9f, irq9F)
+
+IRQ (0xc0, 0xa0, irqA0)
+IRQ (0xc1, 0xa1, irqA1)
+IRQ (0xc2, 0xa2, irqA2)
+IRQ (0xc3, 0xa3, irqA3)
+IRQ (0xc4, 0xa4, irqA4)
+IRQ (0xc5, 0xa5, irqA5)
+IRQ (0xc6, 0xa6, irqA6)
+IRQ (0xc7, 0xa7, irqA7)
+IRQ (0xc8, 0xa8, irqA8)
+IRQ (0xc9, 0xa9, irqA9)
+IRQ (0xca, 0xaa, irqAA)
+IRQ (0xcb, 0xab, irqAB)
+IRQ (0xcc, 0xac, irqAC)
+IRQ (0xcd, 0xad, irqAD)
+IRQ (0xce, 0xae, irqAE)
+IRQ (0xcf, 0xaf, irqAF)
+
+IRQ (0xd0, 0xb0, irqB0)
+IRQ (0xd1, 0xb1, irqB1)
+IRQ (0xd2, 0xb2, irqB2)
+IRQ (0xd3, 0xb3, irqB3)
+IRQ (0xd4, 0xb4, irqB4)
+IRQ (0xd5, 0xb5, irqB5)
+IRQ (0xd6, 0xb6, irqB6)
+IRQ (0xd7, 0xb7, irqB7)
+IRQ (0xd8, 0xb8, irqB8)
+IRQ (0xd9, 0xb9, irqB9)
+IRQ (0xda, 0xba, irqBA)
+IRQ (0xdb, 0xbb, irqBB)
+IRQ (0xdc, 0xbc, irqBC)
+IRQ (0xdd, 0xbd, irqBD)
+IRQ (0xde, 0xbe, irqBE)
+IRQ (0xdf, 0xbf, irqBF)
+
+
+ISR (0xe0, isrTimer)
+ISR (0xe1, isrLINT0)
+ISR (0xe2, isrLINT1)
+ISR (0xe4, isrAssert)
+
+UISR(0xee, isrSyscall)
 ISR (0xef, isrSpurious)
 
 ISR (0xf0, isrIgnore0)

src/kernel/interrupts.cpp

@@ -1,85 +1,38 @@
-#include <stddef.h>
 #include <stdint.h>
 
-#include "kutil/enum_bitfields.h"
-#include "kutil/memory.h"
+#include "apic.h"
 #include "console.h"
+#include "cpu.h"
+#include "debug.h"
 #include "device_manager.h"
+#include "gdt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "log.h"
+#include "scheduler.h"
+#include "syscall.h"
 
-enum class gdt_flags : uint8_t
-{
-	ac = 0x01,
-	rw = 0x02,
-	dc = 0x04,
-	ex = 0x08,
-	r1 = 0x20,
-	r2 = 0x40,
-	r3 = 0x60,
-	pr = 0x80,
-
-	res_1 = 0x10
-};
-IS_BITFIELD(gdt_flags);
-
-struct gdt_descriptor
-{
-	uint16_t limit_low;
-	uint16_t base_low;
-	uint8_t base_mid;
-	uint8_t flags;
-	uint8_t granularity;
-	uint8_t base_high;
-} __attribute__ ((packed));
-
-struct idt_descriptor
-{
-	uint16_t base_low;
-	uint16_t selector;
-	uint8_t ist;
-	uint8_t flags;
-	uint16_t base_mid;
-	uint32_t base_high;
-	uint32_t reserved;		// must be zero
-} __attribute__ ((packed));
-
-struct table_ptr
-{
-	uint16_t limit;
-	uint64_t base;
-} __attribute__ ((packed));
-
-gdt_descriptor g_gdt_table[10];
-idt_descriptor g_idt_table[256];
-table_ptr g_gdtr;
-table_ptr g_idtr;
-
-struct registers;
+static const uint16_t PIC1 = 0x20;
+static const uint16_t PIC2 = 0xa0;
 
 extern "C" {
-	void idt_write();
-	void idt_load();
+	void _halt();
 
-	void gdt_write();
-	void gdt_load();
-
-	void isr_handler(registers);
-	void irq_handler(registers);
+	uintptr_t isr_handler(uintptr_t, cpu_state);
+	uintptr_t irq_handler(uintptr_t, cpu_state);
+	uintptr_t syscall_handler(uintptr_t, cpu_state);
 
 #define ISR(i, name)     extern void name ();
 #define EISR(i, name)    extern void name ();
+#define UISR(i, name)    extern void name ();
 #define IRQ(i, q, name)  extern void name ();
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 }
 
-void idt_dump(const table_ptr &table);
-void gdt_dump(const table_ptr &table);
-
 isr
 operator+(const isr &lhs, int rhs)
 {
@@ -93,9 +46,11 @@ get_irq(unsigned vector)
 	switch (vector) {
 #define ISR(i, name)
 #define EISR(i, name)
+#define UISR(i, name)
 #define IRQ(i, q, name) case i : return q;
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 
@@ -103,43 +58,12 @@ get_irq(unsigned vector)
 	}
 }
 
-void
-set_gdt_entry(uint8_t i, uint32_t base, uint32_t limit, bool is64, gdt_flags flags)
-{
-	g_gdt_table[i].limit_low = limit & 0xffff;
-
-	g_gdt_table[i].base_low = base & 0xffff;
-	g_gdt_table[i].base_mid = (base >> 16) & 0xff;
-	g_gdt_table[i].base_high = (base >> 24) & 0xff;
-
-	g_gdt_table[i].granularity =
-			((limit >> 16) & 0xf) | (is64 ? 0xa0 : 0xc0);
-
-	g_gdt_table[i].flags = static_cast<uint8_t>(flags | gdt_flags::res_1 | gdt_flags::pr);
-}
-
-void
-set_idt_entry(uint8_t i, uint64_t addr, uint16_t selector, uint8_t flags)
-{
-	g_idt_table[i].base_low = addr & 0xffff;
-	g_idt_table[i].base_mid = (addr >> 16) & 0xffff;
-	g_idt_table[i].base_high = (addr >> 32) & 0xffffffff;
-	g_idt_table[i].selector = selector;
-	g_idt_table[i].flags = flags;
-	g_idt_table[i].ist = 0;
-	g_idt_table[i].reserved = 0;
-}
-
 static void
 disable_legacy_pic()
 {
-
-	static const uint16_t PIC1 = 0x20;
-	static const uint16_t PIC2 = 0xa0;
-
 	// Mask all interrupts
-	outb(0xa1, 0xff);
-	outb(0x21, 0xff);
+	outb(PIC2+1, 0xfc);
+	outb(PIC1+1, 0xff);
 
 	// Start initialization sequence
 	outb(PIC1, 0x11); io_wait();
@@ -147,7 +71,7 @@ disable_legacy_pic()
 
 	// Remap into ignore ISRs
 	outb(PIC1+1, static_cast<uint8_t>(isr::isrIgnore0)); io_wait();
-	outb(PIC2+1, static_cast<uint8_t>(isr::isrIgnore0)); io_wait();
+	outb(PIC2+1, static_cast<uint8_t>(isr::isrIgnore8)); io_wait();
 
 	// Tell PICs about each other
 	outb(PIC1+1, 0x04); io_wait();
@@ -164,95 +88,66 @@ enable_serial_interrupts()
 void
 interrupts_init()
 {
-	kutil::memset(&g_gdt_table, 0, sizeof(g_gdt_table));
-	kutil::memset(&g_idt_table, 0, sizeof(g_idt_table));
-
-	g_gdtr.limit = sizeof(g_gdt_table) - 1;
-	g_gdtr.base = reinterpret_cast<uint64_t>(&g_gdt_table);
-
-	set_gdt_entry(1, 0, 0xfffff, false, gdt_flags::rw);
-	set_gdt_entry(2, 0, 0xfffff, false, gdt_flags::rw | gdt_flags::ex | gdt_flags::dc);
-	set_gdt_entry(3, 0, 0xfffff, false, gdt_flags::rw);
-	set_gdt_entry(4, 0, 0xfffff, false, gdt_flags::rw | gdt_flags::ex);
-
-	set_gdt_entry(6, 0, 0xfffff, false, gdt_flags::rw);
-	set_gdt_entry(7, 0, 0xfffff,  true, gdt_flags::rw | gdt_flags::ex);
-
-	gdt_write();
-
-	g_idtr.limit = sizeof(g_idt_table) - 1;
-	g_idtr.base = reinterpret_cast<uint64_t>(&g_idt_table);
-
-#define ISR(i, name)     set_idt_entry(i, reinterpret_cast<uint64_t>(& name), 0x38, 0x8e);
-#define EISR(i, name)    set_idt_entry(i, reinterpret_cast<uint64_t>(& name), 0x38, 0x8e);
-#define IRQ(i, q, name)  set_idt_entry(i, reinterpret_cast<uint64_t>(& name), 0x38, 0x8e);
+#define ISR(i, name)     idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
+#define EISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
+#define UISR(i, name)    idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0xee);
+#define IRQ(i, q, name)  idt_set_entry(i, reinterpret_cast<uint64_t>(& name), 0x08, 0x8e);
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 
-	idt_write();
 	disable_legacy_pic();
 	enable_serial_interrupts();
 
 	log::info(logs::boot, "Interrupts enabled.");
 }
 
-struct registers
-{
-	uint64_t ds;
-	uint64_t rdi, rsi, rbp, rsp, rbx, rdx, rcx, rax;
-	uint64_t interrupt, errorcode;
-	uint64_t rip, cs, eflags, user_esp, ss;
-};
-
-#define print_reg(name, value) cons->printf("         %s: %016lx\n", name, (value));
-
-extern "C" uint64_t get_frame(int frame);
-
-void
-print_stacktrace(int skip = 0)
-{
-	console *cons = console::get();
-	int frame = 0;
-	uint64_t bp = get_frame(skip);
-	while (bp) {
-		cons->printf("    frame %2d: %lx\n", frame, bp);
-		bp = get_frame(++frame + skip);
-	}
-}
-
-void
-isr_handler(registers regs)
+uintptr_t
+isr_handler(uintptr_t return_rsp, cpu_state regs)
 {
 	console *cons = console::get();
 
 	switch (static_cast<isr>(regs.interrupt & 0xff)) {
-	case isr::isrTimer:
-		cons->puts("\nTICK\n");
-		break;
 
-	case isr::isrLINT0:
-		cons->puts("\nLINT0\n");
-		break;
+	case isr::isrGPFault: {
+			cons->set_color(9);
+			cons->puts("\nGeneral Protection Fault:\n");
+			cons->set_color();
 
-	case isr::isrLINT1:
-		cons->puts("\nLINT1\n");
-		break;
+			cons->printf("   errorcode: %lx", regs.errorcode);
+			if (regs.errorcode & 0x01) cons->puts(" external");
 
-	case isr::isrIgnore0:
-	case isr::isrIgnore1:
-	case isr::isrIgnore2:
-	case isr::isrIgnore3:
-	case isr::isrIgnore4:
-	case isr::isrIgnore5:
-	case isr::isrIgnore6:
-	case isr::isrIgnore7:
+			int index = (regs.errorcode & 0xffff) >> 4;
+			if (index) {
+				switch ((regs.errorcode & 0x07) >> 1) {
+				case 0:
+					cons->printf(" GDT[%x]\n", index);
+					gdt_dump();
+					break;
 
-		/*
-		cons->printf("\nIGNORED PIC INTERRUPT %d\n",
-				(regs.interrupt % 0xff) - 0xf0);
-		*/
+				case 1:
+				case 3:
+					cons->printf(" IDT[%x]\n", index);
+					idt_dump();
+					break;
+
+				default:
+					cons->printf(" LDT[%x]??\n", index);
+					break;
+				}
+			} else {
+				cons->putc('\n');
+			}
+			print_regs(regs);
+			/*
+			print_stacktrace(2);
+			print_stack(regs);
+			*/
+
+		}
+		_halt();
 		break;
 
 	case isr::isrPageFault: {
@@ -272,50 +167,91 @@ isr_handler(registers regs)
 			__asm__ __volatile__ ("mov %%cr2, %0" : "=r"(cr2));
 			print_reg("cr2", cr2);
 
+			print_reg("rsp", regs.user_rsp);
 			print_reg("rip", regs.rip);
 
 			cons->puts("\n");
 			print_stacktrace(2);
 		}
-		while(1) asm("hlt");
+		_halt();
+		break;
+
+	case isr::isrTimer: {
+			scheduler &s = scheduler::get();
+			return_rsp = s.tick(return_rsp);
+		}
+		break;
+
+	case isr::isrLINT0:
+		cons->puts("\nLINT0\n");
+		break;
+
+	case isr::isrLINT1:
+		cons->puts("\nLINT1\n");
+		break;
+
+	case isr::isrAssert: {
+			cons->set_color();
+			print_regs(regs);
+			print_stacktrace(2);
+		}
+		_halt();
+		break;
+
+	case isr::isrSyscall: {
+			return_rsp = syscall_dispatch(return_rsp, regs);
+		}
+		break;
+
+	case isr::isrSpurious:
+		// No EOI for the spurious interrupt
+		return return_rsp;
+
+	case isr::isrIgnore0:
+	case isr::isrIgnore1:
+	case isr::isrIgnore2:
+	case isr::isrIgnore3:
+	case isr::isrIgnore4:
+	case isr::isrIgnore5:
+	case isr::isrIgnore6:
+	case isr::isrIgnore7:
+		//cons->printf("\nIGNORED: %02x\n", regs.interrupt);
+		outb(PIC1, 0x20);
+		break;
+
+	case isr::isrIgnore8:
+	case isr::isrIgnore9:
+	case isr::isrIgnoreA:
+	case isr::isrIgnoreB:
+	case isr::isrIgnoreC:
+	case isr::isrIgnoreD:
+	case isr::isrIgnoreE:
+	case isr::isrIgnoreF:
+		//cons->printf("\nIGNORED: %02x\n", regs.interrupt);
+		outb(PIC1, 0x20);
+		outb(PIC2, 0x20);
 		break;
 
 	default:
 		cons->set_color(9);
-		cons->puts("\nReceived ISR interrupt:\n");
+		cons->printf("\nReceived %02x interrupt:\n",
+				(static_cast<isr>(regs.interrupt)));
+
 		cons->set_color();
+		cons->printf("         ISR: %02lx  ERR: %lx\n\n",
+				regs.interrupt, regs.errorcode);
 
-		cons->printf("         ISR: %02lx\n", regs.interrupt);
-		cons->printf("         ERR: %lx\n", regs.errorcode);
-		cons->puts("\n");
-
-		print_reg(" ds", regs.ds);
-		print_reg("rdi", regs.rdi);
-		print_reg("rsi", regs.rsi);
-		print_reg("rbp", regs.rbp);
-		print_reg("rsp", regs.rsp);
-		print_reg("rbx", regs.rbx);
-		print_reg("rdx", regs.rdx);
-		print_reg("rcx", regs.rcx);
-		print_reg("rax", regs.rax);
-		cons->puts("\n");
-
-		print_reg("rip", regs.rip);
-		print_reg(" cs", regs.cs);
-		print_reg(" ef", regs.eflags);
-		print_reg("esp", regs.user_esp);
-		print_reg(" ss", regs.ss);
-
-		cons->puts("\n");
-		print_stacktrace(2);
-		while(1) asm("hlt");
+		print_regs(regs);
+		//print_stacktrace(2);
+		_halt();
 	}
-
 	*reinterpret_cast<uint32_t *>(0xffffff80fee000b0) = 0;
+
+	return return_rsp;
 }
 
-void
-irq_handler(registers regs)
+uintptr_t
+irq_handler(uintptr_t return_rsp, cpu_state regs)
 {
 	console *cons = console::get();
 	uint8_t irq = get_irq(regs.interrupt);
@@ -324,101 +260,16 @@ irq_handler(registers regs)
 		cons->printf("\nReceived unknown IRQ: %d (vec %d)\n",
 				irq, regs.interrupt);
 		cons->set_color();
-
-		print_reg(" ds", regs.ds);
-		print_reg("rdi", regs.rdi);
-		print_reg("rsi", regs.rsi);
-		print_reg("rbp", regs.rbp);
-		print_reg("rsp", regs.rsp);
-		print_reg("rbx", regs.rbx);
-		print_reg("rdx", regs.rdx);
-		print_reg("rcx", regs.rcx);
-		print_reg("rax", regs.rax);
-		cons->puts("\n");
-
-		print_reg("rip", regs.rip);
-		print_reg(" cs", regs.cs);
-		print_reg(" ef", regs.eflags);
-		print_reg("esp", regs.user_esp);
-		print_reg(" ss", regs.ss);
-		while(1) asm("hlt");
+		print_regs(regs);
+		_halt();
 	}
 
 	*reinterpret_cast<uint32_t *>(0xffffff80fee000b0) = 0;
+	return return_rsp;
 }
 
-
-void
-gdt_dump(const table_ptr &table)
+uintptr_t
+syscall_handler(uintptr_t return_rsp, cpu_state regs)
 {
-	log::info(logs::boot, "Loaded GDT at: %lx size: %d bytes", table.base, table.limit+1);
-
-	int count = (table.limit + 1) / sizeof(gdt_descriptor);
-	const gdt_descriptor *gdt =
-		reinterpret_cast<const gdt_descriptor *>(table.base);
-
-	for (int i = 0; i < count; ++i) {
-		uint32_t base =
-			(gdt[i].base_high << 24) |
-			(gdt[i].base_mid << 16) |
-			gdt[i].base_low;
-
-		uint32_t limit =
-			static_cast<uint32_t>(gdt[i].granularity & 0x0f) << 16 |
-			gdt[i].limit_low;
-
-		if (gdt[i].flags & 0x80) {
-			log::debug(logs::boot,
-					"   Entry %3d: Base %x  limit %x  privs %d  flags %s%s%s%s%s%s",
-					i, base, limit, ((gdt[i].flags >> 5) & 0x03),
-
-					(gdt[i].flags & 0x80) ? "P " : "  ",
-					(gdt[i].flags & 0x08) ? "ex " : "   ",
-					(gdt[i].flags & 0x04) ? "dc " : "   ",
-					(gdt[i].flags & 0x02) ? "rw " : "   ",
-
-					(gdt[i].granularity & 0x80) ? "kb " : "b  ",
-					(gdt[i].granularity & 0x60) == 0x60 ? "64" :
-						(gdt[i].granularity & 0x60) == 0x40 ? "32" : "16"
-					);
-		}
-	}
-}
-
-void
-idt_dump(const table_ptr &table)
-{
-	log::info(logs::boot, "Loaded IDT at: %lx size: %d bytes", table.base, table.limit+1);
-
-	int count = (table.limit + 1) / sizeof(idt_descriptor);
-	const idt_descriptor *idt =
-		reinterpret_cast<const idt_descriptor *>(table.base);
-
-	for (int i = 0; i < count; ++i) {
-		uint64_t base =
-			(static_cast<uint64_t>(idt[i].base_high) << 32) |
-			(static_cast<uint64_t>(idt[i].base_mid)  << 16) |
-			idt[i].base_low;
-
-		char const *type;
-		switch (idt[i].flags & 0xf) {
-			case 0x5: type = " 32tsk "; break;
-			case 0x6: type = " 16int "; break;
-			case 0x7: type = " 16trp "; break;
-			case 0xe: type = " 32int "; break;
-			case 0xf: type = " 32trp "; break;
-			default:  type = " ????? "; break;
-		}
-
-		if (idt[i].flags & 0x80) {
-			log::debug(logs::boot,
-					"   Entry %3d: Base:%lx Sel(rpl %d, ti %d, %3d) IST:%d %s DPL:%d", i, base,
-					(idt[i].selector & 0x3),
-					((idt[i].selector & 0x4) >> 2),
-					(idt[i].selector >> 3),
-					idt[i].ist,
-					type,
-					((idt[i].flags >> 5) & 0x3));
-		}
-	}
+	return syscall_dispatch(return_rsp, regs);
 }

src/kernel/interrupts.h

@@ -1,6 +1,7 @@
 #pragma once
 /// \file interrupts.h
 /// Free functions and definitions related to interrupt service vectors
+#include <stdint.h>
 
 
 /// Enum of all defined ISR/IRQ vectors
@@ -8,20 +9,28 @@ enum class isr : uint8_t
 {
 #define ISR(i, name)     name = i,
 #define EISR(i, name)    name = i,
+#define UISR(i, name)    name = i,
 #define IRQ(i, q, name)  name = i,
 #include "interrupt_isrs.inc"
 #undef IRQ
+#undef UISR
 #undef EISR
 #undef ISR
 
 	_zero = 0
 };
 
+/// Helper operator to add an offset to an isr vector
 isr operator+(const isr &lhs, int rhs);
 
 extern "C" {
+	/// Set the CPU interrupt enable flag (sti)
 	void interrupts_enable();
+
+	/// Set the CPU interrupt disable flag (cli)
 	void interrupts_disable();
 }
 
+/// Fill the IDT with our ISRs, and disable the legacy
+/// PIC interrupts.
 void interrupts_init();

src/kernel/interrupts.s

@@ -1,72 +1,13 @@
-extern g_idtr
-extern g_gdtr
-
-global idt_write
-idt_write:
-	lidt [rel g_idtr]
-	ret
-
-global idt_load
-idt_load:
-	sidt [rel g_idtr]
-	ret
-
-global gdt_write
-gdt_write:
-	lgdt [rel g_gdtr]
-	ret
-
-global gdt_load
-gdt_load:
-	sgdt [rel g_gdtr]
-	ret
-
-%macro push_all_and_segments 0
-	push rax
-	push rcx
-	push rdx
-	push rbx
-	push rsp
-	push rbp
-	push rsi
-	push rdi
-
-	mov ax, ds
-	push rax
-%endmacro
-
-%macro pop_all_and_segments 0
-	pop rax
-	mov ds, ax
-	mov es, ax
-	mov fs, ax
-	mov gs, ax
-
-	pop rdi
-	pop rsi
-	pop rbp
-	pop rsp
-	pop rbx
-	pop rdx
-	pop rcx
-	pop rax
-%endmacro
-
-%macro load_kernel_segments 0
-	mov ax, 0x10	; load the kernel data segment
-	mov ds, ax
-	mov es, ax
-	mov fs, ax
-	mov gs, ax
-%endmacro
+%include "push_all.inc"
 
 extern isr_handler
 global isr_handler_prelude
 isr_handler_prelude:
 	push_all_and_segments
-	load_kernel_segments
 
+	mov rdi, rsp
 	call isr_handler
+	mov rsp, rax
 
 	pop_all_and_segments
 
@@ -78,9 +19,10 @@ extern irq_handler
 global irq_handler_prelude
 irq_handler_prelude:
 	push_all_and_segments
-	load_kernel_segments
 
+	mov rdi, rsp
 	call irq_handler
+	mov rsp, rax
 
 	pop_all_and_segments
 
@@ -115,6 +57,7 @@ irq_handler_prelude:
 %endmacro
 
 %define EISR(i, name)     EMIT_EISR name, i
+%define UISR(i, name)     EMIT_ISR name, i
 %define  ISR(i, name)     EMIT_ISR name, i
 %define  IRQ(i, q, name)  EMIT_IRQ name, i
 

src/kernel/io.cpp

@@ -14,24 +14,9 @@ outb(uint16_t port, uint8_t val)
 	__asm__ __volatile__ ( "outb %0, %1" :: "a"(val), "Nd"(port) );
 }
 
-uint64_t
-rdmsr(uint64_t addr)
-{
-	uint32_t low, high;
-	__asm__ __volatile__ ("rdmsr" : "=a"(low), "=d"(high) : "c"(addr));
-	return (static_cast<uint64_t>(high) << 32) | low;
-}
-
 void
-wrmsr(uint64_t addr, uint64_t value)
+io_wait(unsigned times)
 {
-	uint32_t low = value & 0xffffffff;
-	uint32_t high = value >> 32;
-	__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
-}
-
-void
-io_wait()
-{
-	outb(0x80, 0);
+	for (unsigned i = 0; i < times; ++i)
+		outb(0x80, 0);
 }

src/kernel/io.h

@@ -14,18 +14,9 @@ uint8_t inb(uint16_t port);
 /// \arg val   The byte to write
 void outb(uint16_t port, uint8_t val);
 
-/// Read the value of a MSR
-/// \arg addr   The MSR address
-/// \returns    The current value of the MSR
-uint64_t rdmsr(uint64_t addr);
-
-/// Write to a MSR
-/// \arg addr   The MSR address
-/// \arg value  The value to write
-void wrmsr(uint64_t addr, uint64_t value);
-
 /// Pause briefly by doing IO to port 0x80
-void io_wait();
+/// \arg times  Number of times to delay by writing
+void io_wait(unsigned times = 1);
 
 }
 

src/kernel/loader.s

@@ -0,0 +1,25 @@
+%include "push_all.inc"
+
+extern load_process
+
+global ramdisk_process_loader
+ramdisk_process_loader:
+
+	; create_process already pushed a cpu_state onto the stack for us, this
+	; acts both as the cpu_state parameter to load_process, and the saved
+	; state for the following iretq
+	;
+	; Additional parameters:
+	;  rax - the address of the program image
+	;  rbx - the size of the program image
+	;  rcx - the address of this process' process structure
+	mov rdi, rax
+	mov rsi, rbx
+	mov rdx, rcx
+	call load_process
+
+	pop_all_and_segments
+	add rsp, 16		; because the ISRs add err/num
+	iretq
+
+

src/kernel/log.cpp

@@ -10,11 +10,13 @@ static const uint8_t level_colors[] = {0x07, 0x0f, 0x0b, 0x09};
 
 static const char *levels[] = {"debug", " info", " warn", "error", "fatal"};
 static const char *areas[] = {
-	"boot",
-	"mem ",
-	"apic",
-	"dev ",
-	"driv",
+	"boot  ",
+	"memory",
+	"apic  ",
+	"device",
+	"driver",
+	"file  ",
+	"task  ",
 
 	nullptr
 };

src/kernel/log.h

@@ -12,6 +12,8 @@ enum class logs
 	apic,
 	device,
 	driver,
+	fs,
+	task,
 
 	max
 };

src/kernel/main.cpp

@@ -1,75 +1,69 @@
 #include <stddef.h>
 #include <stdint.h>
 
+#include "initrd/initrd.h"
 #include "kutil/assert.h"
-#include "kutil/memory.h"
-
-#include "ahci/driver.h"
-#include "ahci/port.h"
-
+#include "apic.h"
+#include "block_device.h"
 #include "console.h"
 #include "cpu.h"
 #include "device_manager.h"
-#include "font.h"
+#include "gdt.h"
 #include "interrupts.h"
 #include "io.h"
 #include "kernel_data.h"
 #include "log.h"
-#include "memory.h"
 #include "page_manager.h"
+#include "scheduler.h"
 #include "screen.h"
 #include "serial.h"
+#include "syscall.h"
 
 extern "C" {
-	void do_the_set_registers(popcorn_data *header);
 	void kernel_main(popcorn_data *header);
-
 	void *__bss_start, *__bss_end;
 }
 
-extern ahci_driver ahcid;
-
 extern void __kernel_assert(const char *, unsigned, const char *);
 
 void
-init_console(const popcorn_data *header)
+init_console()
 {
 	serial_port *com1 = new (&g_com1) serial_port(COM1);
 	console *cons = new (&g_console) console(com1);
 
-	if (header->frame_buffer) {
-		screen *s = new screen(
-				header->frame_buffer,
-				header->hres,
-				header->vres,
-				header->rmask,
-				header->gmask,
-				header->bmask);
-		font *f = new font(header->font);
-		cons->init_screen(s, f);
-	}
-
 	cons->set_color(0x21, 0x00);
 	cons->puts("Popcorn OS ");
 	cons->set_color(0x08, 0x00);
 	cons->puts(GIT_VERSION " booting...\n");
 
 	log::init(cons);
-	log::enable(logs::apic, log::level::info);
-	log::enable(logs::device, log::level::debug);
+	log::enable(logs::apic, log::level::debug);
+	log::enable(logs::device, log::level::info);
 	log::enable(logs::driver, log::level::debug);
-	log::enable(logs::memory, log::level::debug);
+	log::enable(logs::memory, log::level::info);
+	log::enable(logs::fs, log::level::debug);
+	log::enable(logs::task, log::level::debug);
 }
 
-void do_error_3() { volatile int x = 1; volatile int y = 0; volatile int z = x / y; }
-void do_error_2() { do_error_3(); }
-void do_error_1() { do_error_2(); }
-
 void
 kernel_main(popcorn_data *header)
 {
+#ifdef DEBUG
+	// Run `waf configure --debug` to enable compiling with DEBUG turned on.
+	// Then attach to QEMU's gdb server and `set waiting = false` to start
+	// the kernel. This compensates for GDB's poor handling of QEMU going
+	// through the x86 PC startup and switching to 64 bit mode when you
+	// attach to qemu with the -S option.
+	bool waiting = true;
+	while (waiting);
+#endif
+
 	kutil::assert_set_callback(__kernel_assert);
 
+	gdt_init();
+	interrupts_init();
+
 	page_manager *pager = new (&g_page_manager) page_manager;
 
 	memory_initialize(
@@ -81,44 +75,81 @@ kernel_main(popcorn_data *header)
 			&header->frame_buffer,
 			header->frame_buffer_length);
 
-	init_console(header);
-	// pager->dump_blocks();
+	init_console();
 
-	interrupts_init();
+	log::debug(logs::boot, " Popcorn header is at: %016lx", header);
+	log::debug(logs::boot, "    Framebuffer is at: %016lx", header->frame_buffer);
+	log::debug(logs::boot, "    Kernel data is at: %016lx", header->data);
+	log::debug(logs::boot, "     Memory map is at: %016lx", header->memory_map);
+	log::debug(logs::boot, "ACPI root table is at: %016lx", header->acpi_table);
+	log::debug(logs::boot, "Runtime service is at: %016lx", header->runtime);
+
+	initrd::disk ird(header->initrd);
+	log::info(logs::boot, "initrd loaded with %d files.", ird.files().count());
+	for (auto &f : ird.files())
+		log::info(logs::boot, "  %s%s (%d bytes).", f.executable() ? "*" : "", f.name(), f.size());
+
+	/*
+	pager->dump_pml4(nullptr, 0);
+	pager->dump_blocks(true);
+	*/
 
 	device_manager *devices =
 		new (&device_manager::get()) device_manager(header->acpi_table);
 
 	interrupts_enable();
 
+	/*
 	cpu_id cpu;
 	log::info(logs::boot, "CPU Vendor: %s", cpu.vendor_id());
 	log::info(logs::boot, "CPU Family %x Model %x Stepping %x",
 			cpu.family(), cpu.model(), cpu.stepping());
 
+	auto r = cpu.get(0x15);
+	log::info(logs::boot, "CPU Crystal: %dHz", r.ecx);
+
+	uintptr_t cr4 = 0;
+	__asm__ __volatile__ ( "mov %%cr4, %0" : "=r" (cr4) );
+	log::info(logs::boot, "cr4: %016x", cr4);
+	*/
+
 	devices->init_drivers();
 
-	ahci::port *disk = ahcid.find_disk();
+	/*
+	block_device *disk = devices->get_block_device(0);
 	if (disk) {
-		uint8_t buf[512];
-		kutil::memset(buf, 0, 512);
+		for (int i=0; i<1; ++i) {
+			uint8_t buf[512];
+			kutil::memset(buf, 0, 512);
 
-		disk->read(1, sizeof(buf), buf);
-		while (buf[0] == 0) io_wait();
+			kassert(disk->read(0x200, sizeof(buf), buf),
+					"Disk read returned 0");
 
-		console *cons = console::get();
-		uint8_t *p = &buf[0];
-		for (int i = 0; i < 8; ++i) {
-			for (int j = 0; j < 16; ++j) {
-				cons->printf(" %02x", *p++);
+			console *cons = console::get();
+			uint8_t *p = &buf[0];
+			for (int i = 0; i < 8; ++i) {
+				for (int j = 0; j < 16; ++j) {
+					cons->printf(" %02x", *p++);
+				}
+				cons->putc('\n');
 			}
-			cons->putc('\n');
 		}
+	} else {
+		log::warn(logs::boot, "No block devices present.");
 	}
+	*/
 
-	// do_error_1();
-	// __asm__ __volatile__("int $15");
+	devices->get_lapic()->calibrate_timer();
 
-	g_console.puts("boogity!");
-	do_the_set_registers(header);
+	syscall_enable();
+	scheduler *sched = new (&scheduler::get()) scheduler(devices->get_lapic());
+
+	/*
+	for (auto &f : ird.files()) {
+		if (f.executable())
+			sched->create_process(f.name(), f.data(), f.size());
+	}
+	*/
+
+	sched->start();
 }

src/kernel/memory_bootstrap.cpp

@@ -1,9 +1,32 @@
+#include <utility>
 #include "kutil/assert.h"
-#include "kutil/memory.h"
-#include "memory.h"
+#include "kutil/linked_list.h"
+#include "kutil/slab_allocator.h"
+#include "io.h"
 #include "page_manager.h"
 
 const unsigned efi_page_size = 0x1000;
+const unsigned ident_page_flags = 0xb;
+
+namespace {
+	// Page-by-page initial allocator for the initial page_block allocator
+	struct page_consumer
+	{
+		page_consumer(uintptr_t start) : current(start) {}
+
+		void * operator()(size_t size) {
+			kassert(size == page_manager::page_size, "page_consumer used with non-page size!");
+			void *retval = reinterpret_cast<void *>(current);
+			current += size;
+			return retval;
+		}
+
+		uintptr_t current;
+	};
+}
+
+using block_list = kutil::linked_list<page_block>;
+using block_allocator = kutil::slab_allocator<page_block, page_consumer &>;
 
 enum class efi_memory_type : uint32_t
 {
@@ -103,131 +126,80 @@ desc_incr(const efi_memory_descriptor *d, size_t desc_length)
 			reinterpret_cast<const uint8_t *>(d) + desc_length);
 }
 
-static unsigned
-count_table_pages_needed(page_block *used)
-{
-	page_table_indices last_idx{~0ull};
-	unsigned counts[] = {1, 0, 0, 0};
-
-	for (page_block *cur = used; cur; cur = cur->next) {
-		if (!cur->has_flag(page_block_flags::mapped))
-			continue;
-
-		page_table_indices start{cur->virtual_address};
-		page_table_indices end{cur->virtual_address + (cur->count * page_manager::page_size)};
-
-		counts[1] +=
-			((start[0] == last_idx[0]) ? 0 : 1) +
-			(end[0] - start[0]);
-
-		counts[2] +=
-			((start[0] == last_idx[0] &&
-			 start[1] == last_idx[1]) ? 0 : 1) +
-			(end[1] - start[1]);
-
-		counts[3] +=
-			((start[0] == last_idx[0] &&
-			 start[1] == last_idx[1] &&
-			 start[2] == last_idx[2]) ? 0 : 1) +
-			(end[2] - start[2]);
-
-		last_idx = end;
-	}
-
-	return counts[0] + counts[1] + counts[2] + counts[3];
-
-}
-
-page_block *
-remove_block_for(page_block **list, uint64_t phys_start, uint64_t pages, page_block **cache)
+page_block_list::item_type *
+remove_block_for(page_block_list &list, uintptr_t phys_start, size_t pages, page_block_list &cache)
 {
 	// This is basically just the removal portion of page_manager::unmap_pages,
 	// but with physical addresses, and only ever removing a single block.
 
-	page_block *prev = nullptr;
-	page_block *cur = *list;
-	while (cur && !cur->contains_physical(phys_start)) {
-		prev = cur;
-		cur = cur->next;
-	}
+	for (auto *item : list) {
+		if (!item->contains_physical(phys_start))
+			continue;
 
-	kassert(cur, "Couldn't find block to remove");
+		uint64_t size = page_manager::page_size * pages;
+		uint64_t end = phys_start + size;
+		uint64_t leading = phys_start - item->physical_address;
+		uint64_t trailing = item->physical_end() - end;
 
-	uint64_t size = page_manager::page_size * pages;
-	uint64_t end = phys_start + size;
-	uint64_t leading = phys_start - cur->physical_address;
-	uint64_t trailing = cur->physical_end() - end;
+		if (leading) {
+			uint64_t pages = leading / page_manager::page_size;
 
-	if (leading) {
-		uint64_t pages = leading / page_manager::page_size;
+			page_block_list::item_type *lead_block = cache.pop_front();
 
-		page_block *lead_block = *cache;
-		*cache = (*cache)->next;
+			lead_block->copy(item);
+			lead_block->count = pages;
 
-		lead_block->copy(cur);
-		lead_block->next = cur;
-		lead_block->count = pages;
+			item->count -= pages;
+			item->physical_address += leading;
 
-		cur->count -= pages;
-		cur->physical_address += leading;
+			if (item->virtual_address)
+				item->virtual_address += leading;
 
-		if (cur->virtual_address)
-			cur->virtual_address += leading;
-
-		if (prev) {
-			prev->next = lead_block;
-		} else {
-			prev = lead_block;
-			*list = prev;
+			list.insert_before(item, lead_block);
 		}
+
+		if (trailing) {
+			uint64_t pages = trailing / page_manager::page_size;
+
+			page_block_list::item_type *trail_block = cache.pop_front();
+
+			trail_block->copy(item);
+			trail_block->count = pages;
+			trail_block->physical_address += size;
+
+			item->count -= pages;
+
+			if (item->virtual_address)
+				trail_block->virtual_address += size;
+
+			list.insert_before(item, trail_block);
+		}
+
+		list.remove(item);
+		return item;
 	}
 
-	if (trailing) {
-		uint64_t pages = trailing / page_manager::page_size;
-
-		page_block *trail_block = *cache;
-		*cache = (*cache)->next;
-
-		trail_block->copy(cur);
-		trail_block->next = cur->next;
-		trail_block->count = pages;
-		trail_block->physical_address += size;
-
-		if (cur->virtual_address)
-			trail_block->virtual_address += size;
-
-		cur->count -= pages;
-		cur->next = trail_block;
-	}
-
-	prev->next = cur->next;
-	cur->next = nullptr;
-	return cur;
+	kassert(false, "Couldn't find block to remove");
+	return nullptr;
 }
 
-uint64_t
+void
 gather_block_lists(
-		uint64_t scratch_virt,
+		block_allocator &allocator,
+		block_list &used,
+		block_list &free,
 		const void *memory_map,
 		size_t map_length,
-		size_t desc_length,
-		page_block **free_head,
-		page_block **used_head)
+		size_t desc_length)
 {
-	int i = 0;
-	page_block *free = nullptr;
-	page_block *used = nullptr;
-
-	page_block *block_list = reinterpret_cast<page_block *>(scratch_virt);
 	efi_memory_descriptor const *desc = reinterpret_cast<efi_memory_descriptor const *>(memory_map);
 	efi_memory_descriptor const *end = desc_incr(desc, map_length);
 
 	while (desc < end) {
-		page_block *block = &block_list[i++];
+		auto *block = allocator.pop();
 		block->physical_address = desc->physical_start;
 		block->virtual_address = desc->virtual_start;
 		block->count = desc->pages;
-		block->next = nullptr;
 
 		switch (desc->type) {
 		case efi_memory_type::loader_code:
@@ -263,31 +235,13 @@ gather_block_lists(
 			if (block->virtual_address || !block->physical_address)
 				block->flags |= page_block_flags::mapped;
 
-			used = page_block::insert(used, block);
+			used.push_back(block);
 		} else {
-			free = page_block::insert(free, block);
+			free.push_back(block);
 		}
 
 		desc = desc_incr(desc, desc_length);
 	}
-
-	*free_head = free;
-	*used_head = used;
-	return reinterpret_cast<uint64_t>(&block_list[i]);
-}
-
-page_block *
-fill_page_with_blocks(uint64_t start)
-{
-	uint64_t end = page_align(start);
-	uint64_t count = (end - start) / sizeof(page_block);
-	if (count == 0) return nullptr;
-
-	page_block *blocks = reinterpret_cast<page_block *>(start);
-	for (unsigned i = 0; i < count; ++i)
-		blocks[i].zero(&blocks[i+1]);
-	blocks[count - 1].next = nullptr;
-	return blocks;
 }
 
 void
@@ -306,7 +260,7 @@ copy_new_table(page_table *base, unsigned index, page_table *new_table)
 		for (int i = 0; i < 512; ++i) new_table->entries[i] = 0;
 	}
 
-	base->entries[index] = reinterpret_cast<uint64_t>(new_table) | 0xb;
+	base->entries[index] = reinterpret_cast<uint64_t>(new_table) | ident_page_flags;
 }
 
 static uint64_t
@@ -363,7 +317,7 @@ check_needs_page_ident(page_table *table, unsigned index, page_table **free_page
 
 	page_table *new_table = (*free_pages)++;
 	for (int i=0; i<512; ++i) new_table->entries[i] = 0;
-	table->entries[index] = reinterpret_cast<uint64_t>(new_table) | 0xb;
+	table->entries[index] = reinterpret_cast<uint64_t>(new_table) | ident_page_flags;
 	return 1;
 }
 
@@ -390,12 +344,24 @@ page_in_ident(
 					tables[1]->entries[idx[1]] & ~0xfffull);
 
 			for (; idx[2] < 512; idx[2] += 1, idx[3] = 0) {
+				if (idx[3] == 0 &&
+					count >= 512 &&
+					tables[2]->get(idx[2]) == nullptr) {
+					// Do a 2MiB page instead
+					tables[2]->entries[idx[2]] = phys_addr | 0x80 | ident_page_flags;
+
+					phys_addr += page_manager::page_size * 512;
+					count -= 512;
+					if (count == 0) return pages_consumed;
+					continue;
+				}
+
 				pages_consumed += check_needs_page_ident(tables[2], idx[2], &free_pages);
 				tables[3] = reinterpret_cast<page_table *>(
 						tables[2]->entries[idx[2]] & ~0xfffull);
 
 				for (; idx[3] < 512; idx[3] += 1) {
-					tables[3]->entries[idx[3]] = phys_addr | 0xb;
+					tables[3]->entries[idx[3]] = phys_addr | ident_page_flags;
 					phys_addr += page_manager::page_size;
 					if (--count == 0) return pages_consumed;
 				}
@@ -416,6 +382,14 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 	__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (cr3) );
 	page_table *tables = reinterpret_cast<page_table *>(cr3 & ~0xfffull);
 
+	// We'll need to make sure the options we want in CR4 are set
+	uint64_t cr4;
+	__asm__ __volatile__ ( "mov %%cr4, %0" : "=r" (cr4) );
+	cr4 |= 0x00080; // Enable global pages
+	cr4 |= 0x00200; // Enable FXSAVE/FXRSTOR
+	cr4 |= 0x20000; // Enable PCIDs
+	__asm__ __volatile__ ( "mov %0, %%cr4" :: "r" (cr4) );
+
 	// Now go through EFi's memory map and find a region of scratch space.
 	const unsigned want_pages = 32;
 	uint64_t free_region_start_phys =
@@ -423,12 +397,12 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 
 	// Offset-map this region into the higher half.
 	uint64_t free_region_start_virt =
-		free_region_start_phys + page_manager::high_offset;
+		free_region_start_phys + page_manager::page_offset;
 
 	uint64_t free_next = free_region_start_virt;
 
 	// We'll need to copy any existing tables (except the PML4 which the
-	// bootloader gave us) into our 4 reserved pages so we can edit them.
+	// bootloader gave us) into our reserved pages so we can edit them.
 	page_table_indices fr_idx{free_region_start_virt};
 
 	copy_new_table(&tables[0], fr_idx[0], &tables[1]);
@@ -438,34 +412,27 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 
 	// We now have pages starting at "free_next" to bootstrap ourselves. Start by
 	// taking inventory of free pages.
-	page_block *free_head = nullptr;
-	page_block *used_head = nullptr;
-	free_next = gather_block_lists(
-			free_next, memory_map, map_length, desc_length,
-			&free_head, &used_head);
+	page_consumer allocator(free_next);
+	block_allocator block_slab(page_manager::page_size, allocator);
+	block_list used;
+	block_list free;
 
-	// Unused page_block structs go here - finish out the current page with them
-	page_block *cache_head = fill_page_with_blocks(free_next);
-	free_next = page_align(free_next);
+	gather_block_lists(block_slab, used, free, memory_map, map_length, desc_length);
+	block_slab.allocate(); // Make sure we have extra
+
+	free_next = allocator.current;
 
 	// Now go back through these lists and consolidate
-	page_block *freed = page_block::consolidate(free_head);
-	cache_head = page_block::append(cache_head, freed);
-
-	freed = page_block::consolidate(used_head);
-	cache_head = page_block::append(cache_head, freed);
-
+	block_slab.append(page_block::consolidate(free));
+	block_slab.append(page_block::consolidate(used));
 
 	// Pull out the block that represents the bootstrap pages we've used
-	uint64_t used = free_next - free_region_start_virt;
-	uint64_t used_pages = used / page_manager::page_size;
+	uint64_t used_bytes = free_next - free_region_start_virt;
+	uint64_t used_pages = used_bytes / page_manager::page_size;
 	uint64_t remaining_pages = want_pages - used_pages;
 
-	page_block *removed = remove_block_for(
-			&free_head,
-			free_region_start_phys,
-			used_pages,
-			&cache_head);
+	auto *removed = remove_block_for(free, free_region_start_phys,
+			used_pages, block_slab);
 
 	kassert(removed, "remove_block_for didn't find the bootstrap region.");
 	kassert(removed->physical_address == free_region_start_phys,
@@ -474,16 +441,13 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 	// Add it to the used list
 	removed->virtual_address = free_region_start_virt;
 	removed->flags = page_block_flags::used | page_block_flags::mapped;
-	used_head = page_block::insert(used_head, removed);
+	used.sorted_insert(removed);
 
 	// Pull out the block that represents the rest
-	uint64_t free_next_phys = free_region_start_phys + used;
+	uint64_t free_next_phys = free_region_start_phys + used_bytes;
 
-	removed = remove_block_for(
-			&free_head,
-			free_next_phys,
-			remaining_pages,
-			&cache_head);
+	removed = remove_block_for(free, free_next_phys,
+			remaining_pages, block_slab);
 
 	kassert(removed, "remove_block_for didn't find the page table region.");
 	kassert(removed->physical_address == free_next_phys,
@@ -495,7 +459,7 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 	// Record that we're about to remap it into the page table address space
 	removed->virtual_address = pt_start_virt;
 	removed->flags = page_block_flags::used | page_block_flags::mapped;
-	used_head = page_block::insert(used_head, removed);
+	used.sorted_insert(removed);
 
 	page_manager *pm = &g_page_manager;
 
@@ -509,6 +473,10 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 
 	page_in_ident(&tables[0], pt_start_phys, pt_start_virt, remaining_pages, tables + 4);
 
+	// Make sure the page table is finished updating before we write to memory
+	__sync_synchronize();
+	io_wait();
+
 	// Finally, build an acutal set of kernel page tables that just contains
 	// what the kernel actually has mapped, but making everything writable
 	// (especially the page tables themselves)
@@ -518,9 +486,9 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 	// Give the rest to the page_manager's cache for use in page_in
 	pm->free_table_pages(pml4 + 1, remaining_pages - 1);
 
-	for (page_block *cur = used_head; cur; cur = cur->next) {
-		if (!cur->has_flag(page_block_flags::mapped)) continue;
-		pm->page_in(pml4, cur->physical_address, cur->virtual_address, cur->count);
+	for (auto *block : used) {
+		if (!block->has_flag(page_block_flags::mapped)) continue;
+		pm->page_in(pml4, block->physical_address, block->virtual_address, block->count);
 	}
 
 	// Put our new PML4 into CR3 to start using it
@@ -528,5 +496,8 @@ memory_initialize(const void *memory_map, size_t map_length, size_t desc_length)
 
 	// We now have all used memory mapped ourselves. Let the page_manager take
 	// over from here.
-	g_page_manager.init(free_head, used_head, cache_head);
+	g_page_manager.init(
+			std::move(free),
+			std::move(used),
+			std::move(block_slab));
 }

src/kernel/msr.cpp

@@ -0,0 +1,18 @@
+#include "msr.h"
+
+uint64_t
+rdmsr(msr addr)
+{
+	uint32_t low, high;
+	__asm__ __volatile__ ("rdmsr" : "=a"(low), "=d"(high) : "c"(addr));
+	return (static_cast<uint64_t>(high) << 32) | low;
+}
+
+void
+wrmsr(msr addr, uint64_t value)
+{
+	uint32_t low = value & 0xffffffff;
+	uint32_t high = value >> 32;
+	__asm__ __volatile__ ("wrmsr" :: "c"(addr), "a"(low), "d"(high));
+}
+

src/kernel/msr.h

@@ -0,0 +1,27 @@
+#pragma once
+/// \file msr.h
+/// Routines and definitions for dealing with Model-Specific Registers
+
+#include <stdint.h>
+
+enum class msr : uint32_t
+{
+	ia32_efer			= 0xc0000080,
+	ia32_star			= 0xc0000081,
+	ia32_lstar			= 0xc0000082,
+	ia32_fmask			= 0xc0000084,
+
+	ia32_gs_base		= 0xc0000101,
+	ia32_kernel_gs_base = 0xc0000102
+};
+
+/// Read the value of a MSR
+/// \arg addr   The MSR address
+/// \returns    The current value of the MSR
+uint64_t rdmsr(msr addr);
+
+/// Write to a MSR
+/// \arg addr   The MSR address
+/// \arg value  The value to write
+void wrmsr(msr addr, uint64_t value);
+

src/kernel/page_manager.cpp

@@ -2,21 +2,23 @@
 
 #include "kutil/assert.h"
 #include "kutil/memory_manager.h"
+#include "console.h"
 #include "log.h"
 #include "page_manager.h"
 
 page_manager g_page_manager;
+kutil::memory_manager g_kernel_memory_manager;
 
 
-static addr_t
+static uintptr_t
 pt_to_phys(page_table *pt)
 {
-	return reinterpret_cast<addr_t>(pt) - page_manager::page_offset;
+	return reinterpret_cast<uintptr_t>(pt) - page_manager::page_offset;
 }
 
 
 static page_table *
-pt_from_phys(addr_t p)
+pt_from_phys(uintptr_t p)
 {
 	return reinterpret_cast<page_table *>((p + page_manager::page_offset) & ~0xfffull);
 }
@@ -36,111 +38,63 @@ void mm_grow_callback(void *next, size_t length)
 
 	size_t pages = length / page_manager::page_size;
 	log::info(logs::memory, "Heap manager growing heap by %d pages.", pages);
-	g_page_manager.map_pages(reinterpret_cast<addr_t>(next), pages);
+	g_page_manager.map_pages(reinterpret_cast<uintptr_t>(next), pages);
 }
 
 
-size_t
-page_block::length(page_block *list)
-{
-	size_t i = 0;
-	for (page_block *b = list; b; b = b->next) ++i;
-	return i;
-}
-
-page_block *
-page_block::append(page_block *list, page_block *extra)
-{
-	if (list == nullptr) return extra;
-	else if (extra == nullptr) return list;
-
-	page_block *cur = list;
-	while (cur->next)
-		cur = cur->next;
-	cur->next = extra;
-	return list;
-}
-
-page_block *
-page_block::insert(page_block *list, page_block *block)
-{
-	if (list == nullptr) return block;
-	else if (block == nullptr) return list;
-
-	page_block *cur = list;
-	page_block *prev = nullptr;
-	while (cur && page_block::compare(block, cur) > 0) {
-		prev = cur;
-		cur = cur->next;
-	}
-
-	block->next = cur;
-	if (prev) {
-		prev->next = block;
-		return list;
-	}
-	return block;
-}
-
 int
-page_block::compare(const page_block *lhs, const page_block *rhs)
+page_block::compare(const page_block *rhs) const
 {
-	if (lhs->virtual_address < rhs->virtual_address)
+	if (virtual_address < rhs->virtual_address)
 		return -1;
-	else if (lhs->virtual_address > rhs->virtual_address)
+	else if (virtual_address > rhs->virtual_address)
 		return 1;
 
-	if (lhs->physical_address < rhs->physical_address)
+	if (physical_address < rhs->physical_address)
 		return -1;
-	else if (lhs->physical_address > rhs->physical_address)
+	else if (physical_address > rhs->physical_address)
 		return 1;
 
 	return 0;
 }
 
-page_block *
-page_block::consolidate(page_block *list)
+page_block_list
+page_block::consolidate(page_block_list &list)
 {
-	page_block *freed = nullptr;
-	page_block *cur = list;
+	page_block_list freed;
 
-	while (cur) {
-		page_block *next = cur->next;
+	for (auto *cur : list) {
+		auto *next = cur->next();
 
-		if (next &&
+		while (next &&
 			cur->flags == next->flags &&
 			cur->physical_end() == next->physical_address &&
 			(!cur->has_flag(page_block_flags::mapped) ||
 			cur->virtual_end() == next->virtual_address)) {
 
 			cur->count += next->count;
-			cur->next = next->next;
-
-			next->zero(freed);
-			freed = next;
-			continue;
+			list.remove(next);
+			freed.push_back(next);
 		}
-
-		cur = cur->next;
 	}
 
 	return freed;
 }
 
 void
-page_block::dump(page_block *list, const char *name, bool show_unmapped)
+page_block::dump(const page_block_list &list, const char *name, bool show_unmapped)
 {
 	log::info(logs::memory, "Block list %s:", name);
 
 	int count = 0;
-	for (page_block *cur = list; cur; cur = cur->next) {
+	for (auto *cur : list) {
 		count += 1;
 		if (!(show_unmapped || cur->has_flag(page_block_flags::mapped)))
 			continue;
 
 		if (cur->virtual_address) {
 			page_table_indices start{cur->virtual_address};
-			log::info(logs::memory, "  %lx %x [%6d] %lx (%d,%d,%d,%d)",
+			log::info(logs::memory, "  %016lx %08x [%6d] %016lx (%d,%d,%d,%d)",
 					cur->physical_address,
 					cur->flags,
 					cur->count,
@@ -148,7 +102,7 @@ page_block::dump(page_block *list, const char *name, bool show_unmapped)
 					start[0], start[1], start[2], start[3]);
 		} else {
 			page_table_indices start{cur->virtual_address};
-			log::info(logs::memory, "  %lx %x [%6d]",
+			log::info(logs::memory, "  %016lx %08x [%6d]",
 					cur->physical_address,
 					cur->flags,
 					cur->count);
@@ -159,13 +113,12 @@ page_block::dump(page_block *list, const char *name, bool show_unmapped)
 }
 
 void
-page_block::zero(page_block *set_next)
+page_block::zero()
 {
 	physical_address = 0;
 	virtual_address = 0;
 	count = 0;
 	flags = page_block_flags::free;
-	next = set_next;
 }
 
 void
@@ -175,14 +128,11 @@ page_block::copy(page_block *other)
 	virtual_address = other->virtual_address;
 	count = other->count;
 	flags = other->flags;
-	next = other->next;
 }
 
 
 page_manager::page_manager() :
-	m_free(nullptr),
-	m_used(nullptr),
-	m_block_cache(nullptr),
+	m_block_slab(page_size),
 	m_page_cache(nullptr)
 {
 	kassert(this == &g_page_manager, "Attempt to create another page_manager.");
@@ -190,45 +140,70 @@ page_manager::page_manager() :
 
 void
 page_manager::init(
-	page_block *free,
-	page_block *used,
-	page_block *block_cache)
+	page_block_list free,
+	page_block_list used,
+	page_block_list cache)
 {
-	m_free = free;
-	m_used = used;
-	m_block_cache = block_cache;
-
-	// For now we're ignoring that we've got the scratch pages
-	// allocated, full of page_block structs. Eventually hand
-	// control of that to a slab allocator.
+	m_free.append(free);
+	m_used.append(used);
+	m_block_slab.append(cache);
 
 	consolidate_blocks();
 
 	// Initialize the kernel memory manager
-	addr_t end = 0;
-	for (page_block *b = m_used; b; b = b->next) {
-		if (b->virtual_address < page_offset) {
-			end = b->virtual_end();
+	uintptr_t end = 0;
+	for (auto *block : m_used) {
+		if (block->virtual_address &&
+			block->virtual_address < page_offset) {
+			end = block->virtual_end();
 		} else {
 			break;
 		}
 	}
 
-	extern kutil::memory_manager g_kernel_memory_manager;
 	new (&g_kernel_memory_manager) kutil::memory_manager(
 			reinterpret_cast<void *>(end),
 			mm_grow_callback);
+	kutil::setup::set_heap(&g_kernel_memory_manager);
+
+	m_kernel_pml4 = get_pml4();
+}
+
+page_table *
+page_manager::create_process_map()
+{
+	page_table *table = get_table_page();
+
+	kutil::memset(table, 0, page_size);
+	table->entries[510] = m_kernel_pml4->entries[510];
+	table->entries[511] = m_kernel_pml4->entries[511];
+
+	// Create the initial user stack
+	map_pages(
+		initial_stack - (initial_stack_pages * page_size),
+		initial_stack_pages,
+		true, // This is the ring3 stack, user = true
+		table);
+
+	return table;
+}
+
+void
+page_manager::delete_process_map(page_table *table)
+{
+	// TODO: recurse table entries and mark them free
+	unmap_pages(table, 1);
 }
 
 void
 page_manager::map_offset_pointer(void **pointer, size_t length)
 {
-	addr_t *p = reinterpret_cast<addr_t *>(pointer);
-	addr_t v = *p + page_offset;
-	addr_t c = ((length - 1) / page_size) + 1;
+	uintptr_t *p = reinterpret_cast<uintptr_t *>(pointer);
+	uintptr_t v = *p + page_offset;
+	uintptr_t c = ((length - 1) / page_size) + 1;
 
 	// TODO: cleanly search/split this as a block out of used/free if possible
-	page_block *block = get_block();
+	auto *block = m_block_slab.pop();
 
 	// TODO: page-align
 	block->physical_address = *p;
@@ -239,7 +214,7 @@ page_manager::map_offset_pointer(void **pointer, size_t length)
 		page_block_flags::mapped |
 		page_block_flags::mmio;
 
-	m_used = page_block::insert(m_used, block);
+	m_used.sorted_insert(block);
 
 	page_table *pml4 = get_pml4();
 	page_in(pml4, *p, v, c);
@@ -247,61 +222,43 @@ page_manager::map_offset_pointer(void **pointer, size_t length)
 }
 
 void
-page_manager::dump_blocks()
+page_manager::dump_blocks(bool used_only)
 {
 	page_block::dump(m_used, "used", true);
-	page_block::dump(m_free, "free", true);
-}
-
-page_block *
-page_manager::get_block()
-{
-	page_block *block = m_block_cache;
-	if (block) {
-		m_block_cache = block->next;
-		block->next = 0;
-		return block;
-	} else {
-		kassert(0, "NYI: page_manager::get_block() needed to allocate.");
-		return nullptr;
-	}
+	if (!used_only)
+		page_block::dump(m_free, "free", true);
 }
 
 void
-page_manager::free_blocks(page_block *block)
+page_manager::dump_pml4(page_table *pml4, int max_index)
 {
-	if (!block) return;
-
-	page_block *cur = block;
-	while (cur) {
-		page_block *next = cur->next;
-		cur->zero(cur->next ? cur->next : m_block_cache);
-		cur = next;
-	}
-
-	m_block_cache = block;
+	if (pml4 == nullptr)
+		pml4 = get_pml4();
+	pml4->dump(4, max_index);
 }
 
 page_table *
 page_manager::get_table_page()
 {
 	if (!m_page_cache) {
-		addr_t phys = 0;
+		uintptr_t phys = 0;
 		size_t n = pop_pages(32, &phys);
-		addr_t virt = phys + page_offset;
+		uintptr_t virt = phys + page_offset;
+
+		auto *block = m_block_slab.pop();
 
-		page_block *block = get_block();
 		block->physical_address = phys;
 		block->virtual_address = virt;
 		block->count = n;
-		page_block::insert(m_used, block);
+
+		m_used.sorted_insert(block);
 
 		page_in(get_pml4(), phys, virt, n);
 
 		m_page_cache = reinterpret_cast<free_page_header *>(virt);
 
 		// The last one needs to be null, so do n-1
-		addr_t end = virt + (n-1) * page_size;
+		uintptr_t end = virt + (n-1) * page_size;
 		while (virt < end) {
 			reinterpret_cast<free_page_header *>(virt)->next =
 				reinterpret_cast<free_page_header *>(virt + page_size);
@@ -309,7 +266,7 @@ page_manager::get_table_page()
 		}
 		reinterpret_cast<free_page_header *>(virt)->next = nullptr;
 
-		log::info(logs::memory, "Mappd %d new page table pages at %lx", n, phys);
+		log::debug(logs::memory, "Mappd %d new page table pages at %lx", n, phys);
 	}
 
 	free_page_header *page = m_page_cache;
@@ -320,9 +277,9 @@ page_manager::get_table_page()
 void
 page_manager::free_table_pages(void *pages, size_t count)
 {
-	addr_t start = reinterpret_cast<addr_t>(pages);
+	uintptr_t start = reinterpret_cast<uintptr_t>(pages);
 	for (size_t i = 0; i < count; ++i) {
-		addr_t addr = start + (i * page_size);
+		uintptr_t addr = start + (i * page_size);
 		free_page_header *header = reinterpret_cast<free_page_header *>(addr);
 		header->count = 1;
 		header->next = m_page_cache;
@@ -333,32 +290,37 @@ page_manager::free_table_pages(void *pages, size_t count)
 void
 page_manager::consolidate_blocks()
 {
-	m_block_cache = page_block::append(m_block_cache, page_block::consolidate(m_free));
-	m_block_cache = page_block::append(m_block_cache, page_block::consolidate(m_used));
+	m_block_slab.append(page_block::consolidate(m_free));
+	m_block_slab.append(page_block::consolidate(m_used));
 }
 
 void *
-page_manager::map_pages(addr_t address, size_t count)
+page_manager::map_pages(uintptr_t address, size_t count, bool user, page_table *pml4)
 {
 	void *ret = reinterpret_cast<void *>(address);
-	page_table *pml4 = get_pml4();
+	if (!pml4) pml4 = get_pml4();
 
 	while (count) {
-		kassert(m_free, "page_manager::map_pages ran out of free pages!");
+		kassert(!m_free.empty(), "page_manager::map_pages ran out of free pages!");
 
-		addr_t phys = 0;
+		uintptr_t phys = 0;
 		size_t n = pop_pages(count, &phys);
 
-		page_block *block = get_block();
+		auto *block = m_block_slab.pop();
+
 		block->physical_address = phys;
 		block->virtual_address = address;
 		block->count = n;
 		block->flags =
 				page_block_flags::used |
 				page_block_flags::mapped;
-		page_block::insert(m_used, block);
 
-		page_in(pml4, phys, address, n);
+		m_used.sorted_insert(block);
+
+		log::debug(logs::memory, "Paging in %d pages at p:%016lx to v:%016lx into %016lx table",
+				n, phys, address, pml4);
+
+		page_in(pml4, phys, address, n, user);
 
 		address += n * page_size;
 		count -= n;
@@ -371,40 +333,31 @@ void *
 page_manager::map_offset_pages(size_t count)
 {
 	page_table *pml4 = get_pml4();
-	page_block *free = m_free;
-	page_block *prev = nullptr;
 
-	log::debug(logs::memory, "Got request to offset map %d pages", count);
+	for (auto *free : m_free) {
+		if (free->count < count) continue;
 
-	while (free) {
-		if (free->count < count) {
-			prev = free;
-			free = free->next;
-			continue;
-		}
+		auto *used = m_block_slab.pop();
 
-		page_block *used = get_block();
 		used->count = count;
 		used->physical_address = free->physical_address;
 		used->virtual_address = used->physical_address + page_offset;
 		used->flags =
 			page_block_flags::used |
 			page_block_flags::mapped;
-		page_block::insert(m_used, used);
+
+		m_used.sorted_insert(used);
 
 		free->physical_address += count * page_size;
 		free->count -= count;
 
 		if (free->count == 0) {
-			if (prev)
-				prev->next = free->next;
-			else
-				m_free = free->next;
-
-			free->zero(m_block_cache);
-			m_block_cache = free;
+			m_free.remove(free);
+			free->zero();
+			m_block_slab.push(free);
 		}
 
+		log::debug(logs::memory, "Got request for offset map %016lx [%d]", used->virtual_address, count);
 		page_in(pml4, used->physical_address, used->virtual_address, count);
 		return reinterpret_cast<void *>(used->virtual_address);
 	}
@@ -415,101 +368,110 @@ page_manager::map_offset_pages(size_t count)
 void
 page_manager::unmap_pages(void* address, size_t count)
 {
-	addr_t addr = reinterpret_cast<addr_t>(address);
+	uintptr_t addr = reinterpret_cast<uintptr_t>(address);
+	size_t block_count = 0;
 
-	page_block **prev = &m_used;
-	page_block *cur = m_used;
-	while (cur && !cur->contains(addr)) {
-		prev = &cur->next;
-		cur = cur->next;
-	}
+	for (auto *block : m_used) {
+		if (!block->contains(addr)) continue;
 
-	kassert(cur, "Couldn't find existing mapped pages to unmap");
+		size_t size = page_size * count;
+		uintptr_t end = addr + size;
 
-	size_t size = page_size * count;
-	addr_t end = addr + size;
-
-	while (cur && cur->contains(addr)) {
-		size_t leading = addr - cur->virtual_address;
+		size_t leading = addr - block->virtual_address;
 		size_t trailing =
-			end > cur->virtual_end() ?
-			0 : (cur->virtual_end() - end);
+			end > block->virtual_end() ?
+			0 : (block->virtual_end() - end);
 
 		if (leading) {
 			size_t pages = leading / page_size;
 
-			page_block *lead_block = get_block();
-			lead_block->copy(cur);
-			lead_block->next = cur;
+			auto *lead_block = m_block_slab.pop();
+
+			lead_block->copy(block);
 			lead_block->count = pages;
 
-			cur->count -= pages;
-			cur->physical_address += leading;
-			cur->virtual_address += leading;
+			block->count -= pages;
+			block->physical_address += leading;
+			block->virtual_address += leading;
 
-			*prev = lead_block;
-			prev = &lead_block->next;
+			m_used.insert_before(block, lead_block);
 		}
 
 		if (trailing) {
 			size_t pages = trailing / page_size;
 
-			page_block *trail_block = get_block();
-			trail_block->copy(cur);
-			trail_block->next = cur->next;
+			auto *trail_block = m_block_slab.pop();
+
+			trail_block->copy(block);
 			trail_block->count = pages;
 			trail_block->physical_address += size;
 			trail_block->virtual_address += size;
 
-			cur->count -= pages;
+			block->count -= pages;
 
-			cur->next = trail_block;
+			m_used.insert_after(block, trail_block);
 		}
 
-		addr += cur->count * page_size;
-		page_block *next = cur->next;
+		addr += block->count * page_size;
 
-		*prev = cur->next;
-		cur->next = nullptr;
-		cur->virtual_address = 0;
-		cur->flags = cur->flags & ~(page_block_flags::used | page_block_flags::mapped);
-		m_free = page_block::insert(m_free, cur);
+		block->virtual_address = 0;
+		block->flags = block->flags &
+			~(page_block_flags::used | page_block_flags::mapped);
 
-		cur = next;
+		m_used.remove(block);
+		m_free.sorted_insert(block);
+		++block_count;
 	}
+
+	kassert(block_count, "Couldn't find existing mapped pages to unmap");
 }
 
 void
-page_manager::check_needs_page(page_table *table, unsigned index)
+page_manager::check_needs_page(page_table *table, unsigned index, bool user)
 {
 	if ((table->entries[index] & 0x1) == 1) return;
 
 	page_table *new_table = get_table_page();
 	for (int i=0; i<512; ++i) new_table->entries[i] = 0;
-	table->entries[index] = pt_to_phys(new_table) | 0xb;
+	table->entries[index] = pt_to_phys(new_table) | (user ? 0xf : 0xb);
 }
 
 void
-page_manager::page_in(page_table *pml4, addr_t phys_addr, addr_t virt_addr, size_t count)
+page_manager::page_in(page_table *pml4, uintptr_t phys_addr, uintptr_t virt_addr, size_t count, bool user)
 {
 	page_table_indices idx{virt_addr};
 	page_table *tables[4] = {pml4, nullptr, nullptr, nullptr};
 
+	uint64_t flags = user ?
+		0x00f:	// writethru, user, write, present
+		0x10b;	// global, writethru, write, present
+
 	for (; idx[0] < 512; idx[0] += 1) {
-		check_needs_page(tables[0], idx[0]);
+		check_needs_page(tables[0], idx[0], user);
 		tables[1] = tables[0]->get(idx[0]);
 
 		for (; idx[1] < 512; idx[1] += 1, idx[2] = 0, idx[3] = 0) {
-			check_needs_page(tables[1], idx[1]);
+			check_needs_page(tables[1], idx[1], user);
 			tables[2] = tables[1]->get(idx[1]);
 
 			for (; idx[2] < 512; idx[2] += 1, idx[3] = 0) {
-				check_needs_page(tables[2], idx[2]);
+				if (idx[3] == 0 &&
+					count >= 512 &&
+					tables[2]->get(idx[2]) == nullptr) {
+					// Do a 2MiB page instead
+					tables[2]->entries[idx[2]] = phys_addr | flags | 0x80;
+					phys_addr += page_size * 512;
+					count -= 512;
+					if (count == 0) return;
+					continue;
+				}
+
+				check_needs_page(tables[2], idx[2], user);
 				tables[3] = tables[2]->get(idx[2]);
 
 				for (; idx[3] < 512; idx[3] += 1) {
-					tables[3]->entries[idx[3]] = phys_addr | 0xb;
-					phys_addr += page_manager::page_size;
+					tables[3]->entries[idx[3]] = phys_addr | flags;
+					phys_addr += page_size;
 					if (--count == 0) return;
 				}
 			}
@@ -520,7 +482,7 @@ page_manager::page_in(page_table *pml4, addr_t phys_addr, addr_t virt_addr, size
 }
 
 void
-page_manager::page_out(page_table *pml4, addr_t virt_addr, size_t count)
+page_manager::page_out(page_table *pml4, uintptr_t virt_addr, size_t count)
 {
 	page_table_indices idx{virt_addr};
 	page_table *tables[4] = {pml4, nullptr, nullptr, nullptr};
@@ -549,62 +511,59 @@ page_manager::page_out(page_table *pml4, addr_t virt_addr, size_t count)
 }
 
 size_t
-page_manager::pop_pages(size_t count, addr_t *address)
+page_manager::pop_pages(size_t count, uintptr_t *address)
 {
-	kassert(m_free, "page_manager::pop_pages ran out of free pages!");
+	kassert(!m_free.empty(), "page_manager::pop_pages ran out of free pages!");
 
-	unsigned n = std::min(count, static_cast<size_t>(m_free->count));
-	*address = m_free->physical_address;
+	auto *first = m_free.front();
 
-	m_free->physical_address += n * page_size;
-	m_free->count -= n;
-	if (m_free->count == 0) {
-		page_block *block = m_free;
-		m_free = m_free->next;
+	unsigned n = std::min(count, static_cast<size_t>(first->count));
+	*address = first->physical_address;
 
-		block->zero(m_block_cache);
-		m_block_cache = block;
-	}
+	first->physical_address += n * page_size;
+	first->count -= n;
+	if (first->count == 0)
+		m_block_slab.push(m_free.pop_front());
 
 	return n;
 }
 
 
 void
-page_table::dump(int level, uint64_t offset)
+page_table::dump(int level, int max_index, uint64_t offset)
 {
-	log::info(logs::memory, "Level %d page table @ %lx (off %lx):", level, this, offset);
+	console *cons = console::get();
+
+	cons->printf("\nLevel %d page table @ %lx (off %lx):\n", level, this, offset);
 	for (int i=0; i<512; ++i) {
 		uint64_t ent = entries[i];
 		if (ent == 0) continue;
 
 		if ((ent & 0x1) == 0) {
-			log::info(logs::memory, "  %3d: %lx   NOT PRESENT", i, ent);
+			cons->printf("  %3d: %016lx   NOT PRESENT\n", i, ent);
 			continue;
 		}
 
 		if ((level == 2 || level == 3) && (ent & 0x80) == 0x80) {
-			log::info(logs::memory, "  %3d: %lx   -> Large page at     %lx",
-					i, ent, ent & ~0xfffull);
+			cons->printf("  %3d: %016lx -> Large page at    %016lx\n", i, ent, ent & ~0xfffull);
 			continue;
 		} else if (level == 1) {
-			log::info(logs::memory, "  %3d: %lx   -> Page at           %lx",
-					i, ent, ent & ~0xfffull);
+			cons->printf("  %3d: %016lx -> Page at          %016lx\n", i, ent, ent & ~0xfffull);
 		} else {
-			log::info(logs::memory, "  %3d: %lx   -> Level %d table at %lx",
+			cons->printf("  %3d: %016lx -> Level %d table at %016lx\n",
 					i, ent, level - 1, (ent & ~0xfffull) + offset);
 			continue;
 		}
 	}
 
 	if (--level > 0) {
-		for (int i=0; i<512; ++i) {
+		for (int i=0; i<=max_index; ++i) {
 			uint64_t ent = entries[i];
 			if ((ent & 0x1) == 0) continue;
 			if ((ent & 0x80)) continue;
 
 			page_table *next = reinterpret_cast<page_table *>((ent & ~0xffful) + offset); 
-			next->dump(level, offset);
+			next->dump(level, 511, offset);
 		}
 	}
 }

src/kernel/page_manager.h

@@ -5,13 +5,16 @@
 #include <stddef.h>
 #include <stdint.h>
 
-#include "kutil/memory.h"
 #include "kutil/enum_bitfields.h"
+#include "kutil/linked_list.h"
+#include "kutil/slab_allocator.h"
 
 struct page_block;
 struct page_table;
 struct free_page_header;
 
+using page_block_list = kutil::linked_list<page_block>;
+using page_block_slab = kutil::slab_allocator<page_block>;
 
 /// Manager for allocation of physical pages.
 class page_manager
@@ -21,18 +24,59 @@ public:
 	static const size_t page_size = 0x1000;
 
 	/// Start of the higher half.
-	static const addr_t high_offset = 0xffff800000000000;
+	static const uintptr_t high_offset = 0xffffff0000000000;
 
 	/// Offset from physical where page tables are mapped.
-	static const addr_t page_offset = 0xffffff8000000000;
+	static const uintptr_t page_offset = 0xffffff8000000000;
+
+	/// Initial process thread's stack address
+	static const uintptr_t initial_stack = 0x0000800000000000;
+
+	/// Initial process thread's stack size, in pages
+	static const unsigned initial_stack_pages = 1;
 
 	page_manager();
 
+	/// Helper to get the number of pages needed for a given number of bytes.
+	/// \arg bytes  The number of bytes desired
+	/// \returns    The number of pages needed to contain the desired bytes
+	static inline size_t page_count(size_t bytes)
+	{
+		return (bytes - 1) / page_size + 1;
+	}
+
+	/// Helper to read the PML4 table from CR3.
+	/// \returns  A pointer to the current PML4 table.
+	static inline page_table * get_pml4()
+	{
+		uintptr_t pml4 = 0;
+		__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (pml4) );
+		return reinterpret_cast<page_table *>((pml4 & ~0xfffull) + page_offset);
+	}
+
+	/// Helper to set the PML4 table pointer in CR3.
+	/// \arg pml4  A pointer to the PML4 table to install.
+	static inline void set_pml4(page_table *pml4)
+	{
+		uintptr_t p = reinterpret_cast<uintptr_t>(pml4) - page_offset;
+		__asm__ __volatile__ ( "mov %0, %%cr3" :: "r" (p & ~0xfffull) );
+	}
+
+	/// Allocate but don't switch to a new PML4 table. This table
+	/// should only have global kernel pages mapped.
+	/// \returns      A pointer to the PML4 table
+	page_table * create_process_map();
+
+	/// Deallocate a process' PML4 table.
+	void delete_process_map(page_table *table);
+
 	/// Allocate and map pages into virtual memory.
 	/// \arg address  The virtual address at which to map the pages
 	/// \arg count    The number of pages to map
+	/// \arg user     True is this memory is user-accessible
+	/// \arg pml4     The pml4 to map into - null for the current one
 	/// \returns      A pointer to the start of the mapped region
-	void * map_pages(addr_t address, size_t count);
+	void * map_pages(uintptr_t address, size_t count, bool user = false, page_table *pml4 = nullptr);
 
 	/// Allocate and map contiguous pages into virtual memory, with
 	/// a constant offset from their physical address.
@@ -54,21 +98,27 @@ public:
 	/// Get the physical address of an offset-mapped pointer
 	/// \arg p   Virtual address of memory that has been offset-mapped
 	/// \returns Physical address of the memory pointed to by p
-	inline addr_t offset_phys(void *p) const
+	inline uintptr_t offset_phys(void *p) const
 	{
-		return reinterpret_cast<addr_t>(kutil::offset_pointer(p, -page_offset));
+		return reinterpret_cast<uintptr_t>(kutil::offset_pointer(p, -page_offset));
 	}
 
 	/// Get the virtual address of an offset-mapped physical address
 	/// \arg a   Physical address of memory that has been offset-mapped
 	/// \returns Virtual address of the memory at address a
-	inline void * offset_virt(addr_t a) const
+	inline void * offset_virt(uintptr_t a) const
 	{
 		return kutil::offset_pointer(reinterpret_cast<void *>(a), page_offset);
 	}
 
 	/// Log the current free/used block lists.
-	void dump_blocks();
+	/// \arg used_only  If true, skip printing free list. Default false.
+	void dump_blocks(bool used_only = false);
+
+	/// Dump the given or current PML4 to the console
+	/// \arg pml4       The page table to use, null for the current one
+	/// \arg max_index  The max index of pml4 to print
+	void dump_pml4(page_table *pml4 = nullptr, int max_index = 511);
 
 	/// Get the system page manager.
 	/// \returns  A pointer to the system page manager
@@ -77,9 +127,9 @@ public:
 private:
 	/// Set up the memory manager from bootstraped memory
 	void init(
-			page_block *free,
-			page_block *used,
-			page_block *block_cache);
+			page_block_list free,
+			page_block_list used,
+			page_block_list cache);
 
 	/// Initialize the virtual memory manager based on this object's state
 	void init_memory_manager();
@@ -105,40 +155,26 @@ private:
 	/// to the cache.
 	void consolidate_blocks();
 
-	/// Helper to read the PML4 table from CR3.
-	/// \returns  A pointer to the current PML4 table.
-	static inline page_table * get_pml4()
-	{
-		addr_t pml4 = 0;
-		__asm__ __volatile__ ( "mov %%cr3, %0" : "=r" (pml4) );
-		return reinterpret_cast<page_table *>((pml4 & ~0xfffull) + page_offset);
-	}
-
-	/// Helper to set the PML4 table pointer in CR3.
-	/// \arg pml4  A pointer to the PML4 table to install.
-	static inline void set_pml4(page_table *pml4)
-	{
-		addr_t p = reinterpret_cast<addr_t>(pml4) - page_offset;
-		__asm__ __volatile__ ( "mov %0, %%cr3" :: "r" (p & ~0xfffull) );
-	}
-
 	/// Helper function to allocate a new page table. If table entry `i` in
 	/// table `base` is empty, allocate a new page table and point `base[i]` at
 	/// it.
 	/// \arg base  Existing page table being indexed into
 	/// \arg i     Index into the existing table to check
-	void check_needs_page(page_table *base, unsigned i);
+	/// \art user  True if this is a userspace mapping
+	void check_needs_page(page_table *base, unsigned i, bool user);
 
 	/// Low-level routine for mapping a number of pages into the given page table.
 	/// \arg pml4       The root page table to map into
 	/// \arg phys_addr  The starting physical address of the pages to be mapped
 	/// \arg virt_addr  The starting virtual address ot the memory to be mapped
 	/// \arg count      The number of pages to map
+	/// \art user  True if this is a userspace mapping
 	void page_in(
 			page_table *pml4,
-			addr_t phys_addr,
-			addr_t virt_addr,
-			size_t count);
+			uintptr_t phys_addr,
+			uintptr_t virt_addr,
+			size_t count,
+			bool user = false);
 
 	/// Low-level routine for unmapping a number of pages from the given page table.
 	/// \arg pml4       The root page table for this mapping
@@ -146,7 +182,7 @@ private:
 	/// \arg count      The number of pages to unmap
 	void page_out(
 			page_table *pml4,
-			addr_t virt_addr,
+			uintptr_t virt_addr,
 			size_t count);
 
 	/// Get free pages from the free list. Only pages from the first free block
@@ -155,12 +191,14 @@ private:
 	/// \arg count    The maximum number of pages to get
 	/// \arg address  [out] The address of the first page
 	/// \returns      The number of pages retrieved
-	size_t pop_pages(size_t count, addr_t *address);
+	size_t pop_pages(size_t count, uintptr_t *address);
 
-	page_block *m_free; ///< Free pages list
-	page_block *m_used; ///< In-use pages list
+	page_table *m_kernel_pml4; ///< The PML4 of just kernel pages
+
+	page_block_list m_free; ///< Free pages list
+	page_block_list m_used; ///< In-use pages list
+	page_block_slab m_block_slab; ///< page_block slab allocator
 
-	page_block *m_block_cache; ///< Cache of unused page_block structs
 	free_page_header *m_page_cache; ///< Cache of free pages to use for tables
 
 	friend void memory_initialize(const void *, size_t, size_t);
@@ -196,67 +234,40 @@ IS_BITFIELD(page_block_flags);
 /// linked list of such structures.
 struct page_block
 {
-	addr_t physical_address;
-	addr_t virtual_address;
+	uintptr_t physical_address;
+	uintptr_t virtual_address;
 	uint32_t count;
 	page_block_flags flags;
-	page_block *next;
 
 	inline bool has_flag(page_block_flags f) const { return bitfield_has(flags, f); }
-	inline addr_t physical_end() const { return physical_address + (count * page_manager::page_size); }
-	inline addr_t virtual_end() const { return virtual_address + (count * page_manager::page_size); }
+	inline uintptr_t physical_end() const { return physical_address + (count * page_manager::page_size); }
+	inline uintptr_t virtual_end() const { return virtual_address + (count * page_manager::page_size); }
 
-	inline bool contains(addr_t vaddr) const { return vaddr >= virtual_address && vaddr < virtual_end(); }
-	inline bool contains_physical(addr_t addr) const { return addr >= physical_address && addr < physical_end(); }
+	inline bool contains(uintptr_t vaddr) const { return vaddr >= virtual_address && vaddr < virtual_end(); }
+	inline bool contains_physical(uintptr_t addr) const { return addr >= physical_address && addr < physical_end(); }
 
 	/// Helper to zero out a block and optionally set the next pointer.
-	/// \arg next  [optional] The value for the `next` pointer
-	void zero(page_block *set_next = nullptr);
+	void zero();
 
 	/// Helper to copy a bock from another block
 	/// \arg other  The block to copy from
 	void copy(page_block *other);
 
-	/// \name Page block linked list functions
-	/// Functions to act on a `page_block *` as a linked list
-	/// @{
-
-	/// Count the items in the given linked list.
-	/// \arg list  The list to count
-	/// \returns   The number of entries in the list.
-	static size_t length(page_block *list);
-
-	/// Append a block or list to the given list.
-	/// \arg list   The list to append to
-	/// \arg extra  The list or block to be appended 
-	/// \returns    The new list head
-	static page_block * append(page_block *list, page_block *extra);
-
-	/// Sorted-insert of a block into the list by address.
-	/// \arg list   The list to insert into
-	/// \arg block  The single block to insert
-	/// \returns    The new list head
-	static page_block * insert(page_block *list, page_block *block);
-
 	/// Compare two blocks by address.
-	/// \arg lhs   The left-hand comparator
 	/// \arg rhs   The right-hand comparator
-	/// \returns   <0 if lhs is sorts earlier, >0 if lhs sorts later, 0 for equal
-	static int compare(const page_block *lhs, const page_block *rhs);
+	/// \returns   <0 if this is sorts earlier, >0 if this sorts later, 0 for equal
+	int compare(const page_block *rhs) const;
 
 	/// Traverse the list, joining adjacent blocks where possible.
 	/// \arg list  The list to consolidate
 	/// \returns   A linked list of freed page_block structures.
-	static page_block * consolidate(page_block *list);
+	static page_block_list consolidate(page_block_list &list);
 
 	/// Traverse the list, printing debug info on this list.
 	/// \arg list  The list to print
 	/// \arg name  [optional] String to print as the name of this list
 	/// \arg show_permanent [optional] If false, hide unmapped blocks
-	static void dump(page_block *list, const char *name = nullptr, bool show_unmapped = false);
-
-	/// @}
-
+	static void dump(const page_block_list &list, const char *name = nullptr, bool show_unmapped = false);
 };
 
 
@@ -278,7 +289,7 @@ struct page_table
 		entries[i] = (reinterpret_cast<uint64_t>(p) - pm::page_offset) | (flags & 0xfff);
 	}
 
-	void dump(int level = 4, uint64_t offset = page_manager::page_offset);
+	void dump(int level = 4, int max_index = 511, uint64_t offset = page_manager::page_offset);
 };
 
 
@@ -306,7 +317,7 @@ template <typename T> inline T
 page_align(T p)
 {
 	return reinterpret_cast<T>(
-		((reinterpret_cast<addr_t>(p) - 1) & ~(page_manager::page_size - 1))
+		((reinterpret_cast<uintptr_t>(p) - 1) & ~(page_manager::page_size - 1))
 		+ page_manager::page_size);
 }
 

src/kernel/pci.cpp

@@ -1,11 +1,32 @@
 #include "kutil/assert.h"
+#include "console.h"
 #include "log.h"
 #include "interrupts.h"
 #include "pci.h"
 
-struct pci_cap_msi :
-	public pci_cap
+struct pci_cap_msi
 {
+	pci_cap::type id;
+	uint8_t next;
+	uint16_t control;
+} __attribute__ ((packed));
+
+struct pci_cap_msi32
+{
+	pci_cap::type id;
+	uint8_t next;
+	uint16_t control;
+	uint32_t address;
+	uint16_t data;
+	uint16_t reserved;
+	uint32_t mask;
+	uint32_t pending;
+} __attribute__ ((packed));
+
+struct pci_cap_msi64
+{
+	pci_cap::type id;
+	uint8_t next;
 	uint16_t control;
 	uint64_t address;
 	uint16_t data;
@@ -14,17 +35,33 @@ struct pci_cap_msi :
 	uint32_t pending;
 } __attribute__ ((packed));
 
-struct pci_cap_msix :
-	public pci_cap
-{
-	uint16_t control;
-	uint64_t address;
-	uint16_t data;
-	uint16_t reserved;
-	uint32_t mask;
-	uint32_t pending;
-} __attribute__ ((packed));
 
+void dump_msi(pci_cap_msi *cap)
+{
+	auto cons = console::get();
+	cons->printf("MSI Cap:\n");
+	cons->printf("     id: %02x\n", cap->id);
+	cons->printf("   next: %02x\n", cap->next);
+	cons->printf("control: %04x\n", cap->control);
+	if (cap->control & 0x0080) {
+		pci_cap_msi64 *cap64 = reinterpret_cast<pci_cap_msi64 *>(cap);
+		cons->printf("address: %016x\n", cap64->address);
+		cons->printf("   data: %04x\n", cap64->data);
+		if (cap->control & 0x100) {
+			cons->printf("   mask: %08x\n", cap64->mask);
+			cons->printf("pending: %08x\n", cap64->pending);
+		}
+	} else {
+		pci_cap_msi32 *cap32 = reinterpret_cast<pci_cap_msi32 *>(cap);
+		cons->printf("address: %08x\n", cap32->address);
+		cons->printf("   data: %04x\n", cap32->data);
+		if (cap->control & 0x100) {
+			cons->printf("   mask: %08x\n", cap32->mask);
+			cons->printf("pending: %08x\n", cap32->pending);
+		}
+	}
+	cons->putc('\n');
+};
 
 pci_device::pci_device() :
 	m_base(nullptr),
@@ -60,19 +97,25 @@ pci_device::pci_device(pci_group &group, uint8_t bus, uint8_t device, uint8_t fu
 	uint16_t *command = reinterpret_cast<uint16_t *>(&m_base[1]);
 	*command |= 0x400; // Mask old INTx style interrupts
 
-	log::info(logs::device, "Found PCIe device at %02d:%02d:%d of type %d.%d id %04x:%04x",
-			bus, device, func, m_class, m_subclass, m_vendor, m_device);
+	uint16_t *status = command + 1;
 
-	// Walk the extended capabilities list
-	uint8_t next = m_base[13] & 0xff;
-	while (next) {
-		pci_cap *cap = reinterpret_cast<pci_cap *>(kutil::offset_pointer(m_base, next));
-		next = cap->next;
+	log::info(logs::device, "Found PCIe device at %02d:%02d:%d of type %x.%x.%x id %04x:%04x",
+			bus, device, func, m_class, m_subclass, m_progif, m_vendor, m_device);
 
-		if (cap->id == pci_cap::type::msi) {
-			m_msi = cap;
-			pci_cap_msi *mcap = reinterpret_cast<pci_cap_msi *>(cap);
-			mcap->control |= ~0x1; // Mask interrupts
+	if (*status & 0x0010) {
+		// Walk the extended capabilities list
+		uint8_t next = m_base[13] & 0xff;
+		while (next) {
+			pci_cap *cap = reinterpret_cast<pci_cap *>(kutil::offset_pointer(m_base, next));
+			next = cap->next;
+			log::debug(logs::device, "  - found PCI cap type %02x", cap->id);
+
+			if (cap->id == pci_cap::type::msi) {
+				m_msi = cap;
+				pci_cap_msi *mcap = reinterpret_cast<pci_cap_msi *>(cap);
+				mcap->control &= ~0x70; // at most 1 vector allocated
+				mcap->control |= 0x01; // Enable interrupts, at most 1 vector allocated
+			}
 		}
 	}
 }
@@ -106,14 +149,24 @@ pci_device::set_bar(unsigned i, uint32_t val)
 }
 
 void
-pci_device::write_msi_regs(addr_t address, uint16_t data)
+pci_device::write_msi_regs(uintptr_t address, uint16_t data)
 {
 	kassert(m_msi, "Tried to write MSI for a device without that cap");
 	if (m_msi->id == pci_cap::type::msi) {
 		pci_cap_msi *mcap = reinterpret_cast<pci_cap_msi *>(m_msi);
-		mcap->address = address;
-		mcap->data = data;
-		mcap->control |= 1;
+		if (mcap->control & 0x0080) {
+			pci_cap_msi64 *mcap64 = reinterpret_cast<pci_cap_msi64 *>(m_msi);
+			mcap64->address = address;
+			mcap64->data = data;
+		} else {
+			pci_cap_msi32 *mcap32 = reinterpret_cast<pci_cap_msi32 *>(m_msi);
+			mcap32->address = address;
+			mcap32->data = data;
+		}
+		uint16_t control = mcap->control;
+		control &= 0xff8f; // We're allocating one vector, clear 6::4
+		control |= 0x0001; // Enable MSI
+		mcap->control = control;
 	} else {
 		kassert(0, "MIS-X is NYI");
 	}

src/kernel/pci.h

@@ -1,6 +1,7 @@
 #pragma once
 /// \file pci.h
 /// PCI devices and groups
+
 #include <stdint.h>
 #include "kutil/memory.h"
 
@@ -75,7 +76,7 @@ public:
 	/// Write to the MSI registers
 	/// \arg addr  The address to write to the MSI address registers
 	/// \arg data  The value to write to the MSI data register
-	void write_msi_regs(addr_t addr, uint16_t data);
+	void write_msi_regs(uintptr_t addr, uint16_t data);
 
 	/// Get a bus address, given the bus/device/function numbers.
 	/// \arg bus    Number of the bus

src/kernel/process.cpp

@@ -0,0 +1,64 @@
+#include "process.h"
+
+
+void
+process::wait_on_signal(uint64_t sigmask)
+{
+	waiting = process_wait::signal;
+	waiting_info = sigmask;
+	flags -= process_flags::ready;
+}
+
+void
+process::wait_on_child(uint32_t pid)
+{
+	waiting = process_wait::child;
+	waiting_info = pid;
+	flags -= process_flags::ready;
+}
+
+void
+process::wait_on_time(uint64_t time)
+{
+	waiting = process_wait::time;
+	waiting_info = time;
+	flags -= process_flags::ready;
+}
+
+bool
+process::wake_on_signal(int signal)
+{
+	if (waiting != process_wait::signal ||
+		(waiting_info & (1 << signal)) == 0)
+		return false;
+
+	waiting = process_wait::none;
+	flags += process_flags::ready;
+	return true;
+}
+
+bool
+process::wake_on_child(process *child)
+{
+	if (waiting != process_wait::child ||
+		(waiting_info && waiting_info != child->pid))
+		return false;
+
+	waiting = process_wait::none;
+	flags += process_flags::ready;
+	return true;
+}
+
+
+bool
+process::wake_on_time(uint64_t now)
+{
+	if (waiting != process_wait::time ||
+		waiting_info > now)
+		return false;
+
+	waiting = process_wait::none;
+	flags += process_flags::ready;
+	return true;
+}
+

src/kernel/process.h

@@ -0,0 +1,82 @@
+#pragma once
+/// \file process.h
+/// The processes and related definitions
+
+#include <stdint.h>
+#include "kutil/enum_bitfields.h"
+#include "kutil/linked_list.h"
+#include "page_manager.h"
+
+
+enum class process_flags : uint32_t
+{
+	running   = 0x00000001,
+	ready     = 0x00000002,
+	loading   = 0x00000004,
+
+	const_pri = 0x80000000,
+
+	none      = 0x00000000
+};
+IS_BITFIELD(process_flags);
+
+enum class process_wait : uint8_t
+{
+	none,
+	signal,
+	child,
+	time
+};
+
+/// A process
+struct process
+{
+	uint32_t pid;
+	uint32_t ppid;
+
+	process_flags flags;
+
+	uint16_t quanta;
+
+	uint8_t priority;
+
+	process_wait waiting;
+	uint64_t waiting_info;
+
+	uint32_t return_code;
+
+	uint32_t reserved1;
+
+	uintptr_t rsp;
+	page_table *pml4;
+
+	/// Unready this process until it gets a signal
+	/// \arg sigmask  A bitfield of signals to wake on
+	void wait_on_signal(uint64_t sigmask);
+
+	/// Unready this process until a child exits
+	/// \arg pid  PID of the child to wait for, or 0 for any
+	void wait_on_child(uint32_t pid);
+
+	/// Unready this process until after the given time
+	/// \arg time  The time after which to wake
+	void wait_on_time(uint64_t time);
+
+	/// If this process is waiting on the given signal, wake it
+	/// \argument signal  The signal sent to the process
+	/// \returns  True if this wake was handled
+	bool wake_on_signal(int signal);
+
+	/// If this process is waiting on the given child, wake it
+	/// \argument child  The process that exited
+	/// \returns  True if this wake was handled
+	bool wake_on_child(process *child);
+
+	/// If this process is waiting on a time, check it
+	/// \argument now  The current time
+	/// \returns  True if this wake was handled
+	bool wake_on_time(uint64_t now);
+};
+
+using process_list = kutil::linked_list<process>;
+using process_node = process_list::item_type;

src/kernel/push_all.inc

@@ -0,0 +1,48 @@
+%macro push_all_and_segments 0
+	push rax
+	push rcx
+	push rdx
+	push rbx
+	push rbp
+	push rsi
+	push rdi
+
+	push r8
+	push r9
+	push r10
+	push r11
+	push r12
+	push r13
+	push r14
+	push r15
+
+	mov ax, ds
+	push rax
+%endmacro
+
+%macro pop_all_and_segments 0
+	pop rax
+	mov ds, ax
+	mov es, ax
+	mov fs, ax
+	mov gs, ax
+
+	pop r15
+	pop r14
+	pop r13
+	pop r12
+	pop r11
+	pop r10
+	pop r9
+	pop r8
+
+	pop rdi
+	pop rsi
+	pop rbp
+	pop rbx
+	pop rdx
+	pop rcx
+	pop rax
+%endmacro
+
+; vim: ft=asm

src/kernel/scheduler.cpp

@@ -0,0 +1,301 @@
+#include "apic.h"
+#include "console.h"
+#include "cpu.h"
+#include "gdt.h"
+#include "interrupts.h"
+#include "io.h"
+#include "log.h"
+#include "msr.h"
+#include "page_manager.h"
+#include "scheduler.h"
+
+#include "elf/elf.h"
+#include "kutil/assert.h"
+
+scheduler scheduler::s_instance(nullptr);
+
+const int stack_size = 0x1000;
+const uint64_t rflags_noint = 0x002;
+const uint64_t rflags_int = 0x202;
+
+extern "C" {
+	void ramdisk_process_loader();
+	void load_process(const void *image_start, size_t bytes, process *proc, cpu_state state);
+};
+
+scheduler::scheduler(lapic *apic) :
+	m_apic(apic),
+	m_next_pid(1)
+{
+	auto *idle = m_process_allocator.pop();
+
+	uint8_t last_pri = num_priorities - 1;
+
+	// The kernel idle task, also the thread we're in now
+	idle->pid = 0;
+	idle->ppid = 0;
+	idle->priority = last_pri;
+	idle->rsp = 0; // This will get set when we switch away
+	idle->pml4 = page_manager::get_pml4();
+	idle->quanta = process_quanta;
+	idle->flags =
+		process_flags::running |
+		process_flags::ready |
+		process_flags::const_pri;
+
+	m_runlists[last_pri].push_back(idle);
+	m_current = idle;
+}
+
+void
+load_process(const void *image_start, size_t bytes, process *proc, cpu_state state)
+{
+	// We're now in the process space for this process, allocate memory for the
+	// process code and load it
+	page_manager *pager = page_manager::get();
+
+	log::debug(logs::task, "Loading task! ELF: %016lx [%d]", image_start, bytes);
+
+	// TODO: Handle bad images gracefully
+	elf::elf image(image_start, bytes);
+	kassert(image.valid(), "Invalid ELF passed to load_process");
+
+	const unsigned program_count = image.program_count();
+	for (unsigned i = 0; i < program_count; ++i) {
+		const elf::program_header *header = image.program(i);
+
+		if (header->type != elf::segment_type::load)
+			continue;
+
+		uintptr_t aligned = header->vaddr & ~(page_manager::page_size - 1);
+		size_t size = (header->vaddr + header->mem_size) - aligned;
+		size_t pages = page_manager::page_count(size);
+
+		log::debug(logs::task, "  Loadable segment %02u: vaddr %016lx  size %016lx",
+			i, header->vaddr, header->mem_size);
+
+		log::debug(logs::task, "         - aligned to: vaddr %016lx  pages %d",
+			aligned, pages);
+
+		void *mapped = pager->map_pages(aligned, pages, true);
+		kassert(mapped, "Tried to map userspace pages and failed!");
+
+		kutil::memset(mapped, 0, pages * page_manager::page_size);
+	}
+
+	const unsigned section_count = image.section_count();
+	for (unsigned i = 0; i < section_count; ++i) {
+		const elf::section_header *header = image.section(i);
+
+		if (header->type != elf::section_type::progbits ||
+			!bitfield_has(header->flags, elf::section_flags::alloc))
+			continue;
+
+		log::debug(logs::task, "  Loadable section %02u: vaddr %016lx  size %016lx",
+			i, header->addr, header->size);
+
+		void *dest = reinterpret_cast<void *>(header->addr);
+		const void *src = kutil::offset_pointer(image_start, header->offset);
+		kutil::memcpy(dest, src, header->size);
+	}
+
+	state.rip = image.entrypoint();
+	proc->flags &= ~process_flags::loading;
+
+	log::debug(logs::task, "  Loaded! New process rip: %016lx", state.rip);
+}
+
+void
+scheduler::create_process(const char *name, const void *data, size_t size)
+{
+	uint16_t kcs = (1 << 3) | 0; // Kernel CS is GDT entry 1, ring 0
+	uint16_t cs = (5 << 3) | 3;  // User CS is GDT entry 5, ring 3
+
+	uint16_t kss = (2 << 3) | 0; // Kernel SS is GDT entry 2, ring 0
+	uint16_t ss = (4 << 3) | 3;  // User SS is GDT entry 4, ring 3
+
+	// Set up the page tables - this also allocates an initial user stack
+	page_table *pml4 = page_manager::get()->create_process_map();
+
+	// Create a one-page kernel stack space
+	void *stack0 = kutil::malloc(stack_size);
+	kutil::memset(stack0, 0, stack_size);
+
+	// Stack grows down, point to the end
+	void *sp0 = kutil::offset_pointer(stack0, stack_size);
+
+	cpu_state *state = reinterpret_cast<cpu_state *>(sp0) - 1;
+
+	// Highest state in the stack is the process' kernel stack for the loader
+	// to iret to:
+	state->ds = state->ss = ss;
+	state->cs = cs;
+	state->rflags = rflags_int;
+	state->rip = 0; // to be filled by the loader
+	state->user_rsp = page_manager::initial_stack;
+
+	// Next state in the stack is the loader's kernel stack. The scheduler will
+	// iret to this which will kick off the loading:
+	cpu_state *loader_state = reinterpret_cast<cpu_state *>(sp0) - 2;
+
+	loader_state->ds = loader_state->ss = kss;
+	loader_state->cs = kcs;
+	loader_state->rflags = rflags_noint;
+	loader_state->rip = reinterpret_cast<uint64_t>(ramdisk_process_loader);
+	loader_state->user_rsp = reinterpret_cast<uint64_t>(state);
+
+	loader_state->rax = reinterpret_cast<uint64_t>(data);
+	loader_state->rbx = size;
+
+	uint16_t pid = m_next_pid++;
+	auto *proc = m_process_allocator.pop();
+
+	proc->pid = pid;
+	proc->ppid = 0; // TODO
+	proc->priority = default_priority;
+	proc->rsp = reinterpret_cast<uintptr_t>(loader_state);
+	proc->pml4 = pml4;
+	proc->quanta = process_quanta;
+	proc->flags =
+		process_flags::running |
+		process_flags::ready |
+		process_flags::loading;
+
+	m_runlists[default_priority].push_back(proc);
+
+	loader_state->rcx = reinterpret_cast<uint64_t>(proc);
+
+	log::debug(logs::task, "Creating process %s: pid %d  pri %d", name, proc->pid, proc->priority);
+	log::debug(logs::task, "     RSP0 %016lx", state);
+	log::debug(logs::task, "     PML4 %016lx", pml4);
+}
+
+void
+scheduler::start()
+{
+	log::info(logs::task, "Starting scheduler.");
+	m_tick_count = m_apic->enable_timer(isr::isrTimer, quantum_micros, false);
+}
+
+void scheduler::prune(uint64_t now)
+{
+	// Find processes that aren't ready or aren't running and
+	// move them to the appropriate lists.
+	for (auto &pri_list : m_runlists) {
+		auto *proc = pri_list.front();
+		while (proc) {
+			bool running = proc->flags && process_flags::running;
+			bool ready = proc->flags && process_flags::ready;
+			if (running && ready) {
+				proc = proc->next();
+				continue;
+			}
+
+			auto *remove = proc;
+			proc = proc->next();
+			pri_list.remove(remove);
+
+			if (!(remove->flags && process_flags::running)) {
+				auto *parent = get_process_by_id(remove->ppid);
+				if (parent && parent->wake_on_child(remove)) {
+					m_blocked.remove(parent);
+					m_runlists[parent->priority].push_front(parent);
+					m_process_allocator.push(remove);
+				} else {
+					m_exited.push_back(remove);
+				}
+			} else {
+				m_blocked.push_back(remove);
+			}
+		}
+	}
+
+	// Find blocked processes that are ready (possibly after waking wating
+	// ones) and move them to the appropriate runlist.
+	auto *proc = m_blocked.front();
+	while (proc) {
+		bool ready = proc->flags && process_flags::ready;
+		ready |= proc->wake_on_time(now);
+		if (!ready) {
+			proc = proc->next();
+			continue;
+		}
+
+		auto *remove = proc;
+		proc = proc->next();
+		m_blocked.remove(remove);
+		m_runlists[remove->priority].push_front(remove);
+	}
+}
+
+uintptr_t
+scheduler::schedule(uintptr_t rsp0)
+{
+
+	// TODO: lol a real clock
+	static uint64_t now = 0;
+	prune(++now);
+
+	m_current->rsp = rsp0;
+	m_runlists[m_current->priority].remove(m_current);
+
+	if (m_current->flags && process_flags::ready)
+		m_runlists[m_current->priority].push_back(m_current);
+	else
+		m_blocked.push_back(m_current);
+
+	uint8_t pri = 0;
+	while (m_runlists[pri].empty()) {
+		++pri;
+		kassert(pri < num_priorities, "All runlists are empty");
+	}
+
+	m_current = m_runlists[pri].pop_front();
+	rsp0 = m_current->rsp;
+
+	// Set rsp0 to after the end of the about-to-be-popped cpu state
+	tss_set_stack(0, rsp0 + sizeof(cpu_state));
+	wrmsr(msr::ia32_kernel_gs_base, rsp0);
+
+	// Swap page tables
+	page_table *pml4 = m_current->pml4;
+	page_manager::set_pml4(pml4);
+
+	bool loading = m_current->flags && process_flags::loading;
+	log::debug(logs::task, "Scheduler switched to process %d, priority %d%s.",
+			m_current->pid, m_current->priority, loading ? " (loading)" : "");
+
+	return rsp0;
+}
+
+uintptr_t
+scheduler::tick(uintptr_t rsp0)
+{
+	if (--m_current->quanta == 0) {
+		m_current->quanta = process_quanta;
+		rsp0 = schedule(rsp0);
+	}
+	m_apic->reset_timer(m_tick_count);
+	return rsp0;
+}
+
+process_node *
+scheduler::get_process_by_id(uint32_t pid)
+{
+	// TODO: this needs to be a hash map
+	for (auto *proc : m_blocked) {
+		if (proc->pid == pid) return proc;
+	}
+
+	for (int i = 0; i < num_priorities; ++i) {
+		for (auto *proc : m_runlists[i]) {
+			if (proc->pid == pid) return proc;
+		}
+	}
+
+	for (auto *proc : m_exited) {
+		if (proc->pid == pid) return proc;
+	}
+
+	return nullptr;
+}

src/kernel/scheduler.h

@@ -0,0 +1,86 @@
+#pragma once
+/// \file scheduler.h
+/// The task scheduler and related definitions
+
+#include <stdint.h>
+#include "kutil/slab_allocator.h"
+#include "process.h"
+
+class lapic;
+struct page_table;
+struct cpu_state;
+
+extern "C" uintptr_t isr_handler(uintptr_t, cpu_state);
+
+
+/// The task scheduler
+class scheduler
+{
+public:
+	static const uint8_t num_priorities = 8;
+	static const uint8_t default_priority = num_priorities / 2;
+
+	/// How long the timer quantum is
+	static const uint64_t quantum_micros = 100000;
+
+	/// How many quantums a process gets before being rescheduled
+	static const uint16_t process_quanta = 10;
+
+	/// Constructor.
+	/// \arg apic  Pointer to the local APIC object
+	scheduler(lapic *apic);
+
+	/// Create a new process from a program image in memory.
+	/// \arg name  Name of the program image
+	/// \arg data  Pointer to the image data
+	/// \arg size  Size of the program image, in bytes
+	void create_process(const char *name, const void *data, size_t size);
+
+	/// Start the scheduler working. This may involve starting
+	/// timer interrupts or other preemption methods.
+	void start();
+
+	/// Run the scheduler, possibly switching to a new task
+	/// \arg rsp0  The stack pointer of the current interrupt handler
+	/// \returns   The stack pointer to switch to
+	uintptr_t schedule(uintptr_t rsp0);
+
+	/// Get the current process.
+	/// \returns  A pointer to the current process' process struct
+	inline process * current() { return m_current; }
+
+	/// Look up a process by its PID
+	/// \arg pid  The requested PID
+	/// \returns  The process matching that PID, or nullptr
+	process_node * get_process_by_id(uint32_t pid);
+
+	/// Get a reference to the system scheduler
+	/// \returns  A reference to the global system scheduler
+	static scheduler & get() { return s_instance; }
+
+private:
+	friend uintptr_t syscall_dispatch(uintptr_t, const cpu_state &);
+	friend uintptr_t isr_handler(uintptr_t, cpu_state);
+
+	/// Handle a timer tick
+	/// \arg rsp0  The stack pointer of the current interrupt handler
+	/// \returns   The stack pointer to switch to
+	uintptr_t tick(uintptr_t rsp0);
+
+	void prune(uint64_t now);
+
+	lapic *m_apic;
+
+	uint32_t m_next_pid;
+	uint32_t m_tick_count;
+
+	using process_slab = kutil::slab_allocator<process>;
+	process_slab m_process_allocator;
+
+	process_node *m_current;
+	process_list m_runlists[num_priorities];
+	process_list m_blocked;
+	process_list m_exited;
+
+	static scheduler s_instance;
+};

src/kernel/syscall.cpp

@@ -0,0 +1,96 @@
+#include "console.h"
+#include "cpu.h"
+#include "debug.h"
+#include "msr.h"
+#include "process.h"
+#include "scheduler.h"
+#include "syscall.h"
+
+extern "C" {
+	void _halt();
+	void syscall_handler_prelude();
+}
+
+void
+syscall_enable()
+{
+	// IA32_EFER - set bit 0, syscall enable
+	uint64_t efer = rdmsr(msr::ia32_efer);
+	wrmsr(msr::ia32_efer, efer|1);
+
+	// IA32_STAR - high 32 bits contain k+u CS
+	// Kernel CS: GDT[1] ring 0 bits[47:32]
+	//   User CS: GDT[3] ring 3 bits[63:48]
+	uint64_t star =
+		(((1ull << 3) | 0) << 32) |
+		(((3ull << 3) | 3) << 48);
+	wrmsr(msr::ia32_star, star);
+
+	// IA32_LSTAR - RIP for syscall
+	wrmsr(msr::ia32_lstar,
+		reinterpret_cast<uintptr_t>(&syscall_handler_prelude));
+
+	// IA32_FMASK - FLAGS mask inside syscall
+	wrmsr(msr::ia32_fmask, 0x200);
+}
+
+uintptr_t
+syscall_dispatch(uintptr_t return_rsp, const cpu_state &regs)
+{
+	console *cons = console::get();
+	syscall call = static_cast<syscall>(regs.rax);
+
+	switch (call) {
+	case syscall::noop:
+		break;
+
+	case syscall::debug:
+		cons->set_color(11);
+		cons->printf("\nReceived DEBUG syscall\n");
+		cons->set_color();
+		print_regs(regs);
+		break;
+
+	case syscall::message:
+		cons->set_color(11);
+		cons->printf("\nReceived MESSAGE syscall\n");
+		cons->set_color();
+		break;
+
+	case syscall::pause:
+		{
+			cons->set_color(11);
+
+			auto &s = scheduler::get();
+			auto *p = s.current();
+			p->wait_on_signal(-1ull);
+			cons->printf("\nReceived PAUSE syscall\n");
+			return_rsp = s.tick(return_rsp);
+			cons->set_color();
+		}
+		break;
+
+	case syscall::sleep:
+		{
+			cons->set_color(11);
+
+			auto &s = scheduler::get();
+			auto *p = s.current();
+			p->wait_on_time(regs.rbx);
+			cons->printf("\nReceived SLEEP syscall\n");
+			return_rsp = s.tick(return_rsp);
+			cons->set_color();
+		}
+		break;
+
+	default:
+		cons->set_color(9);
+		cons->printf("\nReceived unknown syscall: %02x\n", call);
+		cons->set_color();
+		_halt();
+		break;
+	}
+
+	return return_rsp;
+}
+

src/kernel/syscall.h

@@ -0,0 +1,20 @@
+#pragma once
+
+#include <stdint.h>
+
+struct cpu_state;
+
+enum class syscall : uint64_t
+{
+	noop,
+	debug,
+	message,
+	pause,
+	sleep,
+
+	last_syscall
+};
+
+void syscall_enable();
+uintptr_t syscall_dispatch(uintptr_t, const cpu_state &);
+

src/kernel/syscall.s

@@ -0,0 +1,24 @@
+%include "push_all.inc"
+
+extern syscall_handler
+global syscall_handler_prelude
+syscall_handler_prelude:
+	push 0		; ss, doesn't matter here
+	push rsp
+	pushf
+	push 0		; cs, doesn't matter here
+	push rcx	; user rip
+	push 0		; bogus interrupt
+	push 0		; bogus errorcode
+	push_all_and_segments
+
+	mov rdi, rsp
+	call syscall_handler
+	mov rsp, rax
+
+	pop_all_and_segments
+	add rsp, 16	; ignore bogus interrupt / error
+	pop rcx		; user rip
+	add rsp, 32 ; ignore cs, flags, rsp, ss
+
+	o64 sysret

src/kernel/wscript

@@ -1,43 +0,0 @@
-
-def configure(ctx):
-    pass
-
-def build(bld):
-    from os.path import join
-
-    sources = bld.path.ant_glob("**/*.cpp")
-    sources += bld.path.ant_glob("**/*.s")
-
-    lds = join(bld.env.ARCH_D, 'kernel.ld')
-
-    bld.program(
-        source = sources,
-        name = 'kernel',
-        includes = '.',
-        target = bld.env.KERNEL_FILENAME,
-        use = 'kutil',
-        linkflags = "-T {}".format(lds),
-    )
-
-    from waflib.Task import Task
-    class objdump(Task):
-        color = 'PINK'
-        def keyword(self):
-            return "Dumping"
-        def __str__(self):
-            node = self.outputs[0]
-            return node.path_from(node.ctx.launch_node())
-        def run(self):
-            from subprocess import check_output
-            args = self.env.objdump + ["--source", "-D", self.inputs[0].abspath()]
-            with file(self.outputs[0].abspath(), 'w') as output:
-                output.write(check_output(args))
-
-    out = bld.path.get_bld()
-
-    dump = objdump(env=bld.env)
-    dump.set_inputs([out.make_node(bld.env.KERNEL_FILENAME)])
-    dump.set_outputs([out.make_node("kernel.dump")])
-    bld.add_to_group(dump)
-
-# vim: ft=python et

src/libraries/elf/elf.cpp

@@ -0,0 +1,50 @@
+#include "elf/elf.h"
+#include "elf/headers.h"
+#include "kutil/memory.h"
+
+static const uint32_t expected_magic = 0x464c457f;  // "\x7f" "ELF"
+
+namespace elf {
+
+elf::elf(const void *data, size_t size) :
+	m_data(data),
+	m_size(size)
+{
+}
+
+bool
+elf::valid() const
+{
+	const file_header *fheader = header();
+
+	return
+		fheader->magic == expected_magic &&
+		fheader->word_size == wordsize::bits64 &&
+		fheader->endianness == encoding::lsb &&
+		fheader->os_abi == osabi::sysV &&
+		fheader->file_type == filetype::executable &&
+		fheader->machine_type == machine::x64 &&
+		fheader->ident_version == 1 &&
+		fheader->version == 1;
+}
+
+const program_header *
+elf::program(unsigned index) const
+{
+	const file_header *fheader = header();
+	uint64_t off = fheader->ph_offset + (index * fheader->ph_entsize);
+	const void *pheader = kutil::offset_pointer(m_data, off);
+	return reinterpret_cast<const program_header *>(pheader);
+}
+
+const section_header *
+elf::section(unsigned index) const
+{
+	const file_header *fheader = header();
+	uint64_t off = fheader->sh_offset + (index * fheader->sh_entsize);
+	const void *sheader = kutil::offset_pointer(m_data, off);
+	return reinterpret_cast<const section_header *>(sheader);
+}
+
+
+} // namespace elf

src/libraries/elf/include/elf/elf.h

@@ -0,0 +1,61 @@
+#pragma once
+#include <stddef.h>
+#include <stdint.h>
+#include "elf/headers.h"
+
+namespace elf {
+
+class elf
+{
+public:
+	/// Constructor: Create an elf object out of ELF data in memory
+	/// \arg data  The ELF data to read
+	/// \arg size  Size of the ELF data, in bytes
+	elf(const void *data, size_t size);
+
+	/// Check the validity of the ELF data
+	/// \returns  true for valid ELF data
+	bool valid() const;
+
+	/// Get the entrypoint address of the program image
+	/// \returns  A pointer to the entrypoint of the program
+	inline uintptr_t entrypoint() const
+	{
+		return static_cast<uintptr_t>(header()->entrypoint);
+	}
+
+	/// Get the number of program sections in the image
+	/// \returns The number of program section entries
+	inline unsigned program_count() const
+	{
+		return header()->ph_num;
+	}
+
+	/// Get a program header
+	/// \arg index  The index number of the program header
+	/// \returns    A pointer to the program header data
+	const program_header * program(unsigned index) const;
+
+	/// Get the number of data sections in the image
+	/// \returns The number of section entries
+	inline unsigned section_count() const
+	{
+		return header()->sh_num;
+	}
+
+	/// Get a section header
+	/// \arg index  The index number of the section header
+	/// \returns    A pointer to the section header data
+	const section_header * section(unsigned index) const;
+
+private:
+	inline const file_header *header() const
+	{
+		return reinterpret_cast<const file_header *>(m_data);
+	}
+
+	const void *m_data;
+	size_t m_size;
+};
+
+}

src/libraries/elf/include/elf/headers.h

@@ -0,0 +1,98 @@
+#pragma once
+#include <stdint.h>
+#include "kutil/enum_bitfields.h"
+
+namespace elf {
+
+enum class wordsize : uint8_t { invalid, bits32, bits64 };
+enum class encoding : uint8_t { invalid, lsb, msb };
+enum class osabi : uint8_t { sysV, hpux, standalone = 255 };
+enum class machine : uint16_t { none, x64 = 0x3e };
+
+enum class filetype : uint16_t
+{
+	none,
+	relocatable,
+	executable,
+	shared,
+	core
+};
+
+
+struct file_header
+{
+	uint32_t magic;
+
+	wordsize word_size;
+	encoding endianness;
+	uint8_t ident_version;
+	osabi os_abi;
+
+	uint64_t reserved;
+
+	filetype file_type;
+	machine machine_type;
+
+	uint32_t version;
+
+	uint64_t entrypoint;
+	uint64_t ph_offset;
+	uint64_t sh_offset;
+
+	uint32_t flags;
+
+	uint16_t eh_size;
+
+	uint16_t ph_entsize;
+	uint16_t ph_num;
+
+	uint16_t sh_entsize;
+	uint16_t sh_num;
+
+	uint16_t sh_str_idx;
+} __attribute__ ((packed));
+
+
+enum class segment_type : uint32_t { null, load, dynamic, interpreter, note };
+
+struct program_header
+{
+	segment_type type;
+	uint32_t flags;
+	uint64_t offset;
+
+	uint64_t vaddr;
+	uint64_t paddr;
+
+	uint64_t file_size;
+	uint64_t mem_size;
+
+	uint64_t align;
+} __attribute__ ((packed));
+
+
+enum class section_type : uint32_t { null, progbits };
+enum class section_flags : uint64_t
+{
+	write = 0x01,
+	alloc = 0x02,
+	exec  = 0x04,
+};
+
+struct section_header
+{
+	uint32_t name_offset;
+	section_type type;
+	section_flags flags;
+	uint64_t addr;
+	uint64_t offset;
+	uint64_t size;
+	uint32_t link;
+	uint32_t info;
+	uint64_t align;
+	uint64_t entry_size;
+} __attribute__ ((packed));
+
+} // namespace elf
+
+IS_BITFIELD(elf::section_flags);

src/libraries/initrd/include/initrd/headers.h

@@ -0,0 +1,38 @@
+#pragma once
+#include <stdint.h>
+#include "kutil/enum_bitfields.h"
+
+namespace initrd {
+
+
+enum class disk_flags : uint16_t
+{
+};
+
+struct disk_header
+{
+	uint16_t file_count;
+	disk_flags flags;
+	uint32_t length;
+	uint8_t checksum;
+	uint8_t reserved[3];
+} __attribute__ ((packed));
+
+
+enum class file_flags : uint16_t
+{
+	executable = 0x01
+};
+
+struct file_header
+{
+	uint32_t offset;
+	uint32_t length;
+	uint16_t name_offset;
+	file_flags flags;
+} __attribute__ ((packed));
+
+} // namepsace initrd
+
+IS_BITFIELD(initrd::disk_flags);
+IS_BITFIELD(initrd::file_flags);

src/libraries/initrd/include/initrd/initrd.h

@@ -0,0 +1,61 @@
+#pragma once
+/// \file initrd.h
+/// Definitions defining the simple inital ramdisk file format used by the
+/// popcorn kernel.
+
+#include <stdint.h>
+#include "kutil/vector.h"
+
+// File format:
+// 1x disk_header
+// Nx file_header
+// filename string data
+// file data
+
+namespace initrd {
+
+struct disk_header;
+struct file_header;
+
+
+/// Encasulates methods for a file on the ramdisk
+class file
+{
+public:
+	file(const file_header *header, const void *start);
+
+	/// Get the filename
+	const char * name() const;
+
+	/// Get the file size
+	const size_t size() const;
+
+	/// Get a pointer to the file data
+	const void * data() const;
+
+	/// Whether this file is an executable
+	bool executable() const;
+
+private:
+	const file_header *m_header;
+	void const *m_data;
+	char const *m_name;
+};
+
+
+/// Encasulates access methods for the ramdisk
+class disk
+{
+public:
+	/// Constructor.
+	/// \arg start  The start of the initrd in memory
+	disk(const void *start);
+
+	/// Get the vector of files on the disk
+	const kutil::vector<file> & files() const { return m_files; }
+
+private:
+	kutil::vector<file> m_files;
+};
+
+} // namespace initrd

src/libraries/initrd/initrd.cpp

@@ -0,0 +1,41 @@
+#include "initrd/headers.h"
+#include "initrd/initrd.h"
+#include "kutil/assert.h"
+#include "kutil/enum_bitfields.h"
+
+namespace initrd {
+
+file::file(const file_header *header, const void *start) :
+	m_header(header)
+{
+	m_data = kutil::offset_pointer(start, m_header->offset);
+	auto *name = kutil::offset_pointer(start, m_header->name_offset);
+	m_name = reinterpret_cast<const char *>(name);
+}
+
+const char * file::name() const { return m_name; }
+const size_t file::size() const { return m_header->length; }
+const void * file::data() const { return m_data; }
+
+bool
+file::executable() const {
+	return bitfield_has(m_header->flags, file_flags::executable);
+}
+
+
+disk::disk(const void *start)
+{
+	auto *header = reinterpret_cast<const disk_header *>(start);
+	size_t length = header->length;
+	uint8_t sum = kutil::checksum(start, length);
+	kassert(sum == 0, "initrd checksum failed");
+
+	auto *files = reinterpret_cast<const file_header *>(header + 1);
+
+	m_files.ensure_capacity(header->file_count);
+	for (int i = 0; i < header->file_count; ++i) {
+		m_files.emplace(&files[i], start);
+	}
+}
+
+} // namespace initrd

src/libraries/kutil/assert.cpp

@@ -1,4 +1,4 @@
-#include "assert.h"
+#include "kutil/assert.h"
 
 namespace kutil {
 

src/libraries/kutil/include/kutil/enum_bitfields.h

@@ -8,27 +8,27 @@ struct is_enum_bitfield { static constexpr bool value = false; };
 #define IS_BITFIELD(name) \
 	template<> struct ::is_enum_bitfield<name> {static constexpr bool value=true;}
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type
-operator & (E lhs, E rhs)
+operator & (E lhs, F rhs)
 {
 	return static_cast<E> (
 		static_cast<typename std::underlying_type<E>::type>(lhs) &
 		static_cast<typename std::underlying_type<E>::type>(rhs));
 }
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type
-operator | (E lhs, E rhs)
+operator | (E lhs, F rhs)
 {
 	return static_cast<E> (
 		static_cast<typename std::underlying_type<E>::type>(lhs) |
 		static_cast<typename std::underlying_type<E>::type>(rhs));
 }
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type
-operator ^ (E lhs, E rhs)
+operator ^ (E lhs, F rhs)
 {
 	return static_cast<E> (
 		static_cast<typename std::underlying_type<E>::type>(lhs) ^
@@ -42,9 +42,9 @@ operator ~ (E rhs)
 	return static_cast<E>(~static_cast<typename std::underlying_type<E>::type>(rhs));
 }
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type&
-operator |= (E &lhs, E rhs)
+operator |= (E &lhs, F rhs)
 {
 	lhs = static_cast<E>(
 		static_cast<typename std::underlying_type<E>::type>(lhs) |
@@ -53,9 +53,9 @@ operator |= (E &lhs, E rhs)
 	return lhs;
 }
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type&
-operator &= (E &lhs, E rhs)
+operator &= (E &lhs, F rhs)
 {
 	lhs = static_cast<E>(
 		static_cast<typename std::underlying_type<E>::type>(lhs) &
@@ -64,9 +64,9 @@ operator &= (E &lhs, E rhs)
 	return lhs;
 }
 
-template <typename E>
+template <typename E, typename F>
 typename std::enable_if<is_enum_bitfield<E>::value,E>::type&
-operator ^= (E &lhs, E rhs)
+operator ^= (E &lhs, F rhs)
 {
 	lhs = static_cast<E>(
 		static_cast<typename std::underlying_type<E>::type>(lhs) ^
@@ -75,6 +75,28 @@ operator ^= (E &lhs, E rhs)
 	return lhs;
 }
 
+template <typename E, typename F>
+typename std::enable_if<is_enum_bitfield<E>::value,E>::type&
+operator -= (E &lhs, F rhs)
+{
+	lhs = static_cast<E>(
+		static_cast<typename std::underlying_type<E>::type>(lhs) &
+		~static_cast<typename std::underlying_type<E>::type>(rhs));
+
+	return lhs;
+}
+
+template <typename E, typename F>
+typename std::enable_if<is_enum_bitfield<E>::value,E>::type&
+operator += (E &lhs, F rhs)
+{
+	lhs = static_cast<E>(
+		static_cast<typename std::underlying_type<E>::type>(lhs) |
+		static_cast<typename std::underlying_type<E>::type>(rhs));
+
+	return lhs;
+}
+
 template <typename E>
 typename std::enable_if<is_enum_bitfield<E>::value,bool>::type
 operator ! (E rhs)
@@ -88,3 +110,11 @@ bitfield_has(E set, E flag)
 {
 	return (set & flag) == flag;
 }
+
+// Overload the logical-and operator to be 'bitwise-and, bool-cast'
+template <typename E>
+typename std::enable_if<is_enum_bitfield<E>::value,bool>::type
+operator && (E set, E flag)
+{
+	return (set & flag) == flag;
+}

src/libraries/kutil/include/kutil/guid.h

@@ -0,0 +1,38 @@
+#pragma once
+/// \file guid.h
+/// Definition of the guid type and related functions
+#include <stdint.h>
+#include "kutil/misc.h"
+
+namespace kutil {
+
+
+/// A GUID
+struct guid
+{
+	uint64_t a, b;
+};
+
+/// Make a GUID by writing it naturally-ordered in code:
+///  AAAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEEEE becomes:
+///  make_guid(0xAAAAAAAA, 0xBBBB, 0xCCCC, 0xDDDD, 0xEEEEEEEEEEEE);
+/// \returns  The guid object
+inline constexpr guid make_guid(uint32_t a, uint16_t b, uint16_t c, uint16_t d, uint64_t e)
+{
+	const uint64_t h =
+		static_cast<uint64_t>(c) << 48 |
+		static_cast<uint64_t>(b) << 32 |
+		a;
+
+	const uint64_t l =
+		static_cast<uint64_t>(byteswap(e & 0xffffffff)) << 32 |
+		(byteswap(e >> 32) & 0xffff0000) |
+		((d << 8) & 0xff00) | ((d >> 8) & 0xff);
+
+	return {h, l};
+}
+
+} // namespace kutil
+
+
+inline bool operator==(const kutil::guid &a, const kutil::guid &b) { return a.a == b.a && a.b == b.b; }

src/libraries/kutil/include/kutil/linked_list.h

@@ -0,0 +1,321 @@
+#pragma once
+/// \file linked_list.h
+/// A generic templatized linked list.
+#include <stddef.h>
+
+namespace kutil {
+
+template <typename T> class linked_list;
+
+
+/// A list node in a `linked_list<T>` or `sortable_linked_list<T>`.
+template <typename T>
+class list_node :
+	public T
+{
+public:
+	using item_type = T;
+	using node_type = list_node<T>;
+
+	/// Dereference operator. Helper to cast this node to the contained type.
+	/// \returns  A pointer to the node, cast to T*.
+	inline item_type & operator*() { return *this; }
+
+	/// Dereference operator. Helper to cast this node to the contained type.
+	/// \returns  A pointer to the node, cast to T*.
+	inline const item_type & operator*() const { return *this; }
+
+	/// Cast operator. Helper to cast this node to the contained type.
+	/// \returns  A reference to the node, cast to T&.
+	inline operator item_type& () { return *this; }
+
+	/// Cast operator. Helper to cast this node to the contained type.
+	/// \returns  A reference to the node, cast to const T&.
+	inline operator const item_type& () { return *this; }
+
+	/// Accessor for the next pointer.
+	/// \returns  The next node in the list
+	inline node_type * next() { return m_next; }
+
+	/// Accessor for the next pointer.
+	/// \returns  The next node in the list
+	inline const node_type * next() const { return m_next; }
+
+	/// Accessor for the prev pointer.
+	/// \returns  The prev node in the list
+	inline node_type * prev() { return m_prev; }
+
+	/// Accessor for the prev pointer.
+	/// \returns  The prev node in the list
+	inline const node_type * prev() const { return m_prev; }
+
+private:
+	friend class linked_list<T>;
+
+	/// Insert an item after this one in the list.
+	/// \arg item  The item to insert
+	void insert_after(node_type *item)
+	{
+		if (m_next) m_next->m_prev = item;
+		item->m_next = m_next;
+		item->m_prev = this;
+		m_next = item;
+	}
+
+	/// Insert an item before this one in the list.
+	/// \arg item  The item to insert
+	void insert_before(node_type *item)
+	{
+		if (m_prev) m_prev->m_next = item;
+		item->m_prev = m_prev;
+		item->m_next = this;
+		m_prev = item;
+	}
+
+	/// Remove this item from its list.
+	void remove()
+	{
+		if (m_next) m_next->m_prev = m_prev;
+		if (m_prev) m_prev->m_next = m_next;
+		m_next = m_prev = nullptr;
+	}
+
+	node_type *m_next;
+	node_type *m_prev;
+};
+
+
+/// An iterator for linked lists
+template <typename T>
+class list_iterator
+{
+public:
+	using item_type = list_node<T>;
+
+	list_iterator(item_type *item) : m_item(item) {}
+
+	inline item_type * operator*() { return m_item; }
+	inline const item_type * operator*() const { return m_item; }
+	inline list_iterator & operator++() { m_item = m_item ? m_item->next() : nullptr; return *this; }
+	inline list_iterator operator++(int) { return list_iterator<T>(m_item ? m_item->next() : nullptr); }
+	inline bool operator!=(const list_iterator<T> &other) { return m_item != other.m_item; }
+
+private:
+	item_type *m_item;
+};
+
+
+/// A templatized doubly-linked list container of `list_node<T>` items.
+template <typename T>
+class linked_list
+{
+public:
+	using item_type = list_node<T>;
+	using iterator = list_iterator<T>;
+
+	/// Constructor. Creates an empty list.
+	linked_list() :
+		m_head(nullptr),
+		m_tail(nullptr)
+	{}
+
+	/// Move constructor. Takes ownership of list elements.
+	linked_list(linked_list<T> &&other) :
+		m_head(other.m_head),
+		m_tail(other.m_tail)
+	{
+		other.m_head = other.m_tail = nullptr;
+	}
+
+	/// Check if the list is empty.
+	/// \returns  true if the list is empty
+	bool empty() const { return m_head == nullptr; }
+
+	/// Count the items in the list.
+	/// \returns  The number of entries in the list.
+	size_t length() const
+	{
+		size_t len = 0;
+		for (item_type *cur = m_head; cur; cur = cur->m_next) ++len;
+		return len;
+	}
+
+	/// Get the item at the front of the list, without removing it
+	/// \returns  The first item in the list
+	inline item_type * front() { return m_head; }
+
+	/// Get the item at the back of the list, without removing it
+	/// \returns  The last item in the list
+	inline item_type * back() { return m_tail; }
+
+	/// Prepend an item to the front of this list.
+	/// \arg item  The node to insert.
+	void push_front(item_type *item)
+	{
+		if (!item)
+			return;
+
+		if (!m_head) {
+			m_head = m_tail = item;
+			item->m_next = item->m_prev = nullptr;
+		} else {
+			m_head->m_prev = item;
+			item->m_next = m_head;
+			item->m_prev = nullptr;
+			m_head = item;
+		}
+	}
+
+	/// Append an item to the end of this list.
+	/// \arg item  The node to append.
+	void push_back(item_type *item)
+	{
+		if (!item)
+			return;
+
+		if (!m_tail) {
+			m_head = m_tail = item;
+			item->m_next = item->m_prev = nullptr;
+		} else {
+			m_tail->m_next = item;
+			item->m_prev = m_tail;
+			item->m_next = nullptr;
+			m_tail = item;
+		}
+	}
+
+	/// Remove an item from the front of this list.
+	/// \returns  The node that was removed
+	item_type * pop_front()
+	{
+		item_type *item = m_head;
+		if (m_head) {
+			m_head = item->m_next;
+			item->m_next = nullptr;
+		}
+		return item;
+	}
+
+	/// Remove an item from the end of this list.
+	/// \returns  The node that was removed
+	item_type * pop_back()
+	{
+		item_type *item = m_tail;
+		if (m_tail) {
+			m_tail = item->m_prev;
+			item->m_prev = nullptr;
+		}
+		return item;
+	}
+
+	/// Append the contents of another list to the end of this list. The other
+	/// list is emptied, and this list takes ownership of its items.
+	/// \arg list  The other list.
+	void append(linked_list<T> &list)
+	{
+		if (!list.m_head) return;
+
+		if (!m_tail) {
+			m_head = list.m_head;
+			m_tail = list.m_tail;
+		} else {
+			m_tail->m_next = list.m_head;
+			m_tail = list.m_tail;
+		}
+
+		list.m_head = list.m_tail = nullptr;
+	}
+
+	/// Append the contents of another list to the end of this list. The other
+	/// list is emptied, and this list takes ownership of its items.
+	/// \arg list  The other list.
+	void append(linked_list<T> &&list)
+	{
+		if (!list.m_head) return;
+
+		if (!m_tail) {
+			m_head = list.m_head;
+			m_tail = list.m_tail;
+		} else {
+			m_tail->m_next = list.m_head;
+			m_tail = list.m_tail;
+		}
+
+		list.m_head = list.m_tail = nullptr;
+	}
+
+	/// Remove an item from the list.
+	/// \arg item  The item to remove
+	void remove(item_type *item)
+	{
+		if (!item) return;
+		if (item == m_head)
+			m_head = item->m_next;
+		if (item == m_tail)
+			m_tail = item->m_prev;
+		item->remove();
+	}
+
+	/// Inserts an item into the list before another given item.
+	/// \arg existing The existing item to insert before
+	/// \arg item     The new item to insert
+	void insert_before(item_type *existing, item_type *item)
+	{
+		if (!item) return;
+
+		if (!existing)
+			push_back(item);
+		else if (existing == m_head)
+			push_front(item);
+		else
+			existing->insert_before(item);
+	}
+
+	/// Inserts an item into the list after another given item.
+	/// \arg existing The existing item to insert after
+	/// \arg item     The new item to insert
+	void insert_after(item_type *existing, item_type *item)
+	{
+		if (!item) return;
+
+		if (!existing)
+			push_front(item);
+		else if (existing == m_tail)
+			push_back(item);
+		else
+			existing->insert_after(item);
+	}
+
+	/// Insert an item into the list in a sorted position. Depends on T
+	/// having a method `int compare(const T *other)`.
+	/// \arg item  The item to insert
+	void sorted_insert(item_type *item)
+	{
+		if (!item) return;
+
+		item_type *cur = m_head;
+		while (cur && item->compare(cur) > 0)
+			cur = cur->m_next;
+
+		insert_before(cur, item);
+	}
+
+	/// Range-based for iterator generator.
+	/// \returns  An iterator to the beginning of the list
+	inline iterator begin() { return iterator(m_head); }
+
+	/// Range-based for iterator generator.
+	/// \returns  A const iterator to the beginning of the list
+	inline const iterator begin() const { return iterator(m_head); }
+
+	/// Range-based for end-iterator generator.
+	/// \returns  An iterator to the end of the list
+	inline const iterator end() const { return iterator(nullptr); }
+
+private:
+	item_type *m_head;
+	item_type *m_tail;
+};
+
+
+} // namespace kutil

src/libraries/kutil/include/kutil/memory.h

@@ -4,22 +4,17 @@
 #include <stdint.h>
 
 
-using addr_t = uint64_t;
-
 void * operator new (size_t, void *p) noexcept;
 
 
 namespace kutil {
 
-/// Allocate memory. Note: this needs to be implemented
-/// by the kernel, or other program using this library.
+/// Allocate memory.
 /// \arg n   The number of bytes to allocate
 /// \returns The allocated memory
 void * malloc(size_t n);
 
-/// Free memory allocated by malloc(). Note: this needs
-/// to be implemented by the kernel, or other program
-/// using this library.
+/// Free memory allocated by malloc().
 /// \arg p   A pointer previously returned by malloc()
 void free(void *p);
 
@@ -35,7 +30,7 @@ void * memset(void *p, uint8_t v, size_t n);
 /// \src     The memory to copy from
 /// \n       The number of bytes to copy
 /// \returns A pointer to the destination memory
-void * memcpy(void *dest, void *src, size_t n);
+void * memcpy(void *dest, const void *src, size_t n);
 
 /// Read a value of type T from a location in memory
 /// \arg p   The location in memory to read
@@ -53,7 +48,7 @@ inline T read_from(const void *p)
 template <typename T>
 inline T * offset_pointer(T *p, ptrdiff_t n)
 {
-	return reinterpret_cast<T *>(reinterpret_cast<addr_t>(p) + n);
+	return reinterpret_cast<T *>(reinterpret_cast<uintptr_t>(p) + n);
 }
 
 /// Return a pointer with the given bits masked out
@@ -61,9 +56,25 @@ inline T * offset_pointer(T *p, ptrdiff_t n)
 /// \arg mask A bitmask of bits to clear from p
 /// \returns  The masked pointer
 template <typename T>
-inline T* mask_pointer(T *p, addr_t mask)
+inline T* mask_pointer(T *p, uintptr_t mask)
 {
-	return reinterpret_cast<T *>(reinterpret_cast<addr_t>(p) & ~mask);
+	return reinterpret_cast<T *>(reinterpret_cast<uintptr_t>(p) & ~mask);
 }
 
+/// Do a simple byte-wise checksum of an area of memory.
+/// \arg p    The start of the memory region
+/// \arg len  The number of bytes in the region
+/// \arg off  An optional offset into the region
+uint8_t checksum(const void *p, size_t len, size_t off = 0);
+
+
+class memory_manager;
+
+namespace setup {
+
+/// Set the heap that malloc() / free() will use.
+/// \arg mm  The heap manager for the heap to use.
+void set_heap(memory_manager *mm);
+
+} // namespace kutil::setup
 } // namespace kutil