rt/kernel

A preemptive, priority-based real-time microkernel for the Raspberry Pi 4B (BCM2711, Cortex-A72, AArch64), plus a userspace built on top of it. That userspace includes a model-railway (Märklin) train-control application. Everything above the kernel runs as ordinary unprivileged (EL0) tasks that talk to each other over message-passing IPC. That includes the device drivers and the C/C++ runtime.

Above shows our Lua shell running inside of the marklinseq program.
Feel free to run the kernel and try out the shell!

The project descends from the CS452 real-time kernel assignments (k1–k4), and is a sister design to my personal cxkernel. It has since grown a real virtual-memory subsystem, an ELF loader, a name server, a POSIX-ish service layer, and a type-safe RPC framework.

Highlights

The kernel is a true microkernel. It handles tasks, scheduling, IPC, interrupt dispatch, and virtual memory, and nothing more:

• Fixed-priority preemptive scheduling. 32 priority levels, round-robin within a level, a 64-task pool, and a 10 ms tick. The highest-priority runnable task always runs, so worst-case latency is bounded by short kernel primitives rather than unbounded kernel work.
• Real virtual memory. Per-task private AArch64 4-level page tables, a page-frame database, a slab allocator, a recursive self-map, lazy-committed stacks and heaps, and shared VMOs for the initrd and device MMIO. Each task gets a fully isolated low-half (TTBR0) address space; the kernel lives in the shared high half (TTBR1).
• Send–Receive–Reply IPC and an AwaitEvent interrupt model that keeps all device policy in userspace. Drivers are just tasks that block on an IRQ.

The userspace runs a full modern C++ stack on bare metal, with no host OS underneath it:

• C++ ported end to end. libc++/libc++abi (STL), libunwind, and a freestanding libc, all statically linked into every program.
• C++ coroutines, re-enabled in libc++. The train controller is written with them.
• Exceptions and RTTI. Real throw/catch over .eh_frame and a custom personality routine.
• Symbolized crash stack traces, resolved entirely on-device by walking the frame-pointer chain and demangling against each task's own ELF symbol table.
• A Lua-driven init, a shell, pseudo-terminals, and interrupt-driven serial I/O servers, all as ordinary tasks over the RPC framework.

Documentation

⚠️ Disclosure ⚠️ Parts of the documentation are generated with Claude. They have all been independently reviewed and edited by me for accuracy. The most detailed documentation is reading the code.

Document	What's in it
docs/kernel.md	Kernel design. Boot flow, tasks/threads, scheduling and real-time characteristics, syscalls, SRR IPC, events, and virtual memory (with the full memory map).
docs/boot.md	Boot and image loading. The hand-rolled first-stage loader, the kernel8.img layout, how the image and TAR initrd are assembled, ELF loading and initial page tables, and the LoaderData handoff to the kernel.
docs/runtime-abi.md	Userspace runtime and library ports. The per-program startup harness, the ABI shims, stdio-as-RPC, the static/non-PIE linking model, crash traces, and the mimalloc / libc / libc++ / libunwind / Lua ports.
docs/writing-a-service.md	Writing a system service. The type-safe RPC framework, the name server, deferred replies, and how to register and launch a service.
docs/writing-a-program.md	Writing a user program. The available libraries and runtime, the add_user_program build factory, spawning child tasks, and a worked example.
CONTRIBUTING.md	Linux/WSL setup and the VS Code workflow.

---

Building and running on macOS

1. Install the toolchain

All dependencies come from Homebrew:

brew install llvm@17 cmake qemu

Toolchain version. The build requires LLVM/Clang 17. CMake validates the compiler, assembler, and objcopy against version 17 at configure time. Homebrew installs it keg-only under /opt/homebrew/opt/llvm@17, which is exactly what the configure command below points at. QEMU must be >= 9 for Raspberry Pi 4B (raspi4b) machine support.

2. Configure the project

Run CMake from the repository root, pointing it at the Homebrew LLVM 17 tools. This is the "configure" step: it generates the build system into ./build.

rm -rf build && cmake \
   -DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm@17/bin/clang \
   -DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm@17/bin/clang++ \
   -DCMAKE_ASM_COMPILER=/opt/homebrew/opt/llvm@17/bin/llvm-mc \
   -DMY_OBJCOPY_EXEC_PATH=/opt/homebrew/opt/llvm@17/bin/llvm-objcopy \
   -DENABLE_QEMU=ON \
   . -B build

-DENABLE_QEMU=ON disables the UART3 hardware CTS/RTS flow-control state machine, which has no counterpart under emulation. Leave it off (the default) when building for real hardware.

Useful configure-time options (all optional, see CMakeLists.txt):

Option	Default	Meaning
ENABLE_QEMU	OFF	Build for QEMU instead of bare metal
ENABLE_DEBUG	OFF	Emit DWARF debug symbols; leaves NDEBUG undefined
ENABLE_LTO	ON	Link-time optimization across all modules
ENABLE_KERNEL_LOGS	ON	Compile in kernel log statements
ENABLE_KERNEL_ASSERTS	ON	Compile in kernel assertions
KOPTLEVEL / UOPTLEVEL	-O3	Kernel / userspace optimization level
DEFAULT_BOOTARGS	init=sbin/initd logmask=7	Boot arguments baked into the loader

3. Build

cmake --build build -j8

The final artifact is build/kernel8.img: the raw loader binary concatenated with a TAR-format initial ramdisk (initrd). The initrd holds the kernel ELF plus every userspace program and the etc/init.lua startup script.

4. Run under QEMU

The launcher script wires up the raspi4b machine, the device tree, and serial I/O. It resolves the image relative to its own location but reads the device tree from the current directory, so run it from scripts/:

cd scripts
./launch_qemu.sh

You should see the kernel boot log on UART0, followed by the userspace sh shell prompt. UART0 output is also teed to build/UART0.log.

To debug with LLDB/GDB, launch QEMU stopped and waiting for a debugger on TCP port 1234:

cd scripts
./launch_qemu.sh -debug

then attach (target remote localhost:1234) and load symbols from build/rtkernel.elf.

Repository layout

loader/       First-stage loader: DTB parsing, boot args, initial page tables
kernel/
  core/       Threads, scheduler, syscalls, IPC, events, VM, slab allocator
  platform/   AArch64/BCM2711 specifics: vectors, exceptions, IRQ, UART, MMU
  include/    Kernel headers (core/ and platform/)
lib/
  user/       Userspace runtime: syscall wrappers, RPC, services, crt0, threads
  libkernel/  Shared kernel-side helpers: logging, asserts, perf counters
  cxx/        Freestanding C++ support (string, format, containers, units)
  elf/ tar/   ELF and TAR (initrd) parsers
  third-party/ Ported libc (newlib), libc++, libunwind, mimalloc, Lua
               (see docs/runtime-abi.md for the port details)
userspace/
  services/   initd, clockd, ttyS, ptyd, marklind, idle, ...
  tools/      sh, dmesg, marklinctl, marklinseq, tmux-lite
  tc/         Train-control application and UI
  assignments/ Original k1-k4 milestone programs
scripts/      launch_qemu.sh, toolchain/options CMake, initrd builder, DTB

See CONTRIBUTING.md for the Linux/WSL setup and VS Code workflow.