Implementation Plan

Implementation Plan

Purpose

This page is the implementation roadmap for building XAI OS from an empty source tree to a developer-preview CPU-only AI operating system.

Contents

• Global Rules
• Phase 0: Repository and Build Foundation
• Phase 1: QEMU Boot
• Phase 2: Kernel Core
• Phase 3: Memory Management
• Phase 4: Exceptions, Interrupts, and Timers
• Phase 5: SMP and Per-Core State
• Phase 6: VirtIO Block and Network
• Phase 7: Minimal Userspace and Control Plane
• Phase 8: AI Cell Runtime
• Phase 9: Memory Arenas and Shared Weights
• Phase 10: Low-Latency TCP and UDP
• Phase 11: CPU AI Runtime
• Phase 12: App-Agent Source, Git, and Build Workflow
• Phase 13: Intel Desktop Port
• Phase 14: Intel Xeon Port
• Phase 15: ARM/NVIDIA Port
• Phase 16: Security and Release Hardening

Global Rules

• One deliverable per commit or pull request.
• Kernel code is freestanding C99 plus minimal architecture assembly.
• C++ is allowed in userspace, runtime, and AI components, not early kernel core.
• No libc assumptions inside the kernel.
• No heap allocation before kernel heap initialization.
• No dynamic allocation in scheduler, networking, AI Cell, or memory hot paths after service warmup.
• No migration unless a service explicitly opts into it.
• No hidden background work on leased AI cores.
• No post-READY page faults on hot AI cores by default.
• Every architecture feature needs a generic fallback.
• Every hot-path structure documents cache-line ownership.
• Performance claims remain design targets until benchmarked against a tuned Linux/BSD baseline.

Phase 0: Repository and Build Foundation

Goal: create a clean source tree and reproducible macOS-hosted build skeleton.

Deliverables: Makefile, scripts/, target source directories, ignored build output directories, host bootstrap checks.

Files/directories: Makefile, scripts/macos-bootstrap.sh, scripts/build-image.sh, scripts/run-qemu-x86_64.sh, scripts/test.sh, boot/, kernel/, userspace/, runtime/, ai/, bench/, tests/.

Implementation requirements: keep generated files under build/, out/, or dist/; fail with actionable messages when QEMU, LLVM, LLD, Python, Git, image tools, or OVMF are missing.

Tests: make all, make image, make test.

Definition of done: all commands run and no generated artifacts are tracked.

Risks: host dependency drift on macOS.

Do not: add kernel logic before the build and QEMU scripts are repeatable.

Phase 1: QEMU Boot

Goal: boot a UEFI loader in QEMU on macOS and reach kernel.elf.

Deliverables: UEFI loader skeleton, disk image generation, serial boot log, kernel handoff structure.

Files/directories: boot/uefi/, boot/uefi/boot_info.h, kernel/core/kmain.c, scripts/build-image.sh, scripts/run-qemu-x86_64.sh.

Implementation requirements: use UEFI Boot Services only before ExitBootServices; validate ELF64; gather memory map and ACPI RSDP; print deterministic serial logs.

Tests: QEMU serial boot, missing kernel error path, invalid ELF error path.

Definition of done: boot log shows loader start and kernel entry.

Risks: hardcoded QEMU memory assumptions.

Do not: use GRUB, BIOS boot, or hidden host-specific paths.

Phase 2: Kernel Core

Goal: establish kernel entry, logging, panic, assertions, and status/types.

Deliverables: freestanding kernel entry, serial logger, panic path, assertion helper.

Files/directories: kernel/core/, kernel/include/xaios/.

Implementation requirements: no libc; no heap; no red zone; panic includes file, line, and message.

Tests: boot log, panic smoke test, assertion smoke test.

Definition of done: controlled panic prints useful diagnostics.

Risks: accidental host libc dependency.

Do not: introduce userspace abstractions into kernel core.

Phase 3: Memory Management

Goal: implement PMM, VMM, kernel heap, hugepage metadata, and arena foundations.

Deliverables: page allocator, page tables, mapping API, control-plane heap, model arena metadata.

Files/directories: kernel/mm/, kernel/arch/aarch64/, kernel/include/xaios/.

Implementation requirements: reserve kernel, boot info, ACPI, MMIO, framebuffer, and loader regions; map kernel text RX, rodata RO, data RW; support 4 KiB pages first and hugepage metadata later.

Tests: allocate/free page test, translation checks for RX text, RO rodata, RW data, and MMIO metadata. Read-only mapping fault tests follow once exception diagnostics exist.

Definition of done: memory totals print and reserved regions are never allocated.

Risks: corrupting boot memory map ownership.

Do not: enable AI hot paths before post-READY fault counting exists.

Phase 4: Exceptions, Interrupts, and Timers

Goal: handle CPU exceptions, local APIC/x2APIC, and monotonic time.

Deliverables: GDT, IDT, trap handlers, page-fault diagnostics, monotonic clock.

Files/directories: kernel/arch/aarch64/, kernel/irq/.

Implementation requirements: print fault address and error code; fatal unknown exceptions panic; avoid a periodic scheduler tick design.

Tests: controlled page fault, generic counter monotonicity, GIC discovery.

Definition of done: controlled faults are diagnosed and the kernel remains debuggable.

Risks: timer design leaking into future hot AI cores.

Do not: add a generic tick scheduler.

Phase 5: SMP and Per-Core State

Goal: bring up multiple CPUs and enforce the first no-migration core lease model.

Deliverables: CPU registry, per-core data, MPIDR logging, core lease API, AI hot-core role state, lease owner metadata, IRQ-routing-away metadata, tick-suppression metadata, migration counters, involuntary context-switch counters.

Files/directories: kernel/arch/aarch64/smp.c, kernel/core/cpu.c, kernel/core/per_cpu.c, kernel/sched/core_lease.c.

Implementation requirements: CPU 0 is housekeeping and cannot be leased; PSCI starts non-boot CPUs idle reserved; a valid lease marks a non-boot CPU as an AI hot core; duplicate leases are rejected; releasing a lease returns the CPU to reserved-idle state; hot-core telemetry must report zero migration and zero involuntary context switches until a real scheduler exists.

Tests: QEMU four-CPU boot, MPIDR log, lease CPU 1 as an AI hot core, reject duplicate lease, reject CPU 0 lease, release CPU 1, emit telemetry with zero migration and context-switch totals.

Definition of done: the smoke test proves lease state transitions and zero migration/context-switch counters for the QEMU AI hot-core path.

Risks: accidental load balancing.

Do not: introduce global scheduling for hot workers.

Phase 6: VirtIO Block and Network

Goal: support QEMU storage and basic packet exchange.

Deliverables: VirtIO-MMIO discovery, VirtIO feature negotiation, VirtIO block reads, VirtIO net RX/TX, Ethernet/ARP smoke path.

Files/directories: kernel/dev/acpi/, kernel/dev/pci/, kernel/dev/virtio/, kernel/net/, kernel/fs/.

Implementation requirements: fail closed on unsupported VirtIO features; validate packet lengths; add PCI config parsing later.

Tests: read known block, exchange ARP packet, mount the versioned read-only filesystem from VirtIO block, parse the config manifest, drop malformed frames.

Definition of done: QEMU guest can read a test block and exchange a test packet.

Risks: driver scope explosion.

Do not: start physical NIC work before VirtIO tests pass.

Phase 7: Minimal Userspace and Control Plane

Goal: run minimal init and expose a basic admin/control path.

Deliverables: service lifecycle, VirtIO-backed versioned read-only filesystem, parsed config manifest, real EL0 /init ELF loading, explicit syscall table, process/capability metadata, user pointer validation, bad syscall tests, userspace service-manager policy, /init, osctl skeleton.

Files/directories: kernel/user/, userspace/init/, userspace/osctl/, runtime/include/xaios/, runtime/src/.

Implementation requirements: service states, invalid command handling, EL0 syscall entry, user/kernel permission split, user-accessible ELF text/rodata/stack mappings, guarded user stack pages, and bad syscall handling.

Tests: /init lifecycle logs and exits through the supervisor from EL0; EL0 /init logs, configures restart/log policy, records service-manager logs, issues osctl, attempts a forbidden restart, and exits through SVC; unknown syscalls and invalid user pointers return errors; bad command does not crash kernel; process counters report rejected syscalls.

Definition of done: disk-loaded EL0 /init starts, controls service state, and exits cleanly in QEMU.

Risks: growing POSIX scope too early.

Do not: port broad libc before kernel/user ABI is stable.

Phase 8: AI Cell Runtime

Goal: implement AI Cell manifests, resource validation, lifecycle, telemetry, and service supervision.

Deliverables: manifest validation, core lease integration, memory arena reservation, NIC queue/workspace binding, cell state machine, counters.

Files/directories: kernel/runtime/, kernel/sched/, userspace/services/, runtime/include/xaios/cell.h.

Implementation requirements: invalid resource requests fail; conflicts fail; lifecycle transitions are logged; KV/cache and source-index arenas reserve PMM pages and VMM mappings.

Tests: valid cell starts/stops; invalid manifest rejected; missing model arena rejected; invalid workspace rejected; overlapping core lease rejected; duplicate NIC queue rejected; migration counter remains zero.

Definition of done: AI Cell create/destroy works in QEMU.

Risks: granting undeclared resources.

Do not: allow migration by default.

Phase 9: Memory Arenas and Shared Weights

Goal: implement model arena, KV/cache arena, source index arena, and shared read-only weights.

Deliverables: generic arena APIs, read-only shared model mapping, private KV/cache mapping, source-index arenas, build-output arenas, log arenas, prefault support, arena counters.

Files/directories: kernel/mm/arena.c, kernel/runtime/model_arena.c, kernel/runtime/ai_cell.c, kernel/include/xaios/arena.h, runtime/include/xaios/mem.h, ai/runtime/.

Implementation requirements: model weights are read-only and shared; KV/cache is private; source indexes, build outputs, logs, and telemetry have explicit arena kinds; no swap; post-READY faults counted.

Tests: arena create/acquire/release/destroy; two cells share one model mapping; AI Cell prepare creates private KV/cache, source-index, build-output, and log arenas; write to shared weights faults safely.

Definition of done: generic arena manager and shared/private AI Cell arena paths pass QEMU smoke.

Risks: hidden duplicate model copies.

Do not: silently allocate private weight copies.

Phase 10: Low-Latency TCP and UDP

Goal: build latency-first TCP/UDP paths for embedded app agent command traffic.

Deliverables: per-core queue ownership, flow steering, UDP MVP, TCP design skeleton, latency counters.

Files/directories: kernel/net/, runtime/include/xaios/net.h, bench/tcp_latency/, bench/udp_latency/.

Implementation requirements: owner core per flow; polling active, interrupt fallback idle; copy-small/zero-copy-large policy.

Tests: UDP echo latency, TCP connect/request smoke, queue ownership telemetry.

Definition of done: p50/p95/p99/p999 counters are emitted.

Risks: small gains versus kernel-bypass Linux.

Do not: claim performance without tuned Linux/BSD baseline.

Phase 11: CPU AI Runtime

Goal: add CPU-only model runtime skeleton and dispatch boundary.

Deliverables: model loader, tokenizer boundary, scalar kernels, CPU feature dispatch, benchmark harness.

Files/directories: ai/runtime/, ai/tokenizer/, ai/kernels/scalar/, ai/kernels/x86_64/, ai/kernels/aarch64/.

Implementation requirements: no CUDA/Metal/GPU dependency; scalar fallback always works; feature paths are detected.

Tests: load small test model artifact or fixture, run deterministic decode stub, verify shared weights/private KV.

Definition of done: CPU-only runtime skeleton runs under an AI Cell.

Risks: model-format scope creep.

Do not: depend on a vendor accelerator.

Phase 12: App-Agent Source, Git, and Build Workflow

Goal: support source indexing, patch generation boundary, build/test sandbox, Git worktree, and rollback.

Deliverables: source index service skeleton, Git worktree agent skeleton, build/test sandbox metadata, build artifact arena wiring, rollback metadata, build/test cell integration, audit log.

Files/directories: ai/source_index/, ai/git_agent/, userspace/services/, tests/integration/.

Implementation requirements: review-before-apply default; tests before commit/push; push permission separate; sandbox manifests validate repo/worktree/build/artifact paths; sandbox network access is denied by default; rollback restores the last good source revision on failed deploy.

Tests: sandbox manifest validates and rejects invalid paths/network access; build artifact arena is reserved; build lifecycle emits start/finish markers; rollback returns to the previous revision; patch fixture applies in worktree, build/test command runs, failed test blocks commit.

Definition of done: app-agent workflow can propose and reject/apply a safe fixture patch.

Risks: unsafe code modification.

Do not: give agents unrestricted repo or network access.

Phase 13: Intel Desktop Port

Goal: validate the first real performance target on Intel Desktop.

Deliverables: P-core/E-core policy, SMT policy, NVMe/NIC support, benchmark gates.

Files/directories: kernel/arch/x86_64/, kernel/dev/nvme/, kernel/dev/net/, bench/.

Implementation requirements: hot AI workers on P-cores; background work on E-cores where available; no post-READY faults.

Tests: migration, page fault, TCP/UDP latency, bandwidth, sustained core benchmark.

Definition of done: JSON benchmark artifacts compare against tuned Linux/BSD.

Risks: limited memory channels hide bandwidth gains.

Do not: generalize desktop results to Xeon or ARM/NVIDIA.

Phase 14: Intel Xeon Port

Goal: support serious multi-agent NUMA server deployment.

Deliverables: NUMA memory policy, server NIC queues, multi-queue NVMe, IOMMU, many AI Cells, admission control.

Files/directories: kernel/arch/x86_64/numa.c, kernel/dev/, kernel/mm/numa.c, bench/.

Implementation requirements: keep latency-sensitive cell in one NUMA node unless configured; queue locality visible.

Tests: multi-cell benchmark, NUMA locality checks, queue locality checks.

Definition of done: many AI Cells run predictably under measured resource contracts.

Risks: tuned Linux NUMA baseline may already be strong.

Do not: allow accidental cross-node memory.

Phase 15: ARM/NVIDIA Port

Goal: port XAI OS to ARM/NVIDIA N1X-compatible AArch64 systems.

Deliverables: AArch64 boot, page tables, exceptions, GIC, SMMU, cluster policy, platform telemetry.

Files/directories: kernel/arch/aarch64/, kernel/dev/gic/, kernel/dev/smmu/, scripts/run-qemu-aarch64.sh.

Implementation requirements: generic AArch64 before NVIDIA-specific optimization; no CUDA/GPU dependency.

Tests: boot, interrupt routing, SMMU protection, cluster placement, memory benchmark.

Definition of done: AI Cell lifecycle runs on target or emulator with platform counters.

Risks: firmware opacity and missing vendor documentation.

Do not: block generic AArch64 work on board-specific features.

Phase 16: Security and Release Hardening

Goal: prepare a developer preview with safe defaults.

Deliverables: signed update path, rollback, public-key SSH, secret scanning, audit logs, CI smoke boots, release notes.

Files/directories: userspace/sshd/, userspace/osctl/, scripts/, .github/workflows/, tests/.

Implementation requirements: no password login by default; capability revocation; no secrets in logs or benchmarks.

Tests: security policy tests, rollback test, CI QEMU boot, benchmark JSON validation.

Definition of done: external developer can boot XAI OS, run sample CPU-only embedded app agent workflow, and compare basic counters.

Risks: release before safety model is enforceable.

Do not: publish production-ready claims.

Related Pages

• Codex Work Packages
• Build System
• Testing and Benchmarking
• Open Decisions
• Risk Register