Skip to content

QEMU Full OS Core Workdown

Jump to bottom
Node1 edited this page Jun 18, 2026 · 20 revisions

QEMU Full OS Core Workdown

Purpose

This page tracks the work required to move XAI OS from a QEMU readiness gate to a real QEMU full OS core.

The QEMU hardware-readiness gate is useful, but it is not the same as a complete OS. Intel hardware work should not start until the QEMU core has stable process, userspace control-plane, persistence, networking, CPU-AI runtime, security, telemetry, and filesystem contracts.

Contents

Completion Rule

Do not start Intel hardware code until this QEMU workdown has a release candidate gate with stable ABI, telemetry, filesystem, process, security, and control-plane contracts.

Phase Q1: Real Process Lifecycle

Goal: stop treating /init as a one-way kernel-hosted test and make EL0 process execution return cleanly to kernel supervision.

Current status:

  • Done: process states: empty, loaded, running, exited, failed.
  • Done: process lifecycle transition counters.
  • Done: sys_exit marks the current process exited or failed.
  • Done: EL0 exit returns to a kernel continuation.
  • Done: process lifecycle telemetry.
  • Done: child service descriptor parsing from /etc/xaios-init.conf.
  • Done: /init defines, starts, and reports the child /svc/source-index service record through osctl.
  • Done: invalid-transition and failed-exit lifecycle tests.

Definition of done: QEMU boot proves /init loads, runs, exits, and the kernel resumes after EL0. Telemetry includes loaded, running, exited, failed, transition counters, child service descriptor count, and service transition count.

Verification:

  • make qemu-smoke
  • make qemu-readiness-gate

Phase Q2: Real Userspace Control Plane

Goal: move policy decisions out of kernel fixtures and into userland service management.

Required work:

  • Done: split userland into /init and /bin/service-manager payloads.
  • Done: add /etc/services/source-index.svc as the first read-only service descriptor.
  • Done: add descriptor-read and service status/start/stop/restart/rollback/update syscalls.
  • Done: keep service operations behind capability checks.
  • Done: prove missing rollback/update capabilities fail closed.
  • Done: emit control-plane syscall, denial, and descriptor-read telemetry.

Definition of done: QEMU boot proves /bin/service-manager runs as a second EL0 process, reads the ROFS service descriptor, manages /svc/source-index, and reports control-plane telemetry.

Verification:

  • make qemu-smoke
  • make qemu-readiness-gate

Phase Q3: Real Filesystem and Persistence

Goal: replace metadata-only persistence with serialized state records and mutable VirtIO-backed state.

Required work:

  • Done: mutable state region on the VirtIO test block image.
  • Done: serialized boot, service, workspace, sandbox, and update records.
  • Done: checksum and version validation for persistence records.
  • Done: persisted records reload after a QEMU reboot.
  • Done: rollback restores prior service and workspace records.
  • Done: read-only system image and mutable state boundaries are explicit.

Definition of done: make qemu-persistence-reboot proves the same VirtIO state image survives a two-boot QEMU run, reports persistence_boot_loads, and validates rollback after reload.

Verification:

  • make qemu-smoke
  • make qemu-persistence-reboot
  • make qemu-readiness-gate

Phase Q4: Networking Depth

Goal: move beyond parser smoke paths into queue-backed packet flow.

Required work:

  • Done: queue-backed RX/TX packet ownership.
  • Done: UDP flow table with queue/core ownership.
  • Done: minimal TCP state machine with SYN timeout accounting.
  • Done: packet lifetime/drop counters.
  • Done: malformed packet tests remain enforced.
  • Done: p50/p95/p99/p999 correctness telemetry.

Definition of done: make qemu-smoke proves packets move through queue metadata, creates a UDP flow, expires a TCP SYN flow, and reports packet lifecycle/drop counters plus p999 latency telemetry.

Verification:

  • make qemu-smoke
  • make qemu-readiness-gate

Phase Q5: CPU-Only AI Runtime Depth

Goal: turn the deterministic CPU-AI stub into real boundaries that can later host actual CPU-only model code.

Required work:

  • Done: model-loader interface and manifest format.
  • Done: tokenizer/runtime boundary.
  • Done: shared read-only model weights.
  • Done: private KV/cache arenas per AI Cell.
  • Done: multi-cell shared-weight tests.
  • Done: model load failure tests.
  • Done: GPU acceleration remains outside the runtime contract.

Definition of done: make qemu-smoke requires model load, load failure, tokenizer, runtime, KV/cache write, shared-weight bind, and GPU rejection telemetry.

Verification:

  • make qemu-smoke
  • make qemu-readiness-gate

Phase Q6: Security Enforcement

Goal: make capability policy enforce service, filesystem, Git/workspace, update, rollback, and build/test permissions.

Required work:

  • Complete: extend capability checks across service operations.
  • Complete: enforce filesystem read/write boundaries.
  • Complete: enforce Git workspace write and patch permissions.
  • Complete: enforce build/test sandbox permissions.
  • Complete: strengthen signed update validation beyond the current stub.
  • Complete: add rollback authorization tests.
  • Complete: ensure secrets are rejected from logs, updates, and benchmark records.

Definition of done: denied operations fail closed and emit telemetry counters for capability, filesystem, workspace, sandbox, rollback, update-policy, credential, and signature rejection paths.

Verification:

  • make qemu-smoke
  • make qemu-readiness-gate

Phase Q7: QEMU Release Candidate

Goal: freeze the QEMU full OS core contract.

Required work:

  • Complete: expand make qemu-readiness-gate to include CPU matrix tiers.
  • Complete: freeze syscall ABI.
  • Complete: freeze telemetry schema.
  • Complete: freeze filesystem format.
  • Complete: freeze persistence record format.
  • Complete: freeze service descriptor format.
  • Complete: document what remains out of scope for Intel bring-up.

Definition of done: a single QEMU release-candidate gate proves process, userspace control, persistence, networking, CPU-AI runtime, security, CPU matrix, and documentation contracts are stable enough to start Intel Desktop work.

Release-candidate contract:

  • contracts/qemu-rc-v1.json
  • schema: xaios.qemu.release_candidate_contract.v1
  • generated CPU report: build/qemu-cpu-matrix-report.json

Phase Q8: Full Filesystem

Goal: move beyond the read-only boot filesystem and single-sector persistence records toward a real mutable filesystem area.

Required work:

  • Complete: reserve a separate VirtIO-backed mutable filesystem sector range.
  • Complete: add checksum-protected filesystem metadata.
  • Complete: add explicit read-write mount policy.
  • Complete: add create/update/read/delete file operations.
  • Complete: add commit and rollback boundaries for mutable files.
  • Complete: add QEMU smoke and benchmark gates for mutable filesystem telemetry.
  • Complete: add a real block allocator instead of fixed one-sector file slots.
  • Complete: add directory records and path traversal beyond fixed prefixes.
  • Complete: add larger files spanning multiple sectors.
  • Complete: add crash-consistency transaction replay.
  • Complete: connect service, workspace, and update state to the mutable filesystem.

Definition of done: QEMU proves mutable filesystem mount, mutation, checksum validation, allocation, directory records, multi-sector files, journal replay, delete behavior, reboot load, and rollback behavior without weakening the read-only boot filesystem boundary. A later production-ready filesystem can add a POSIX-like surface and larger storage area, but the QEMU contract no longer depends on fixed one-sector file slots.

Phase Q9: Process Supervisor

Goal: make the service manager behave like a small supervisor instead of a single child-service smoke test.

Required work:

  • Complete: track parent-child service tree edges.
  • Complete: store child parent metadata in the service record.
  • Complete: allow service configuration to apply to child services, not only /init.
  • Complete: enforce child restart policy and max_restarts.
  • Complete: add a controlled child crash path through the userspace service manager.
  • Complete: restart a failed child when policy allows it.
  • Complete: clean up service runtime state after exit or crash.
  • Complete: count service logs globally and per service.
  • Complete: count process address-space reclamation after EL0 process exit.
  • Complete: add smoke, benchmark, readiness, and contract gates for supervisor telemetry.

Definition of done: /bin/service-manager defines /svc/source-index as a child of /init, starts it, proves restart denial while policy is never, reconfigures it to always, logs a service event, injects a controlled crash, and observes the supervisor cleanup/restart path. Telemetry includes service_tree_edges, service_restarts, service_crashes, service_cleanups, service_log_records, and user_process_reclaims.

Phase Q10: Network Stack Maturity

Goal: turn the QEMU network path from a queue-backed smoke fixture into a stricter TCP/UDP lifecycle contract.

Required work:

  • Complete: route packets through deterministic flow-to-queue selection.
  • Complete: keep established UDP flows pinned to their owning queue and cell.
  • Complete: add explicit RX/TX/completion accounting for queue rings.
  • Complete: add UDP flow hit tracking.
  • Complete: add UDP idle expiry.
  • Complete: add TCP established/closed counters.
  • Complete: add TCP retransmit-before-timeout policy for SYN-received flows.
  • Complete: keep TCP timeout behavior after retransmit.
  • Complete: add telemetry for queue backpressure drops and flow/core mismatches.
  • Complete: expand smoke, benchmark, readiness, and RC contract gates.

Definition of done: QEMU boot proves UDP flow reuse, UDP expiry, TCP establishment, TCP retransmit, TCP timeout, queue RX/TX/completion accounting, zero queue backpressure drops, and zero flow/core mismatches. make qemu-smoke, python3 ./scripts/qemu-benchmark.py, and make qemu-readiness-gate require the mature networking telemetry.

Phase Q11: CPU-Only Model Runtime

Goal: turn the earlier CPU-AI runtime boundary into a file-backed QEMU model runtime contract.

Required work:

  • Complete: add /models/cpu-ai-mvp.xaiosmodel to the VirtIO-backed read-only filesystem.
  • Complete: increase the ROFS header to 1024 bytes so the filesystem can carry the model file cleanly.
  • Complete: validate model magic, version, quantization, flags, ranges, tokenizer size, private KV/cache requirement, and payload checksum.
  • Complete: reject missing model files, checksum failures, undersized tokenizer tables, GPU-required models, and undersized KV/cache bindings.
  • Complete: bind a tokenizer table from the model file instead of hardcoded token ids.
  • Complete: dispatch deterministic CPU decode through a runtime id and count kernel dispatches.
  • Complete: keep shared model weights read-only and private KV/cache per AI Cell.
  • Complete: expand smoke, benchmark, readiness, telemetry, and RC contract gates.

Definition of done: QEMU boot proves a CPU-only model file is loaded from ROFS, admitted into a shared read-only model arena, used by multiple AI Cells with private KV/cache arenas, and rejected correctly for malformed or GPU-required model configurations. Telemetry includes model file loads/rejects, loaded bytes, manifest validations, tokenizer binds, CPU kernel dispatches, admission rejects, and checksum failures.

Phase Q12: AI Cell Production Contract

Goal: turn AI Cell setup from fixture-style manifests into a stable resource contract that later platform ports can preserve.

Required work:

  • Complete: define the AIC1 version 1 descriptor ABI with fixed 112-byte layout.
  • Complete: require CPU-only AI, fixed core, shared-model, private KV/cache, NIC queue, and Git workspace flags.
  • Complete: validate descriptor magic, version, size, reserved fields, bounded names, and checksum.
  • Complete: admit descriptors through the existing AI Cell manifest validation path.
  • Complete: account PMM/VMM-backed arena reservations in current and peak page/byte counters.
  • Complete: bind AI Cells to real network queue ownership and release those queues during destroy.
  • Complete: track workspace bind and release lifecycle.
  • Complete: reject duplicate core leases, missing model arenas, duplicate NIC queues, and duplicate workspaces.
  • Complete: keep shared model weights reusable while private KV/cache remains per AI Cell.
  • Complete: expand smoke, benchmark, readiness, telemetry, and RC contract gates for descriptor and resource-contract counters.

Definition of done: QEMU boot proves valid AI Cell descriptors are accepted, malformed descriptors are rejected, resource conflicts fail closed, arenas are reserved and released, queue/workspace ownership is balanced, and the final telemetry reports zero currently reserved AI Cell arena pages with nonzero peak usage. make qemu-smoke, python3 ./scripts/qemu-benchmark.py, and make qemu-readiness-gate require the AI Cell contract telemetry.

Phase Q13: Security Hardening

Goal: make the QEMU security policy explicit enough that later update, admin, and hardware-port work cannot bypass it accidentally.

Required work:

  • Complete: add XAIOS_CAP_ADMIN as a separate administrative capability.
  • Complete: require both XAIOS_CAP_UPDATE and XAIOS_CAP_ADMIN for signed update authorization.
  • Complete: replace the earlier update-token stub with a structured development public-key signature format.
  • Complete: require strictly monotonic update generations and reject replayed updates.
  • Complete: track update key accepts and rejects.
  • Complete: reject sandbox path traversal and duplicate slash escape attempts.
  • Complete: preserve rollback authorization fail-closed behavior.
  • Complete: emit telemetry for admin denials, update authorization, replay rejection, key accept/reject, and sandbox escape rejection.
  • Complete: expand smoke, benchmark, readiness, telemetry, and RC contract gates for the hardened policy.

Definition of done: QEMU boot proves missing admin capability, bad update key, replayed update generation, sandbox path escape, credential material, rollback denial, and signed update authorization paths are all exercised and counted. make qemu-smoke, python3 ./scripts/qemu-benchmark.py, and make qemu-readiness-gate require the security-hardening telemetry.

Phase Q14: Update and Rollback System

Goal: make signed updates a transaction lifecycle with rollback points, boot fallback, failed-update recovery, and persisted update records.

Required work:

  • Complete: add a kernel update transaction runtime.
  • Complete: persist update transaction records into /state/updates/update.state.
  • Complete: create update rollback points through the persistence layer.
  • Complete: stage signed update transactions after update/admin authorization.
  • Complete: commit staged update transactions.
  • Complete: mark failed staged updates and recover them through boot fallback.
  • Complete: roll back committed updates through an authorized rollback point.
  • Complete: emit telemetry for transactions, stages, commits, failures, recoveries, rollbacks, boot fallback, persisted records, rollback points, and rejects.
  • Complete: expand smoke, benchmark, readiness, telemetry, and RC contract gates for the update rollback system.

Definition of done: QEMU boot proves one staged update fails and recovers through boot fallback, and another staged update commits and rolls back through the same rollback path. Update records include generation, state, target, and rollback metadata. make qemu-smoke, python3 ./scripts/qemu-benchmark.py, and make qemu-readiness-gate require the update rollback telemetry.

Phase Q15: Admin Control Plane

Goal: make the QEMU admin surface explicit and testable before the final full OS release-candidate gate.

Required work:

  • Complete: grant admin capability only to the EL0 service-manager process.
  • Complete: report the SSH-only, no-password-login admin policy.
  • Complete: inspect service status through admin status <service>.
  • Complete: persist admin status exports at /state/services/admin.state.
  • Complete: retrieve log metadata through admin logs <service>.
  • Complete: enforce a remote-safe admin command allowlist.
  • Complete: reject unsafe remote admin commands.
  • Complete: emit telemetry for policy exports, status exports, log reads, remote-safe accepts/rejects, and command denials.
  • Complete: expand smoke, benchmark, readiness, telemetry, and RC contract gates for the admin control plane.

Definition of done: QEMU boot proves /bin/service-manager runs admin policy, service status, status export, log retrieval, and remote-safe command checks from EL0. python3 ./scripts/qemu-benchmark.py reports admin_control_plane_active=true, and make qemu-readiness-gate requires the admin telemetry and contract section.

Phase Q16: Full OS Release Candidate

Goal: close the QEMU full OS track with one final release-candidate gate before Intel Desktop implementation starts.

Required work:

  • Complete: add make qemu-full-os-rc as the final QEMU full OS gate.
  • Complete: run the full make qemu-readiness-gate matrix from the final gate.
  • Complete: validate the frozen QEMU contract at contracts/qemu-rc-v1.json.
  • Complete: verify the source syscall/capability ABI matches the frozen contract.
  • Complete: validate benchmark, preview, readiness, and CPU matrix reports.
  • Complete: require all correctness benchmark gates to pass.
  • Complete: preserve performance_claims_allowed=false.
  • Complete: emit build/qemu-full-os-rc-report.json with qemu_full_os_complete=true only when every RC check passes.

Definition of done: make qemu-full-os-rc exits 0, writes a report with schema xaios.qemu.full_os_release_candidate.v1, marks milestone 42 complete, and allows Intel Desktop bring-up to become the next active implementation phase.

Phase Q17: Post-51 QEMU Hardening

Goal: keep improving correctness on macOS/QEMU while physical Intel, Xeon, AMD, and ARM/NVIDIA hardware is not yet available.

Current status:

  • Done: milestone 52 QEMU regression suite with build/qemu-milestone-52-regression-suite.json.
  • Done: milestone 53 fault injection suite with build/qemu-milestone-53-fault-injection.json.
  • Done: milestone 54 ABI contract tests with build/qemu-milestone-54-abi-contract.json.
  • Done: milestone 55 deterministic boot loop with build/qemu-milestone-55-boot-loop.json.
  • Done: milestone 56 userspace control-plane suite with build/qemu-milestone-56-userspace-suite.json.
  • Done: milestone 57 network stack suite with build/qemu-milestone-57-network-suite.json.
  • Done: milestone 58 CPU-only AI runtime simulator suite with build/qemu-milestone-58-cpu-ai-suite.json.
  • Done: milestone 59 developer UX suite with build/qemu-milestone-59-developer-ux.json.
  • Done: aggregate post-51 report with build/qemu-post51-gate-report.json.

Definition of done: make qemu-post51-gate runs milestones 52-59 and writes a passing aggregate report. These are QEMU correctness gates only and do not replace physical hardware validation.

Verification:

  • make qemu-post51-gate

Related Pages

Wiki Footer

Purpose

This page defines the GitHub Wiki footer.

XAI OS is a design-stage server OS for CPU-only embedded AI agents.

Main repo | Wiki home

License to be decided.

Wiki Navigation

Purpose

This page defines the GitHub Wiki navigation sidebar.

Overview

Architecture

Implementation

Platforms

Production

Reference

Clone this wiki locally