Replace QEMU ACPI helper with a pure Rust generator

[kvinwang]

Summary

• add a pure Rust qemu-acpi crate that reproduces Q35 ACPI/fw_cfg blobs for dstack's supported QEMU compatibility profiles
• support the full valid dstack topology domain, including up to 4096 vCPUs, dynamic 128 KiB table-blob growth, PCI slot limits, PXB GPU topology, NUMA/SRAT memory boundaries, hotplug policy, SMM, PIC, and PCI hole sizing
• replace the dstack-acpi-tables subprocess in dstack-mr with in-process generation
• remove custom QEMU build stages, runtime libraries, package pins, and obsolete build documentation from KMS and Verifier images

Compatibility validation

The complete measured blob set (etc/acpi/tables, etc/table-loader, and etc/acpi/rsdp) was compared byte for byte against the pinned production compatibility reference across:

• QEMU 8.0–8.2, 9.0–9.2, 10.0–10.2, and 11.0–11.2 compatibility profiles
• normal, NUMA/hugepage, and NUMA/8-GPU-PXB topologies for every listed profile
• 1, 2, 8, 64, 256, 512, 1024, 1536, 2048, and 4096 vCPUs
• NIC counts 0/1/2, volume counts 0/1/4, GPU counts 0/1/8/32, and NVSwitch counts 0/1/4
• PCI hotplug on/off, PIC, SMM, root-verity modes, vvfat/9p sharing, and explicit PCI hole sizes
• memory-layout boundaries from 1 MiB through 1 TiB, including Q35's 2816 MiB low-memory split and AMD 1 TiB hole relocation

The fixture provenance is pinned to kvinwang/qemu-tdx@9de6fdfff3a84103b83ca6b2e8c4fb8e05cf9195. Test-only instrumentation patches and reproducible differential scripts are included in crates/qemu-acpi/scripts.

The compatibility implementation was also cross-validated against genuine upstream releases rather than only the dstack fork:

• QEMU 10.2.4 (3e0bcba1ca7d6607ca49a988d165f052a3a53323)
• QEMU 11.0.3 (aeec49e8170de7846f476124602cf7acd400c3df)

For both releases, normal, NUMA, and NUMA/PXB outputs matched all three blobs byte for byte. No genuine upstream 11.1 release exists at the time of this audit, so 11.1 remains explicitly documented as a production-fork compatibility profile.

Golden fixtures now cover tables + loader + rsdp for both ordinary and NUMA configurations. Loader construction is table-driven and includes optional SRAT checksum and all RSDT pointer commands in QEMU's exact build order.

Security

• production generation contains no unwrap or expect
• AML lengths, ACPI headers, offsets, arithmetic, table ranges, and loader traversal use checked operations and structured errors
• malformed/truncated table and hostile boundary inputs are tested
• strict workspace clippy::unwrap_used and clippy::expect_used gates pass

Tests

• production compatibility differential: 10 profiles × 3 topologies × 3 blobs
• genuine upstream differential: 2 releases × 3 topologies × 3 blobs
• cargo test -p qemu-acpi
• cargo test -p dstack-mr -p dstack-mr-cli
• cargo clippy -- -D warnings -D clippy::expect_used -D clippy::unwrap_used --allow unused_variables
• cargo fmt --all -- --check
• shellcheck crates/qemu-acpi/scripts/*.sh
• reuse lint
• prek run --from-ref origin/next --to-ref HEAD --show-diff-on-failure

Local Docker execution was attempted, but this runner's mandatory sudo su kvin -c "docker ..." path is unavailable because the environment has no_new_privileges and a non-root-owned /etc/sudo.conf. The Docker build checks in CI provide the authoritative container validation.

[Copilot]

Pull request overview

This PR removes the runtime/build-time dependency on a custom QEMU binary for ACPI table extraction by introducing an in-repo, pure-Rust qemu-acpi crate and switching dstack-mr to generate Q35 ACPI/fw_cfg blobs in-process. It also simplifies KMS/Verifier container builds and documentation by deleting the legacy QEMU/ACPI helper stages and pinned-package inputs.

Changes:

• Added dstack/crates/qemu-acpi to generate QEMU-compatible ACPI blobs (with fixtures and golden tests).
• Updated dstack-mr to use the new crate instead of invoking dstack-acpi-tables.
• Removed QEMU build stages/pins from Verifier and KMS builder images, plus updated docs and CI docker build checks accordingly.

Reviewed changes

Copilot reviewed 29 out of 51 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
REUSE.toml	Removes obsolete QEMU pinned-package path and annotates new ACPI fixture binaries for REUSE compliance.
prek.toml	Drops shellcheck exclusion for deleted config-qemu.sh.
dstack/vmm/src/config.rs	Updates config docs to reflect new measurement path wording.
dstack/verifier/builder/shared/qemu-pinned-packages.txt	Removes obsolete pinned package list for deleted QEMU/ACPI stage.
dstack/verifier/builder/Dockerfile	Deletes the acpi-builder stage and removes QEMU runtime artifacts from the final image.
dstack/verifier/builder/build-image.sh	Stops generating/verifying pinned packages for the removed acpi-builder stage.
dstack/kms/dstack-app/builder/shared/qemu-pinned-packages.txt	Removes obsolete pinned package list for deleted QEMU/ACPI stage.
dstack/kms/dstack-app/builder/README.md	Updates builder README to describe in-binary, pure-Rust ACPI measurement support.
dstack/kms/dstack-app/builder/Dockerfile	Removes the QEMU/ACPI build stage and dstack-acpi-tables installation.
dstack/kms/dstack-app/builder/build-image.sh	Stops extracting pinned packages for the removed QEMU stage and simplifies build invocation.
dstack/gateway/dstack-app/builder/README.md	Removes mention of dstack-acpi-tables from the gateway builder README.
dstack/dstack-mr/src/acpi.rs	Replaces subprocess QEMU invocation and ad-hoc parsing with qemu_acpi::build() integration.
dstack/dstack-mr/Cargo.toml	Adds qemu-acpi workspace dependency.
dstack/crates/qemu-acpi/src/topology.rs	Introduces machine topology model and validation for the generator.
dstack/crates/qemu-acpi/src/tables.rs	Implements ACPI blob mutation/generation and golden-vector tests.
dstack/crates/qemu-acpi/src/srat.rs	Adds SRAT table construction for NUMA/hugepage layouts.
dstack/crates/qemu-acpi/src/profile.rs	Adds QEMU version parsing and compatibility-family mapping.
dstack/crates/qemu-acpi/src/lib.rs	Exposes the public API (build, config types, error types) for ACPI blob generation.
dstack/crates/qemu-acpi/src/fw_cfg.rs	Generates fw_cfg loader/rsdp blobs and applies checksums/pointers.
dstack/crates/qemu-acpi/src/cpu.rs	Generates AML CPU objects and notify-case snippets for CPU hotplug support.
dstack/crates/qemu-acpi/src/aml_patch.rs	Provides AML package-length encoding/adjustment helpers used during mutation.
dstack/crates/qemu-acpi/fixtures/README.md	Documents provenance and regeneration rules for binary fixtures.
dstack/crates/qemu-acpi/examples/dump.rs	Adds an example program to generate and dump tables for debugging.
dstack/crates/qemu-acpi/Cargo.toml	Defines the new crate and its dependencies.
dstack/Cargo.toml	Adds qemu-acpi to the workspace members and workspace dependencies.
dstack/Cargo.lock	Records the new crate and dependency graph changes.
dstack/build/shared/config-qemu.sh	Removes obsolete QEMU build configuration script.
docs/tutorials/kms-build-configuration.md	Removes instructions and dependencies for copying/building dstack-acpi-tables into images.
docs/tutorials/attestation-verification.md	Replaces QEMU helper build instructions with in-process ACPI generation note.
.github/workflows/docker-build-check.yml	Removes CI steps for building/verifying the deleted acpi-builder stages and pins.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[kvinwang]

Security hardening update: all production unwrap/expect paths in the new generator were removed. AML lengths, ACPI headers, arithmetic, table ranges, and fw_cfg traversal now use checked operations and return structured errors. Added tests for truncated AML, zero/maximum hostile counts, every compatibility family at CPU transition/max boundaries, u64::MAX memory and PCI-hole values. The full workspace now passes its strict clippy::unwrap_used/expect_used gate locally, and byte-for-byte QEMU differential output remains unchanged.

[kvinwang]

Addressed the independent differential review in e5af262bb:

• NUMA loader correctness: fw_cfg now discovers optional SRAT, emits its ADD_CHECKSUM in QEMU build order, and derives every RSDT ADD_POINTER from the actual RSDT length. The previously missing SRAT checksum and fifth RSDT pointer are present. New normal and NUMA golden fixtures compare tables, loader, and rsdp, not only tables.
• Reference provenance: fixtures now pin the exact repository, branch, full revision (9de6fdfff3a84103b83ca6b2e8c4fb8e05cf9195), build flags, image inputs, and commands. Test-only all-blob dump patches plus executable differential scripts are included.
• Genuine upstream validation: built upstream QEMU 10.2.4 (3e0bcba1ca...) and 11.0.3 (aeec49e817...) and compared normal, NUMA, and NUMA/PXB configurations. All three blobs match byte for byte in all six cases. The docs explicitly state that 11.1 has no genuine upstream release yet and is a production-fork compatibility profile.
• Readability: rewrote fw_cfg.rs with descriptive names/types and loader ABI/order documentation; documented every Layout patch-point field and the derivation procedure; removed the dead width bindings.
• Safety: parsing and offset calculations remain checked and malformed/truncated tests were added. Strict workspace clippy with unwrap_used/expect_used denied passes.

The production-fork differential script also passed 10 compatibility versions × normal/NUMA/NUMA-PXB, with tables + loader + rsdp byte-identical in all 30 cases.