feat: macOS support via Hypervisor.framework (aarch64)#1681
Open
siennathesane wants to merge 14 commits into
Open
feat: macOS support via Hypervisor.framework (aarch64)#1681siennathesane wants to merge 14 commits into
siennathesane wants to merge 14 commits into
Conversation
Introduce a new crate shared by hyperlight-host and the upcoming hvf_surrogate helper process: - core: thin wrapper over the Hypervisor.framework C API (VM/vCPU lifecycle, register access, vCPU run loop with ESR exit decoding) used by both the direct backend and the surrogate server. - proto: length-prefixed JSON IPC protocol (requests/responses, shm-backed memory references) for delegating a sandbox's VM to a surrogate process, needed because HVF allows only one VM per process. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- Move kvm-bindings/kvm-ioctls/mshv-bindings/mshv-ioctls from cfg(unix) to cfg(linux) target dependencies; kvm-ioctls does not compile on macOS even though the code is feature-gated off. - Add an optional hvf cargo feature (applevisor-sys + hyperlight-hvf) and an hvf cfg alias: all(feature = "hvf", target_os = "macos", target_arch = "aarch64"). Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
Three related changes to make the host memory layer work on macOS: - Widen POSIX implementations (mmap/mprotect paths, PreparedFileMapping, interrupt config getters) from cfg(linux) to cfg(unix). - Size guard pages to the host page size (16 KiB on Apple silicon) instead of the 4 KiB guest page granule; mprotect requires host-page-aligned ranges. Also fixes the same latent issue on 16K/64K-page aarch64 Linux kernels. - Back all guest memory by named POSIX shm objects on macOS and carry the backing metadata (object name, offset) through a macOS HostRegionBase, mirroring the Windows type-level split. This allows a surrogate process to map the same memory into a per-sandbox HVF VM (HVF allows only one VM per process). Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
Add a Hypervisor.framework backend for Apple silicon: - HvfVm implementing the VirtualMachine trait over the shared hyperlight_hvf::core wrapper (map/unmap with slot tracking, regs/fpu/ sregs accessors, vCPU run with ESR-decoded exits: IO-page MMIO writes become IoOut/Halt, aborts become MmioRead/MmioWrite, WFI is skipped, vtimer exits are ignored). - MacOSInterruptHandle using hv_vcpus_exit (the only HVF call allowed from a non-owning thread) for sandbox cancellation. - HypervisorType::Hvf detection and wiring into HyperlightVm::new for aarch64; CpuVendor::current() for aarch64-macos and a Hypervisor::Hvf snapshot variant. - Note: HVF binds a VM to its creating process (one VM per process), so this backend supports one live sandbox per process; multi-sandbox support will delegate VMs to surrogate processes. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- dev/hvf-entitlements.plist: com.apple.security.hypervisor, required by Hypervisor.framework; test and surrogate binaries are ad-hoc codesigned with it. - Justfile test-hvf recipe: build, codesign, and run hyperlight-host tests with the hvf feature. - Bump pinned cargo-hyperlight to 0.1.13 for guest builds (macOS hosts additionally need the toolchain to run through rustup with the pinned 1.94 toolchain; newer rustc rejects custom target specs). Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
HVF allows only one VM per process, so multi-sandbox support delegates each sandbox's VM to a per-sandbox surrogate process: - hvf_surrogate: a small server binary owning one HVF VM, speaking the hyperlight_hvf::proto protocol over an inherited unix socket. The vCPU thread executes all VM operations; a reader thread dispatches requests and answers Cancel immediately via hv_vcpus_exit; a writer thread serializes responses. Guest memory is mapped from shared POSIX shm objects by name, never copied. - hvf_surrogate_manager: pooled surrogate lifecycle mirroring the Windows manager (rust-embed extraction, hash-stamped binary name, env sizing via HYPERLIGHT_INITIAL_SURROGATES / HYPERLIGHT_MAX_SURROGATES, kill-on-drop). The extracted binary is ad-hoc codesigned with the hypervisor entitlement before first spawn. - build.rs: nested cargo build of the surrogate on macOS/aarch64 when the hvf feature is enabled. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
Multi-sandbox support for macOS: each sandbox's VM now lives in a
pooled surrogate process, with the host acting as an IPC client.
- HvfSurrogateVm implements the VirtualMachine trait by forwarding over
the proto socket (hvf.rs split into hvf/{mod,direct,surrogate}.rs).
- MacOSInterruptHandle generalizes to Local (hv_vcpus_exit) and Remote
(Cancel frame over a mutex-shared socket writer) targets.
- HyperlightVm::new selects direct vs surrogate mode;
HYPERLIGHT_MAX_SURROGATES=0 selects direct mode with a clean
single-VM error via a process-global NoSurrogateGuard.
- File-backed guest regions carry their path through PreparedFileMapping
so the surrogate can mmap them (MAP_PRIVATE read-only: hv_vm_map
rejects read-only MAP_SHARED).
- Fixes uncovered by the newly-enabled parallel suite:
* surrogate Cancel never fired (vCPU id 0 collided with the no-vCPU
sentinel; store id+1), with a protocol-level regression test;
* surrogate must not shm_unlink after open (broke snapshot remaps);
the host owns the name lifecycle;
* implement reset_vcpu for both backends (needed by snapshot
restore), via a new proto request.
- Full unit suite without --test-threads=1: 268 passed / 11 failed
(pre-S3: 100/175). The remaining 11 are a pre-existing 4K-vs-16K
page-size assumption cluster in the map_region/mmap tests, unrelated
to HVF.
Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- snapshot_goldens/platform.rs: add the Hypervisor::Hvf platform arm (aarch64-apple); v1.0-aarch64-hvf-apple goldens generated and all 5 verification cases pass locally (distributed via the golden registry like the other platforms). - hyperlight_libc/build.rs: use llvm-ar/llvm-ranlib on macOS hosts; Xcode ar silently drops ELF members, producing an empty libhyperlight_libc.a and failing C guest links. - integration_test.rs: macOS arm of the HostRegionBase compile guard; cap interrupt_infinite_moving_loop_stress_test at 48 threads on macOS (HVF allows 127 concurrent VMs system-wide, measured). - hvf_surrogate_manager: default pool max 128 (was 64) against the measured system cap; pool backpressure near the cap is by design. Integration binaries all green locally: integration_test 36/0, sandbox_host_tests 14/0, wit_test 43/0. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
The mapped-region tests constructed MemoryRegions with 4 KiB alignment assumptions, but HyperlightVm::map_region validates against the host page size (16 KiB on Apple silicon). Derive test region alignment from page_size::get() so the tests assert the intended Overlapping errors on any host page size; on 4K hosts they are byte-identical to before. Full lib suite is now 279 passed / 0 failed / 8 ignored on macOS. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- New build-test-macos-hvf job on GitHub-hosted macos-26 arm64 runners: installs cargo-hyperlight from a pinned git rev (crates.io 0.1.13 does not compile on macOS), builds rust and C guests, and runs the test-hvf recipe. Requires macOS 26 for the 4 KiB IPA granule API. Snapshot-golden verification is deferred until the hvf goldens are published to the registry. - Fix the test-hvf recipe to execute test binaries directly after codesigning instead of re-invoking cargo test: codesigning touches the binaries, which cargo treats as a rebuild trigger, silently discarding the entitlement and failing every sandbox test with HV_DENIED. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
aarch64 builds with no active backend (e.g. macOS without the hvf feature, or aarch64-linux with --no-default-features) failed to compile: HyperlightVm::new referenced the interrupt handle and VmError unconditionally. Return NoHypervisorFound early in that configuration and cfg-gate the now-dead register constants, VmExit variants and reset_vcpu defaults so -D warnings clippy configs stay clean. Also match arm64 (macOS uname -m) in the clippy feature script's aarch64 exclusions, and include hyperlight-hvf in MSRV verification. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- VM creation now fails with a clear IpaGranuleUnsupported error when the 4 KiB IPA granule cannot be configured (pre-macOS 26), instead of silently falling back to a 16 KiB granule that cannot map 4 KiB-aligned guest regions. - Box the Regs/FpuState IPC payloads (clippy large_enum_variant) and collapse the host-side HypervisorError::HvfError variants into a single one carrying hyperlight_hvf::core::HvfError, which also fixes the enum_variant_names lint in the kvm,mshv3,hvf feature combo. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
- getting-started.md: macOS platform section — hvf cargo feature, macOS 26 requirement, the com.apple.security.hypervisor entitlement and ad-hoc codesigning, one-VM-per-process and the surrogate model, guest toolchain notes (rustup, LLVM, cargo-hyperlight from git). - hyperlight-surrogate-development-notes.md: contrast the HVF surrogate (a real server process with an IPC protocol) with the WHP surrogate (an empty suspended address-space container). - README.md: list Hypervisor.framework as a supported hypervisor. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
Follows the crate's existing pattern for justified invariants (clippy::expect_used only fires in release): the socket is only accessible while the process is checked out of the pool. Signed-off-by: Sienna Meridian Satterwhite <sienna@sunbeam.pt>
siennathesane
requested review from
andreiltd,
danbugs,
dblnz,
devigned,
jprendes,
jsturtevant,
ludfjig,
simongdavies,
squillace and
syntactically
as code owners
July 22, 2026 19:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
macOS support via Hypervisor.framework (aarch64)
Adds a new
hvfbackend for Apple silicon, with multi-sandbox support via per-sandbox surrogate processes.Backend
HvfVmimplements theVirtualMachinetrait over a new sharedhyperlight-hvfcrate (vCPU lifecycle, register access, ESR-decoded exits: IO-page MMIO writes →IoOut/Halt, aborts →MmioRead/MmioWrite, WFI skipped, vtimer ignored). Cancellation useshv_vcpus_exit, the only HVF call allowed from a non-owning thread. Requires macOS 26 (4 KiB IPA granule; a clearIpaGranuleUnsupportederror is raised on older systems) and thecom.apple.security.hypervisorentitlement (seedev/hvf-entitlements.plist).Surrogate architecture
HVF allows only one VM per process and has no cross-process mapping API, so each sandbox's VM lives in a pooled
hvf_surrogatehelper process. Unlike the WHP surrogate (an empty suspended address-space container), this is a real server: vCPU thread + reader/writer threads speaking a small JSON-over-unix-socket protocol (hyperlight_hvf::proto). Guest memory is shared by named POSIX shm objects, never copied. The surrogate binary is nested-built byhyperlight-host's build script, embedded via rust-embed, and ad-hoc codesigned with the entitlement at extraction time.Pool sizing honors
HYPERLIGHT_INITIAL_SURROGATES/HYPERLIGHT_MAX_SURROGATES(defaults 0/128; HVF allows 127 concurrent VMs system-wide).HYPERLIGHT_MAX_SURROGATES=0selects the direct in-process backend, limited to one live sandbox per process.Also in this PR
map_region/mmap tests made host-page-size aware (pre-existing aarch64 test debt)cargo-hyperlight0.1.13 (CI installs a pinned git rev because the crates.io release doesn't compile on macOS);llvm-arfor C guest libc archivingbuild-test-macos-hvfCI job onmacos-26runners;just test-hvfrecipe that builds, ad-hoc codesigns, and runs the suitegetting-started.md, HVF section in the surrogate development notes, README hypervisor listTesting
On macOS 26 (Apple silicon), all via the surrogate path by default:
v1.0-aarch64-hvf-apple): 5/5 verified locally — publishing to the golden registry is a follow-upclippy-exhaustive(debug + release), nightlyfmt-check, MSRV 1.89: greenFollow-ups (out of scope)
I am happy to be the maintainer for macOS code, if needed :)