machine: snapshot master FPU at prepare time for cross-process forks#85
Merged
Conversation
Fork construction and fork reset copied the master's FPU state with a live KVM_GET_FPU against the master's vCPU fd. That works only in the process that created the VM: KVM vCPU ioctls require the caller's mm to be the VM-creating mm, so a child that inherited the master over fork() and builds its own VM from the master's copy-on-write memory gets -EIO. Snapshot the FPU registers into the Machine at prepare_copy_on_write() time, while the master's vCPU fd is still ioctl-able, and have fork construction and reset_to() read that snapshot via prepared_fpu_registers() instead. The master is frozen after prepare and never runs again, so the snapshot equals a live KVM_GET_FPU -- behavior is unchanged for the in-process case. The sregs half of this is already covered by TINYKVM_USE_SYNCED_SREGS (kvm_run page instead of KVM_GET_SREGS); add a hard guard so disabling it can't silently reintroduce the -EIO on the sregs path. Co-Authored-By: Claude <noreply@anthropic.com>
fwsGonzo
reviewed
Jul 6, 2026
| cross-process fork construction breaks with -EIO. */ | ||
| #if !TINYKVM_USE_SYNCED_SREGS | ||
| #error "TINYKVM_USE_SYNCED_SREGS must be enabled for cross-process fork construction" | ||
| #endif |
Member
There was a problem hiding this comment.
Does this break ARM? I thought I saw that ARM didn't support this?
Collaborator
Author
There was a problem hiding this comment.
yes. this is true.
checked on native ARM64. The existing ARM test suite passed, but I found one ARM-specific gap: PR 85 only snapshotted prepared FPU state in the non-ARM vcpu.cpp; ARM’s prepare_copy_on_write() was still leaving the prepared FP/SIMD snapshot zeroed. I pushed bb38a40, which snapshots it on ARM too and adds a regression test for fork construction and reset_to(). /tests/run_unit_tests.sh passes on ARM64.
sloppy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fork construction (
Machine(const Machine&, const MachineOptions&)) and forkreset (
reset_to) copy the master's FPU state with a liveKVM_GET_FPUagainstthe master's vCPU fd. That only works in the process that created the VM: KVM
vCPU ioctls require the caller's
mmto be the VM-creatingmm. A child thatinherited the master over
fork()and builds its own VM from the master'scopy-on-write memory gets
-EIOon that ioctl and cannot construct a fork.Change
Snapshot the master's FPU registers into the
Machineatprepare_copy_on_write()time — while the master's vCPU fd is still ioctl-able —and have fork construction and
reset_to()read that snapshot via the newprepared_fpu_registers()accessor instead of a liveKVM_GET_FPU.prepare and never runs again, so the snapshot is bit-identical to what a live
KVM_GET_FPUwould return at fork time.prepared_fpu_registers()assertsm_prepped,and the snapshot is only taken in
prepare_copy_on_write().covered by
TINYKVM_USE_SYNCED_SREGS(reads the mmap'dkvm_runpage insteadof
KVM_GET_SREGS). Added a#errorguard so disabling that define can'tsilently reintroduce the
-EIOon the sregs path.Cost is one
tinykvm_fpuregs(~fxsave/xsave layout) perMachine, always.Testing
cmake --build buildclean (only pre-existing sign-compare warnings insrc/tests.cpp).tests/run_unit_tests.sh:test_resetpasses 46/47 assertions. The onefailing assertion (
output_is_hello_world) also fails identically onmaster— it comes from a test that compiles a static guest ELF with
gccat runtimeand is environmental on my host, unrelated to this change. Same for
test_tegridy. All other suites pass.Motivation
Enables a process-per-VM model where a single-threaded zygote
fork()s workerprocesses that each construct a TinyKVM fork from a master inherited over
fork()— isolating tenant workloads in separate address spaces / jailedprocesses rather than threads.