vm: exempt clone/restore provisioning from the child VM's CPU ceiling (#186) - #187
Conversation
…#186) Since #182 the VMM pays its own Guaranteed-at-N quota from the first instruction, so restore's eager memory copy runs on 1 core of budget: under a 20-VM storm, copy-mode clone P90 doubled (530 -> 1032ms). Clone/restore now launch with cpu.max at max — weight, fence, and placement still bind, so loading competes fairly inside the fence rather than running unbounded — and the finite quota (then burst) is armed after the memory load, before resume. Every pre-arm failure leaves a paused VMM with no guest work done; after resume the cap is already live, so no uncapped-running state exists to converge. Plain boot keeps arming at spawn.
An unexported helper sat between Arm and Remove; public-above-private puts it with the other control-file writers.
Hardware round (testbed, m-985a070 vs pre-fix 3542c8a, 20-VM storm, fence 0-14)Kill injection — PASS, textbook. cocoon killed 150ms into a copy-mode clone (inside the load window): left a Post-resume ceiling — verified. Immediately after a clone returns, the scope reads a finite Storm A/B — inconclusive on this host, honestly reported. The testbed exhibits a scheduler undersaturation anomaly (runnable tasks cap at ~50%/core; host userspace busy-loops show the same, so it is not cocoon/cgroup): the 20-VM storm only fills ~8 of 15 fenced cores, quotas never bind, and both binaries clone in a healthy 300–450ms band (pre p50/p90 308/358, fixed 412/436 — the ~100ms delta is within the band and unexplained but not the issue's failure mode). The 1032ms pathology regime is unreachable here; the A/B needs a host that actually saturates (the vk fleet where the original numbers came from is the right place). |
Implements #186 as frozen (arm-before-resume sequencing).
What
Clone and restore launch the VMM into its final scope with
cpu.max = "max <period>"— weight, fence, placement, and burst=0 bind from the first instruction, so snapshot memory loading competes fairly inside the fence instead of being throttled to the guest's N-core budget. After the memory load and paused-state device work complete, the finite quota (then burst) is armed; only then does the guest resume. Plain boot keeps arming at spawn — behavior byte-identical.Motivating numbers (issue): under a 20-VM CPU storm, copy-mode clone P90 went 530 → 1032ms once the child scope + fence applied; the same round showed the fence's control-plane win intact (
vm list133 → 5ms P90). This moves the provisioning window to the right side of that boundary.Why arm-before-resume
Every failure before arming leaves a paused VMM: an arming failure aborts the launch; a cocoon death leaves no running guest workload, and the existing recovery authorities (restore quarantine via
PrepareStart, clone stale-create reclaim) converge it by kill. After resume the cap is already live. The uncapped-running state is dissolved rather than patched — resident supervision adopting a live VMM never sees a VM whose ceiling is still open.cgroup.PreparegainsdeferQuota;cgroup.Armwrites finitecpu.maxthen burst (idempotent, retry-converges)Backend.ArmCPUQuotacalled at four sites: CH/FC × clone/restore, each between memory load and resumeNot in this PR
Hardware acceptance per the issue (storm A/B copy-mode P90 back to the ~600ms class, post-resume burn capped at exactly N, the two kill-injection points) runs as a follow-up round on the testbed.