Skip to content

Harden hypervisor process liveness checks - #363

Open
yummybomb wants to merge 3 commits into
hypeship/generalize-vgpu-devicefrom
hypeship/hypervisor-liveness
Open

Harden hypervisor process liveness checks#363
yummybomb wants to merge 3 commits into
hypeship/generalize-vgpu-devicefrom
hypeship/hypervisor-liveness

Conversation

@yummybomb

@yummybomb yummybomb commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Layer 1 of the vendor VFIO vGPU stack (generalize-vgpu-devicethisvendor-vfio-backendvendor-vfio-vgpu). Pure hypervisor-process hardening with no vGPU-specific code; reviewable in isolation.

The upper layers guard vGPU release decisions on "is this instance's hypervisor still alive", so the liveness answer has to be trustworthy first:

  • Unify liveness checks on ProcessExists — one exported, EPERM-aware, zombie-filtering definition instead of scattered bare kill(pid, 0) probes. EPERM means the process exists but cannot be signaled; treating it as dead would be wrong.
  • Wait for non-child hypervisor exit before finishing kill — after a hypeman restart the hypervisor is not our child, so Wait4 returns ECHILD immediately and the kill loop finished before the process had exited. Poll for actual exit in that case.
  • HypervisorProcessExists: verify socket ownership — a bare PID probe treats any process that reused a stored hypervisor PID as the owning VMM. On Linux, require the PID to own the instance's hypervisor socket before reporting it alive.

Testing

  • go build ./..., go vet clean
  • go test -race ./lib/instances/ targeted suites pass (TestCreateInstanceWithNetwork requires image pulls + iptables and fails in this environment on the unmodified base as well)

Note

Medium Risk
Changes process-exit waiting on delete and how hypervisor PIDs are validated; mistakes could leave stray VMMs or mis-report liveness for downstream vGPU teardown.

Overview
Unifies hypervisor liveness behind exported ProcessExists (EPERM-aware, Linux zombie filtering via /proc) and HypervisorProcessExists (on Linux, requires the PID to match hypervisor.ResolveProcessPID for the instance socket so PID reuse is not treated as the VMM).

Call sites that used the private processExists now use ProcessExists (resolveRuntimeHypervisorPID, refreshHypervisorPID, tests).

Delete path: after SIGKILL, the wait loop no longer treats Wait4 ECHILD as success when the hypervisor is not hypeman’s child (e.g. after restart); it polls until kill(pid, 0) returns ESRCH.

Adds TestHypervisorProcessExistsRejectsLivePIDWithoutSocketOwnership.

Reviewed by Cursor Bugbot for commit 915f0ef. Bugbot is set up for automated code reviews on this repo. Configure here.

kill(pid, 0) returning EPERM means the process exists but cannot be
signaled, and a zombie PID passes a bare kill(0) probe. Export the
EPERM-aware, zombie-filtering processExists helper so every hypervisor
liveness check shares one definition.
After a hypeman restart the hypervisor is not our child, so Wait4
returns ECHILD immediately and the kill loop finished before the
process had exited. Poll for actual process exit in that case.
A bare liveness probe treats any process that reused a stored
hypervisor PID as the owning VMM. Require the PID to own the
instance's hypervisor socket on Linux before reporting it alive.
@yummybomb
yummybomb force-pushed the hypeship/hypervisor-liveness branch from 3c8b92f to 915f0ef Compare August 6, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant