Harden rollout VM lab startup#96
Merged
chrisbliss18 merged 3 commits intov2from May 1, 2026
Merged
Conversation
added 3 commits
April 30, 2026 22:36
Make the VM lab staging path start db, v1, and v2 guests before waiting for SSH. This keeps snapshot-backed smoke targets self-contained when the baseline snapshots leave the domains shut off, which avoids a quiet install-v2 wait during rollout rehearsals. Document the explicit start-topology command and clarify that the Makefile VM lab targets cover VM startup before staging artifacts and running snapshot flows.
Refine the VM lab startup helper so it only auto-starts domains in known safe inactive states and refuses ambiguous libvirt states like paused or suspended. The previous version relied on virsh start failing, which was correct but less useful for an operator running a rollout rehearsal. Also document the exact prefix-derived lab domains that start-topology may touch so the VM lifecycle boundary is visible near the setup instructions.
Tighten start-topology after repeated rollout rehearsal review. The helper now validates the complete db/v1/v2 topology before starting any domain, refuses crashed or otherwise unexpected libvirt states instead of restarting them, and keeps the operator-facing error focused on the prefix or setup step to fix. This prevents partial auto-start behavior when the lab prefix is wrong or the topology is incomplete, while preserving the intended self-start behavior for cleanly shut-off snapshot-backed lab VMs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This hardens the rollout VM lab so snapshot-backed rehearsal targets can start from an offline lab topology without hanging in the staging step.
The lab harness now adds
start-topologyand calls it frominstall-v2, so the db/v1/v2 lab guests are started before SSH waits and artifact staging begin. The helper is intentionally narrow: it only touches the three prefix-derived lab domains, validates that the complete db/v1/v2 topology exists before starting anything, auto-starts only cleanlyshut offVMs, and refuses unexpected states such ascrashed,paused, orsuspendedfor operator inspection.The VM lab docs were updated to make that behavior explicit near the setup instructions and snapshot-backed flow notes.
Why
During rollout readiness rehearsal,
make rollout-vm-lab-snapshot-all-smokeassumed the VMs were already running before staging v2. When the baseline snapshots left domains shut off, the target could sit quietly waiting for SSH. This change makes the rehearsal target self-contained while keeping VM lifecycle automation tightly scoped and easy to audit.Example output
Powered-off lab topology now starts explicitly before staging:
Wrong prefix or missing topology fails before any start attempts:
Unsafe libvirt states are refused for inspection:
Validation
bash -n scripts/rollout-vm-lab.shgit diff --checkmake rollout-vm-lab-stage-v2from a powered-off topologystart-topologysimulationstart-topologysimulationmake rollout-vm-lab-snapshot-execute-smokemake rollout-docs-verify