feat(#420): resume after reboot + install state file (fast idempotent re-runs) - #501
Conversation
Two legitimate reboots (Windows feature enablement; Docker/WSL first boot) can interrupt the install. Before, every interruption ended with "re-find and re-paste the one-liner", and each re-run re-walked the whole install. Now: - Schema-versioned state file under %USERPROFILE%\.tracebloc\install-state.json. Pure, unit-tested helpers (New-InstallState, ConvertTo-InstallState, Add-CompletedStage, Test-StateHasStage, Test-InstallStateCurrent) + thin I/O wrappers (Read/Save/Set-StageComplete/Set-InstallComplete). A corrupt or incompatible-schema file degrades to a fresh state -- never a throw. The state is ADVISORY: every stage still self-verifies (tools re-checked, cluster re-derived), so a stale checkpoint can never skip real work. - Resume-after-reboot via a RunOnce continuation. On a reboot the installer checkpoints 'features-reboot-pending' and registers HKCU RunOnce (Get-ResumeCommand reuses the #421 elevation arg-builder + -Resume) so the install resumes at next sign-in with no re-pasting -- for both auto-reboot and manual -NoReboot. -Resume is forwarded through the admin-gate self-elevation. Only the durable -File form carries -Resume; the irm|iex shim has no param block (#421), and the state file drives the one-liner path anyway. Cleared on success so it never fires spuriously. - Fast idempotent re-runs. Each of the 6 steps checkpoints on completion; on a fresh run where a prior install completed AND the tools + cluster are still present, the installer prints "already installed -- nothing to do" and exits 0. The gate verifies real presence, not just the checkpoint, so the claim is honest. Tests: pure state helpers (round-trip, corrupt/empty/wrong-schema -> fresh, dedup), Get-ResumeCommand (File form carries -File+switches+-Resume; one-liner omits -Resume), Save/Read I/O round-trip + corrupt-file degradation (mocked path, no home writes), and source guards for the -Resume plumbing, reboot-exit RunOnce, per-step checkpoints, success clear+complete, and the honest fast-path gate. Closes #420 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
👋 Heads-up — Code review queue is at 36 / 30 Above the WIP limit. The team convention is to review existing PRs before opening new work. Open PRs currently in Code review (oldest first):
Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.) |
|
👋 Heads-up — Code review queue is at 37 / 30 Above the WIP limit. The team convention is to review existing PRs before opening new work. Open PRs currently in Code review (oldest first):
Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.) |
Bugbot (High): Set-InstallComplete ran unconditionally before the exit-code check, so a failed client state (bad_creds/crash/image_pull/image_pull_ca) still persisted completed=true. A re-run then hit the tools+cluster fast path, printed "already installed -- nothing to do" and exited 0 -- blocking the documented remediation (the summary tells the operator to re-run). Add Test-InstallSucceeded (connected/starting = success) as the SINGLE source of truth shared by the completion checkpoint and the exit code, so they can't drift. Set-InstallComplete now runs only when Test-InstallSucceeded; the exit code uses the same predicate. RunOnce is still cleared unconditionally (no reboot is pending once the walk finishes). Unit test across connected/starting + failure states, plus source guards for the gated completion and shared-predicate exit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bugbot round 2 on the new fast path:
- High: the probe only checked the cluster NAME appeared in `k3d cluster list`, not
that it was running -- so a completed install with a STOPPED cluster (or dead
node) printed "nothing to do" and exited 0, skipping New-K3dCluster's start/repair.
- Medium: it ran a bare `k3d cluster list` with no deadline, so a wedged Docker
engine hung the fast path at the start of every re-run (violates the bounded-
external-command rule).
Replace Test-ClusterPresent with Test-ClusterRunning: BOUNDED via Start-Job +
Wait-JobWithProgress (15s) + Remove-Job, parsing `-o json` through a pure
Test-ClusterRunningInList that requires serversRunning >= 1 for the named cluster
(present-but-stopped -> false -> fall through to the repair walk). Fast-path message
now states what was verified ("installed and the cluster is running").
Tests: Test-ClusterRunningInList (running/stopped/absent/corrupt/no-count), a
source guard that the probe is job+deadline bounded, and the updated fast-path gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saadqbal
left a comment
There was a problem hiding this comment.
Solid, well-tested state machine and the honest self-verify framing is the right call. Three things I'd want resolved before merge — two are correctness, one is dead weight. Details inline; happy to be wrong on the first if the split-account flow is out of scope for resume.
… stages, split-account note Asad's review on #501: - Completion counted `starting`, but that's Get-NotReadyState's catch-all for a client that isn't Ready yet -- so a client that never comes up got completed=true and armed the fast path, skipping remediation. Completion now requires the client to be CONNECTED (new Test-InstallConnected); the exit code keeps the more lenient connected||starting (Test-InstallSucceeded) so a slow-but-starting client doesn't hard-fail the run. The two predicates differ on purpose and say why. - The per-stage checkpoints were written but never read (the six steps set shared $script: state, so a resume must re-walk them; speed comes from each step's own self-skip). Dropped the dead machinery -- stages array, Add-CompletedStage, Test-StateHasStage, Set-StageComplete, Test-StageComplete, per-step calls, and the features-reboot-pending checkpoint. State is now just {schema, completed}. - Resume scope clarified: the reboot happens in Step 1 during the elevating account's session, so HKCU RunOnce is correct for that account. Added a Hint for the split -DailyUser case (a different user signing in after the reboot must re-run). Tests updated: pure helpers (no stages), Set-InstallComplete round-trip, Test-InstallConnected vs Test-InstallSucceeded across states, split-account Hint guard, and a guard that the stage machinery stays gone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1be7c57. Configure here.
… failure Bugbot round on the review rework: - High: the fast path claimed "nothing to do" from completed + tools + a running cluster WITHOUT checking the client workloads are Ready (the bash assess path requires Ready workloads), and `completed` was never cleared when a later walk failed. So a re-run started because the client is down could print "already installed" and skip the remediation. Now the fast path also requires Test-ClientHealthy (finds the release namespace via Get-InstalledClientInfo, then a SHORT bounded `kubectl rollout status --timeout=5s` per client deployment), and a walk that doesn't end connected calls Clear-InstallCompleted so a stale flag can't keep the fast path armed. Extracted Get-ClientDeploymentNames as the shared source of truth for the readiness gate and the health check (no duplication). - Low: the force-reinstall hint hard-coded ~\.tracebloc\install-state.json; it now interpolates Get-InstallStatePath so it's correct under an overridden HOST_DATA_DIR. Tests: Test-ClientHealthy (unknown/no-ns/all-ready/not-ready via mocks), Get-ClientDeploymentNames, Clear-InstallCompleted round-trip, and updated fast-path + hint-path + completion source guards. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
/fr-pass |

#420 — Resume after reboot + install state file (fast idempotent re-runs)
Two legitimate reboots (Windows feature enablement; Docker/WSL first boot) can interrupt the install. Before, every interruption ended with "re-find and re-paste the one-liner", and each re-run re-walked the whole install (up to ~90s of
wsl --update, winget checks, full preflight) — which reads as "the installer is slow/broken".What's in
%USERPROFILE%\.tracebloc\install-state.json. Pure, unit-tested helpers (New-InstallState,ConvertTo-InstallState,Add-CompletedStage,Test-StateHasStage,Test-InstallStateCurrent) + thin I/O wrappers (Read/Save/Set-StageComplete/Set-InstallComplete). Corrupt or incompatible-schema file → a fresh state, never a throw. The state is advisory: every stage still self-verifies (tools re-checked, cluster re-derived), so a stale checkpoint can never skip real work.features-reboot-pendingand registers an HKCU RunOnce continuation (Get-ResumeCommandreuses the feat(installer): self-elevate and guide wrong-terminal users (cmd / unelevated PowerShell) #421 elevation arg-builder +-Resume) so the install resumes at next sign-in with no re-pasting — for both auto-reboot and manual-NoReboot.-Resumeis forwarded through the admin-gate self-elevation. Only the durable-Fileform carries-Resume(theirm|iexshim has no param block, per feat(installer): self-elevate and guide wrong-terminal users (cmd / unelevated PowerShell) #421; the state file drives the one-liner path anyway). Cleared on success so it never fires spuriously.already installed -- nothing to doand exits 0. The gate verifies real presence, not just the checkpoint, so the "nothing to do" claim is honest.Acceptance criteria
-Resume; live reboot is host-gated, unit-tested here).Tests
Pure state helpers (round-trip; corrupt/empty/wrong-schema → fresh; append/dedup/order),
Get-ResumeCommand(durable-Fileform carries-File+ switches +-Resume; one-liner omits-Resume),Save/ReadI/O round-trip + corrupt-file degradation (mocked path — no writes to the real home dir), and source guards for the-Resumeplumbing, reboot-exit RunOnce, per-step checkpoints, success clear+complete, and the honest fast-path gate. Full suite: 315 passed / 0 failed / 9 skipped; check-style + check-drift clean;scripts/manifest.sha256regenerated (R8).Host-gated (noted, not in this PR)
The literal end-to-end reboot→resume needs a real Windows host — same class as the scheduled Windows e2e leg (#436). The state-file + RunOnce + fast-path logic is unit-tested here.
Closes #420
Note
Medium Risk
Changes first-run and reboot behavior for the Windows installer (RunOnce, elevation, exit codes) and gates “nothing to do” on cluster/client health; mistakes could skip repair or leave stale resume entries, but logic is advisory with live probes and extensive unit tests.
Overview
Adds resume-after-reboot and a schema-versioned install state file under
HOST_DATA_DIRso interrupted Windowsinstall-k8s.ps1runs can continue without re-pasting the one-liner, and healthy machines can exit quickly on repeat runs.When Windows features need a reboot, the installer registers an HKCU RunOnce continuation (via
Get-ResumeCommand/Register-ResumeAfterReboot) and forwards a new-Resumeswitch through self-elevation (Get-ElevationCommand/Invoke-SelfElevate). The resumed run skips the “nothing to do” fast path and shows a resume banner. RunOnce is cleared on a full successful walk; completion is written toinstall-state.jsononly when the client isconnected(not merelystarting), and a stalecompletedflag is cleared on failure so remediation isn’t skipped.The fast path exits 0 when state says completed and live checks pass: all tools on PATH, k3d cluster running (bounded
k3d cluster listjob), and client deployments Ready (Test-ClientHealthy/ sharedGet-ClientDeploymentNames). Corrupt or wrong-schema state degrades to fresh state without throwing.Pester coverage for state helpers, cluster list parsing, resume command wiring, completion vs exit predicates, and manifest hash update for
install-k8s.ps1.Reviewed by Cursor Bugbot for commit 792aec3. Bugbot is set up for automated code reviews on this repo. Configure here.