feat(installer): stop-and-check gate — recognize an existing install

[LukasWodka]

Summary

Adds a read-only "stop-and-check" gate to the installer so re-running it on an already-set-up machine no longer drags the user through full provisioning. On a verifiably healthy machine it prints one line and hands off to tracebloc (the home screen), then exits 0. Otherwise it falls through to today's 5-step flow, unchanged.

This is PR 2 of 2 (PR 1 = the status-aware tracebloc home screen, tracebloc/cli#244). Together they fix the "I just wanted to use the CLI / reinstall, and suddenly a whole new environment gets provisioned" problem.

Behavior

The gate runs after the banner, before the 5-step roadmap. assess_existing_install() classifies the machine (read-only, bounded, never fatal):

• healthy — cluster running AND a tracebloc release present AND all core workloads Ready (mysql-client + jobs-manager + requests-proxy) AND the CLI present → prints ✓ Already set up on this machine — no need to run the installer again., then runs tracebloc (the home screen) and exit 0. Never re-provisions, never re-prompts login.
• degraded — cluster stopped, a workload not Ready, or the CLI missing → prints an honest one-liner naming what's off ("your secure environment is stopped / still starting up / the CLI isn't installed"), then falls through to today's flow to reconcile.
• fresh — no cluster, or no tracebloc release → today's 5-step install, unchanged.

--force / --reinstall (or TRACEBLOC_FORCE_REINSTALL=1) bypasses the gate entirely. Non-interactive (curl | bash): healthy still short-circuits (the hand-off is output-only, reads no stdin); degraded/fresh run the normal flow.

Safety (this is a public, cosign-verified installer)

• assess is strictly read-only — k3d cluster list, kubectl get, helm list (via the reused detect_installed_client), has. It never starts the cluster, runs helm upgrade, mints credentials, or writes anything, and it does not call the mutating _handle_existing_cluster.
• No false "healthy." Every probe degrades to non-healthy on any error / timeout / missing tool / uncertainty — a machine that needs setup is never told "already set up." (Mutation-proven.)
• Manifest integrity. scripts/lib/assess.sh is added to both bootstrap FILES arrays and manifest.sha256 is regenerated (gen-manifest.sh --check → up to date), so the cosign bootstrap still verifies; install-bootstrap.bats file-count lists updated (15 → 16).
• The healthy path uses exit 0 (not exec) so the install_cleanup EXIT trap still runs — a verified no-op at gate time (before any credential / sudo-keepalive is set).
• provision.sh untouched, to stay clear of the active #838 provision rewrite.

Single source of truth for "ready"

Extracted the client-workload deployment list into _client_workload_deployments() (common.sh); both wait_for_client_ready (the post-install gate) and the assess readiness probe consume it, so the installer's definition of "ready" and the gate's can't drift.

Code review

Adversarial review before opening — both invariants (no false-healthy; non-mutating + manifest-consistent) confirmed and mutation-proven. Fixed in the second commit:

• [should-fix] the readiness gate checked only jobs-manager; a machine with jobs-manager up but requests-proxy (training egress) or mysql-client down could be called "healthy" → now requires all three, via the shared helper.
• [should-fix] an unguarded awk-pipe assignment that could abort the installer under set -e/pipefail (SIGPIPE) → guarded to match its jq sibling.

Deferred (fast-follow)

Surgical per-layer repair of a degraded machine (start-only / CLI-only / reconcile-only, and skipping re-login when already authenticated) — deliberately left out to keep this PR off provision.sh and clear of #838. Rebases cleanly once #838 lands.

Test plan

• scripts/tests/assess.bats (31 tests): each probe (cluster-running jq + awk fallback; workload-ready per-deployment incl. requests-proxy / mysql-client down; CLI present); all classify branches; the hand-off (runs tracebloc, exit 0, fallback); and the main() branch (healthy → hand-off + exit 0; --force bypass; degraded/fresh → flow). Assertions are fail-loud (grep-backed + single-bracket status) to dodge the macOS bats [[ ]] blindspot; the healthy short-circuit and the all-three-workloads gate are mutation-verified.
• shellcheck --severity=error clean; bash -n clean; gen-manifest.sh --check up to date.
• Full suite: only 2 pre-existing macOS-local failures (reproduce on pristine develop, unrelated files) — Linux CI is authoritative for bats.

Checklist

• Targets develop
• Read-only assess; no false-healthy (mutation-proven)
• Manifest regenerated + bootstrap FILES in sync
• provision.sh untouched (#838 coordination)
• Tests added + mutation-checked
• Human review (Bugbot + reviewer on Monday)

---

Pairs with tracebloc/cli#244. Hand-off degrades gracefully: shows today's home screen on the released CLI, the new status-aware one once cli#244 ships.

---

Note

Medium Risk
Changes behavior of the cosign-verified public installer entrypoint; a false healthy would skip needed setup, though the design biases toward fall-through on any doubt and adds extensive bats coverage.

Overview
Adds a read-only stop-and-check gate so re-running the installer on an already-healthy machine skips full provisioning and hands off to tracebloc instead.

New scripts/lib/assess.sh classifies the host as fresh, degraded, or healthy using bounded k3d/kubectl/helm probes (never mutating). Only healthy short-circuits: running cluster, tracebloc release, all core workloads ready, and CLI present. Uncertainty always falls through to the normal install flow—never a false “already set up.”

install-k8s.sh invokes the gate after the banner; --force / --reinstall and TRACEBLOC_FORCE_REINSTALL=1 bypass it. assess.sh is guarded when missing (stale bootstrap) like other optional libs.

_client_workload_deployments() in common.sh centralizes readiness (mysql-client, jobs-manager, requests-proxy); wait_for_client_ready and the assess gate both use it so “ready” cannot drift.

Bootstrap FILES, gen-manifest.sh, manifest.sha256, help text, and assess.bats / bootstrap tests updated for the new module.

^{Reviewed by Cursor Bugbot for commit 18b113a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

👋 Heads-up — Code review queue is at 46 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• backend#1040 — docs(rfc): configurable preprocessing & imputation (RFC 0003) · author: @LukasWodka · reviewer: @saadqbal
• backend#1076 — fix(taxonomy): drop dead instance_segmentation category (#1005) · author: @LukasWodka · reviewer: @saqlainsyed007
• backend#1077 — feat(global_meta): audit/purge command for legacy per-row GlobalMeta (#1007) · author: @LukasWodka · no reviewer assigned
• backend#1084 — fix(metrics): register semseg metrics — pattern-allow iou_class_N + seed static rows (#641) · author: @LukasWodka · reviewer: @saqlainsyed007
• backend#1085 — fix(api): CycleView returns the JSONField confusion matrix; legacy fallback logged (#1075) · author: @LukasWodka · reviewer: @saqlainsyed007
• cli#229 — fix(data): deprecation notices on renamed aliases + clear no-client message (#879) · author: @LukasWodka · reviewer: @saadqbal
• cli#230 — fix(data delete): resolve the table name case-insensitively; fail closed (#1027) · author: @LukasWodka · reviewer: @saqlainsyed007
• cli#231 — fix(doctor): surface container restart history, not just crash-loop-now (#1028 item 4) · author: @LukasWodka · no reviewer assigned
• cli#232 — feat(ingest): guide users from dataset to creating a use case · author: @LukasWodka · reviewer: @saadqbal
• cli#233 — fix(data ingest): preflight & message polish — schema-type validation, friendly missing-label, help copy, clearer errors (fix(install): visible wait on held apt lock instead of a silent spinner (#740) #213 test: cross-repo fresh-shell last-mile E2E harness (install -> CLI -> cluster info -> push) #214 Post-install self-verification of CLI usability (#738) #215 fix(client): allow training pods to reach mysql-client (v1.2.1) #76) · author: @LukasWodka · no reviewer assigned

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)