Skip to content

fix(doctor): stop the cluster-down cascade of misleading remedies#354

Closed
LukasWodka wants to merge 1 commit into
developfrom
fix/doctor-reachability-gate
Closed

fix(doctor): stop the cluster-down cascade of misleading remedies#354
LukasWodka wants to merge 1 commit into
developfrom
fix/doctor-reachability-gate

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Fixes #350 · epic backend#1142 (Theme A).

Problem

When the cluster API is unreachable (Docker/k3d stopped — the reported prod case), tb doctor ran all 8 downstream checks anyway; each invented a domain-specific cause with the wrong fix (storage / chart-reinstall / RBAC), burying the one true cause under ~6 false ✖/⚠.

Fix

  • New StatusUnknown (Worst() ignores it → verdict/exit code unchanged).
  • isUnreachable(err) distinguishes a transport error from ErrNoParentRelease/RBAC/NotFound (API answered).
  • Run() short-circuits on an unreachable API: one honest Cluster reachable ✖ + 7 StatusUnknown ("couldn't check"). checkBackendEgress still runs (probes from this machine). When the API IS reachable, every check behaves exactly as before — isolated-failure messaging preserved.
  • checkReachable names the endpoint and, for a loopback server, gives the local-cluster remedy ("start Docker Desktop, then k3d cluster start").
  • StatusUnknown renders as a neutral · line (no remedy).

Before → after (cluster down)

Before: 9 lines, ~6 wrong remedies (RBAC / StorageClass / "reinstall the chart").
After: Cluster reachable ✖ "the cluster API server at https://127.0.0.1:6550 isn't answering — is the cluster running?" → start Docker Desktop, then k3d cluster start; the rest render · <check> — could not check — cluster API unreachable.

Verification

TestRun_UnreachableCascade (fake clientset + connection-refused reactor) asserts the one ✖ names the endpoint + start remedy, the 7 cluster checks are StatusUnknown with no remedy, backend egress still ok, and Worst()=Fail (exit code preserved). gofmt/vet clean; doctor + cli packages green.


Note

Low Risk
Read-only diagnostic UX and verdict logic only; reachable clusters keep the previous per-check behavior, covered by new tests.

Overview
When the Kubernetes API cannot be reached (e.g. local cluster stopped), tracebloc doctor no longer runs every downstream check and surfaces bogus fixes (PVC, chart reinstall, proxy). It now treats transport failures as a single root cause and keeps the rest honest.

doctor package: Adds StatusUnknown (ignored by Worst() so exit codes stay driven by real ✖/⚠). isUnreachable() separates dial/timeout/DNS errors from cases where the API answered (missing chart, RBAC). Run() short-circuits on unreachable APIs: one Cluster reachable failure, cluster-dependent checks as neutral “couldn’t check”, while Backend egress still runs from the CLI machine. checkReachable uses ServerURL to name the endpoint and suggests starting Docker/k3d for loopback servers.

CLI: Passes ServerURL into doctor options and prints StatusUnknown as a neutral info line (no remedy).

Tests cover the unreachable cascade and loopback transport messaging.

Reviewed by Cursor Bugbot for commit e5c9047. Bugbot is set up for automated code reviews on this repo. Configure here.

When the cluster API is unreachable (Docker/k3d stopped, wrong server,
network), doctor ran all 8 downstream checks anyway and each invented a
domain-specific cause with the wrong fix: PVC "usable StorageClass", proxy
"chart too old", requests-proxy "not found -> reinstall the chart", pods/RBAC
"kubeconfig user can list pods". The one true cause (cluster stopped) was
buried under ~6 false verdicts. Root mechanism: findDeployment/jobsManagerEnv
collapse an API transport error and genuine absence into the same nil/empty.

- Add StatusUnknown ("could not check") — Worst() ignores it, so it never
  affects the verdict/exit code.
- isUnreachable(err): a transport/connectivity error (connection refused,
  timeout, DNS, TLS; net.Error/url.Error) that is NOT ErrNoParentRelease and
  NOT an RBAC/NotFound status (those mean the API answered).
- Run() short-circuits on isUnreachable(relErr): emit the one honest
  "Cluster reachable" ✖ and mark the 7 cluster checks StatusUnknown.
  checkBackendEgress still runs (probes the backend from this machine, not the
  cluster API). Display order unchanged. When the API IS reachable, every check
  runs and reports exactly as before (isolated-failure messaging preserved).
- checkReachable names the endpoint ("the cluster API server at <url> isn't
  answering — is the cluster running?") and, for a loopback server, gives the
  local-cluster remedy ("start Docker Desktop, then `k3d cluster start`")
  instead of the kubeconfig/chart remedy.
- doctor renders StatusUnknown as a neutral · line (no ✖/⚠, no remedy).

Verified by TestRun_UnreachableCascade (fake clientset, connection-refused
reactor): one ✖ naming 127.0.0.1:6550 + start-cluster remedy, 7 StatusUnknown
with no remedy, backend egress still ok, Worst()=Fail (exit code unchanged).

Fixes #350. Part of epic tracebloc/backend#1142 (Theme A).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka LukasWodka self-assigned this Jul 20, 2026
@LukasWodka
LukasWodka requested a review from saadqbal July 20, 2026 11:56
@LukasWodka

Copy link
Copy Markdown
Contributor Author

@BugBot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e5c9047. Configure here.

LukasWodka added a commit that referenced this pull request Jul 21, 2026
…(cli#351 Part 1) (#356)

* fix(doctor): stop the cluster-down cascade of misleading remedies

When the cluster API is unreachable (Docker/k3d stopped, wrong server,
network), doctor ran all 8 downstream checks anyway and each invented a
domain-specific cause with the wrong fix: PVC "usable StorageClass", proxy
"chart too old", requests-proxy "not found -> reinstall the chart", pods/RBAC
"kubeconfig user can list pods". The one true cause (cluster stopped) was
buried under ~6 false verdicts. Root mechanism: findDeployment/jobsManagerEnv
collapse an API transport error and genuine absence into the same nil/empty.

- Add StatusUnknown ("could not check") — Worst() ignores it, so it never
  affects the verdict/exit code.
- isUnreachable(err): a transport/connectivity error (connection refused,
  timeout, DNS, TLS; net.Error/url.Error) that is NOT ErrNoParentRelease and
  NOT an RBAC/NotFound status (those mean the API answered).
- Run() short-circuits on isUnreachable(relErr): emit the one honest
  "Cluster reachable" ✖ and mark the 7 cluster checks StatusUnknown.
  checkBackendEgress still runs (probes the backend from this machine, not the
  cluster API). Display order unchanged. When the API IS reachable, every check
  runs and reports exactly as before (isolated-failure messaging preserved).
- checkReachable names the endpoint ("the cluster API server at <url> isn't
  answering — is the cluster running?") and, for a loopback server, gives the
  local-cluster remedy ("start Docker Desktop, then `k3d cluster start`")
  instead of the kubeconfig/chart remedy.
- doctor renders StatusUnknown as a neutral · line (no ✖/⚠, no remedy).

Verified by TestRun_UnreachableCascade (fake clientset, connection-refused
reactor): one ✖ naming 127.0.0.1:6550 + start-cluster remedy, 7 StatusUnknown
with no remedy, backend egress still ok, Worst()=Fail (exit code unchanged).

Fixes #350. Part of epic tracebloc/backend#1142 (Theme A).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): requests-proxy check — result-egress framing + honest ✔ (Part 1)

The "Service Bus egress (requests-proxy)" check had the direction backwards and
overclaimed its ✔:
- It said a down requests-proxy makes experiments "stay Pending". Wrong path:
  requests-proxy is the OUTBOUND relay (training pods POST epoch results/FLOPs
  → it forwards to Service Bus). Scheduling is a separate inbound path
  jobs-manager consumes directly. A down proxy stalls result/weights egress
  MID-RUN; it does not block scheduling. The old remedy misroutes triage.
- The ✔ is inferred from Deployment ReadyReplicas, but requests-proxy has no
  readiness probe (backend#1143), so "Ready" only means the container started,
  not that Service Bus egress works.

Reword the doc comment + all three lines: missing/not-ready remedies now say
"result egress stalls mid-run (scheduling unaffected)"; the OK detail says
"deployment-ready only; egress not directly probed". Status stays OK on a
healthy install (no new ⚠, exit code unchanged). Name const kept for now.

TestCheckRequestsProxy_Wording locks it: no line says "Pending", and the ✔
admits egress isn't probed.

Part 2 (a real in-cluster Service Bus-egress probe) is deferred — it needs the
requests-proxy /healthz + readiness probe (backend#1143 + a client-runtime
health endpoint) that don't exist yet.

Fixes #351 (Part 1). Part of epic tracebloc/backend#1142.
Stacked on #354 (fix/doctor-reachability-gate) — merge that first.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodka added a commit that referenced this pull request Jul 21, 2026
…ete fixes

Rebuilds `tb doctor` around the one question a workspace owner has: is my
secure environment connected to tracebloc and ready to run training — and if
not, exactly what to do. The 9 Kubernetes-flavoured checks (release/chart/PVC/
pods/pull-secrets/proxy/Service-Bus/kubectl) collapse into two plain lines, and
everything technical moves behind --verbose.

Default view:

  Signed in as lukas@tracebloc.io
  Secure environment "lukas-test"

  Connected to tracebloc
  Ready to run training

- No banner, no kubeconfig block, no numbers, no Kubernetes vocabulary.
- Each health line expands on failure to the specific plain problem + ONE
  concrete action (a command, never a kubectl): stopped env -> "start Docker
  Desktop"; expired -> "tb login"; no env -> the one-line installer; results
  can't flow / component down -> email support with `tb doctor --diagnose`.
- Not-connected degrades readiness honestly to "can't check" (the #354
  reachability gate, folded in here).
- Remedies resolve `tb` vs `tracebloc` like the home screen.
- `tb doctor --verbose` keeps the full technical breakdown (kubeconfig +
  every granular check) for support.
- `tb doctor --diagnose` now owns the redacted support bundle (moved off
  install-k8s.sh, which the user may not have on disk).

summarizeDoctor (the roll-up) is unit-tested for every failure bucket; the
render + verdict are covered via the doctorRunFn seam; no k8s vocabulary is
allowed to leak into the default view (asserted).

Supersedes #354 (doctor cascade) and #351 (requests-proxy wording) — this
rewrites the same output and folds in their fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Superseded and already shipped. #356 was stacked on this branch, so its merge to develop carried this reachability gate in full (StatusUnknown / checkReachable / isUnreachable / isLoopback / the unreachable short-circuit). The doctor redesign (#365) has been rebased onto that develop and no longer cherry-picks this branch. Closing as superseded — no code lost. (Branch left intact.)

@LukasWodka LukasWodka closed this Jul 21, 2026
LukasWodka added a commit that referenced this pull request Jul 21, 2026
…xes, --diagnose) (#365)

* feat(doctor): plain-language redesign — connected + ready, with concrete fixes

Rebuilds `tb doctor` around the one question a workspace owner has: is my
secure environment connected to tracebloc and ready to run training — and if
not, exactly what to do. The 9 Kubernetes-flavoured checks (release/chart/PVC/
pods/pull-secrets/proxy/Service-Bus/kubectl) collapse into two plain lines, and
everything technical moves behind --verbose.

Default view:

  Signed in as lukas@tracebloc.io
  Secure environment "lukas-test"

  Connected to tracebloc
  Ready to run training

- No banner, no kubeconfig block, no numbers, no Kubernetes vocabulary.
- Each health line expands on failure to the specific plain problem + ONE
  concrete action (a command, never a kubectl): stopped env -> "start Docker
  Desktop"; expired -> "tb login"; no env -> the one-line installer; results
  can't flow / component down -> email support with `tb doctor --diagnose`.
- Not-connected degrades readiness honestly to "can't check" (the #354
  reachability gate, folded in here).
- Remedies resolve `tb` vs `tracebloc` like the home screen.
- `tb doctor --verbose` keeps the full technical breakdown (kubeconfig +
  every granular check) for support.
- `tb doctor --diagnose` now owns the redacted support bundle (moved off
  install-k8s.sh, which the user may not have on disk).

summarizeDoctor (the roll-up) is unit-tested for every failure bucket; the
render + verdict are covered via the doctorRunFn seam; no k8s vocabulary is
allowed to leak into the default view (asserted).

Supersedes #354 (doctor cascade) and #351 (requests-proxy wording) — this
rewrites the same output and folds in their fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): classify reachability + count image-pull in readiness

Addresses the Lint failure and both Bugbot findings on #365.

- Deadcode (Lint): the redesign derives the exit code in the cli layer from
  the two rolled-up health lines (worseStatus), so doctor.Worst is no longer
  reachable from the binary. Delete it (the allowlist's own guidance is
  "prefer deleting"); the two Run() tests that used it as an overall-verdict
  assertion keep a small test-local worstStatus helper.

- Bugbot HIGH — no-chart misclassified as offline: a reachable cluster with
  no tracebloc installed was reported as "isn't answering" and handed the
  kubectl remedy, and readiness never appeared. checkReachable now tags the
  "Cluster reachable" result with a ReachState (Unreachable / NoEnv / Error);
  summarizeDoctor words each failure from that class — NoEnv -> "No secure
  environment installed here" + the one-line installer (never a kubectl),
  Error -> support, Unreachable (and any unclassified fail) -> "isn't
  answering". Legacy hand-built results default safely to Unreachable.

- Bugbot MEDIUM — image-pull skipped in readiness: a failing "Image pull
  secret" check now rolls up to "Not ready — the training images can't be
  pulled" instead of a false Ready + exit 0.

summarizeDoctor gains table cases for the no-env and image-pull buckets (and
a guard that the no-env remedy never leaks kubectl). gofmt/vet/lint/deadcode/
file-budget green; doctor + cli packages pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): treat 0.0.0.0 / host.docker.internal as a local cluster

Bugbot MEDIUM on #365: isLoopback only matched localhost / 127.0.0.0/8 / ::1,
so a stopped k3d cluster whose kubeconfig advertises 0.0.0.0 (the host k3d
writes when the installer pins no explicit --api-port host — the bash
install-k8s.sh path, unlike the Windows .ps1 which pins 127.0.0.1:6550) fell
through to the generic "check your secure environment is running" line instead
of the redesign's main local-failure remedy, "Start Docker Desktop".

isLoopback now also treats the unspecified/wildcard bind addresses (0.0.0.0,
::) and Docker Desktop's host alias (host.docker.internal) as local. None of
these is ever a genuinely-remote endpoint, so there's no false-positive risk.

Tests: a dedicated TestIsLoopback table (local vs remote hosts) and a
checkReachable assertion that a stopped 0.0.0.0 endpoint gets the start-Docker
remedy + ReachUnreachable. Also locks in the doctor-side reach classification
that feeds the summary rollup — ReachNoEnv for a missing chart, ReachError
otherwise.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): don't blame the network for a server error; gate readiness on connected

Two Bugbot findings on #365:

- WhoAmI misclassification: every non-401/426 error set tokenReachable=false,
  so a backend that ANSWERED with an error (5xx/403/decode) was rolled up as
  "can't reach tracebloc from here" with a network/proxy remedy — contradicting
  a green Backend-egress check and exiting 2 for a tracebloc-side problem.
  Split the WhoAmI outcome into a tokenState: tokenUnreachable (genuine
  transport failure -> network remedy) vs tokenServerErr (the backend answered,
  just not 200 -> "tracebloc didn't confirm your session (server error)", retry
  / email support, never a proxy remedy).

- False readiness check: readiness only degraded to "can't check" when Cluster
  reachable failed, so other Connected failures (Service Bus down, backend/token
  unreachable, server error) still allowed a green "Ready to run training" next
  to a Connected ✖ — even though training can't complete while disconnected.
  Readiness now degrades whenever Connected is not OK.

Tests: TestSummarizeDoctor gains a tokenServerErr case (asserts support remedy,
not a network/proxy one) and a "disconnected but cluster healthy" case (ready
must not be a green check under a Connected ✖). All 8 existing call sites move
from the bool to the tokenState enum.

gofmt/vet/lint/deadcode green; doctor + cli packages pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): stop early returns from dropping detected signals

Two Bugbot findings on #365, both the same anti-pattern — an early return
throwing away something doctor already learned:

- --diagnose skipped the roll-up: it returned right after writing the bundle,
  so the file recorded only the raw k8s checks (never the WhoAmI/session outcome
  or the Connected/Ready verdict), and it always exited 0 even when checks
  failed — while the remedies tell owners to email that very bundle for
  session/server errors. --diagnose is now additive: it runs summarizeDoctor,
  writes session + verdict + detail into the bundle, and falls through to the
  real verdict/exit code (2 when something's wrong, not a misleading 0).

- Kubeconfig/clientset failure hid a session fault: after WhoAmI set
  tokenUnreachable/tokenServerErr, a failed local-env read returned with only
  "no / can't connect to secure environment", steering owners to reinstall
  while a live backend/session problem went unmentioned. Both no-env branches
  now surface the session fault first via noteSessionProblem (a no-op when the
  session is fine; 401/426 remain hard stops upstream).

Adds tokenLabel (bundle session line) + a TestRunClusterDoctor_DiagnoseBundle
that asserts the bundle records session/connected/ready and that a failing
check still exits 2 under --diagnose.

gofmt/vet/lint/deadcode green; doctor + cli packages pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): don't let the indicative backend-egress probe contradict WhoAmI

Bugbot on #365: the Connected roll-up ORed tokenUnreachable with a failing
"Backend egress (from this machine)" check, so after a successful WhoAmI a
backend-egress probe miss (a different CLIENT_ENV host, a transient miss) still
rendered "Not connected — can't reach tracebloc from here" + a check-your-network
remedy, contradicting the session probe that had just reached the backend.

WhoAmI is the definitive from-this-machine reachability+auth signal; the
backend-egress check is explicitly indicative-not-definitive (it probes the
cluster's configured host from here, not the cluster's real egress path). So it
no longer feeds the Connected headline — it stays a --verbose/--diagnose
diagnostic. Connected now keys on tok (+ reachability + Service Bus egress).

Test: a backend-egress miss with a healthy session keeps Connected OK and never
prints the network remedy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): --diagnose always leaves a bundle; never masks the verdict exit

Two Bugbot findings on #365, one root cause — early returns bypassing the inline
--diagnose block:

- Diagnose was only honored after the full probe, so every early exit (soft
  session fault, no environment, clientset error) wrote no bundle — the exact
  states whose remedies tell the user to run `doctor --diagnose`.
- If checks failed (exit 2) but the bundle write errored, the run returned the
  write failure (exit 3), masking the health verdict.

Replace the inline write with a deferred writer registered right after the
sign-in gate: it fires on every exit path (so --diagnose always leaves a bundle
once authenticated — "not signed in" is still answered by login, not a bundle),
and via the named return it only sets the exit code when nothing worse already
did, so a bundle hiccup never masks a Fail verdict. writeDiagnoseBundle records
partial state (session + an "exited before probe" note) when the roll-up didn't
run and skips the k8s section when there's no resolved environment.

Tests: --diagnose on a clientset-fail early exit still writes a bundle and
preserves exit 3.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): no --diagnose bundle claiming a confirmed session on 401/426

Bugbot on #365: the deferred bundle writer was registered before the WhoAmI
401/426 hard stops, which return without updating tok. A bundle written on an
expired (401) or upgrade-required (426) session therefore recorded
"session: confirmed" — misleading triage.

Move the defer registration to after the session probe: 401/426 return first
(their fix is login/update, not a bundle), and every path past there has an
accurate tok, so --diagnose still leaves a truthful bundle on the soft-fault /
no-environment / clientset-error exits.

Test: a 401 + --diagnose exits 2 and writes no bundle file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): a session fault exits 2 even when local config also fails

Bugbot on #365: on the no-environment / clientset-error early exits, a Fail-level
session state (tokenUnreachable / tokenServerErr) that noteSessionProblem had just
printed was still returned as exitLocalEnv (3) — while the same session states
exit 2 (exitChecksFailed) on the full probe path. A script keying on
"exit 2 = a problem was found" would miss the session fault whenever local config
also failed.

earlyExitCode(tok) now returns exit 2 when a session/connectivity fault was
detected, else the local-env code (3), so the exit code is consistent across paths.

Test: a transport-error session (tokenUnreachable) + no environment exits 2 and
surfaces both the session fault and the no-environment line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): don't name a secure environment that isn't installed

Bugbot on #365: the identity header printed `Secure environment "<name>"` from the
resolved kubeconfig namespace before the probe. When the cluster is reachable but
no tracebloc chart is installed (ReachNoEnv), the Connected line then says none is
installed here — so the default view asserted the environment both exists and
does not.

Move the header to after the roll-up and print it only when an environment is
actually installed (reach state != ReachNoEnv). Nothing prints between it and
"Signed in as …", so the two context lines still read as a pair. Every other reach
state (running, stopped, RBAC) means an environment exists, so it's still named.

Test: a ReachNoEnv run shows "No secure environment installed here", never names
one, and exits 2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): unify the three "no environment" exits; surface session faults on all

Bugbot on #365: a session fault (tokenUnreachable / tokenServerErr) was hidden on
the reachable-but-no-chart (ReachNoEnv) path, because that path flowed through
summarizeDoctor whose Connected line ignores tok — unlike the missing-kubeconfig
and clientset-error exits, which surface it via noteSessionProblem.

Route ReachNoEnv through the same short-circuit as the other two "no environment"
states: noteSessionProblem (surfaces any session fault) + "No secure environment
on this machine yet" + the installer + earlyExitCode (a session fault dominates
the exit). Remove the now-redundant ReachNoEnv case from summarizeDoctor — one
source of truth. This also preserves the earlier fix (never name an environment
that isn't installed): the short-circuit returns before the identity header.

Tests: ReachNoEnv with a healthy session exits 3 and names no environment; with a
session fault it surfaces the fault and exits 2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): --verbose details + honest bundle outcome on early exits

Two Bugbot findings on #365, both from the no-environment short-circuits
returning before the main render:

- --verbose (Medium): renderDoctorDetails only ran on the full path, so
  `tb doctor --verbose` printed no Details on the ReachNoEnv / clientset failure
  paths — exactly where support needs the kubeconfig + granular output. Extracted
  renderDetailsIfVerbose and call it at every exit that has a resolved config.

- --diagnose bundle (Low): on ReachNoEnv the roll-up never ran, so the bundle
  recorded "exited before the environment could be probed" even though the checks
  WERE collected and written below. The outcome line is now three-way: verdict
  when summarized, "no roll-up verdict (granular checks below)" when probed but
  not summarized, and "before the cluster was probed" only when truly not probed.

Test: --verbose on a ReachNoEnv run prints the Details section with the checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): stuck-Pending pods roll up to Not-ready, not a false green

summarizeDoctor only degraded the "Ready to run training" verdict on Pod-health
StatusFail (crash-loop). Pods stuck Pending past the grace window surface as
StatusWarn, so an environment where training can't schedule still rolled up to
✔ "Ready to run training" and the "Everything looks good" verdict. Treat
Pod-health Warn as not-ready (training genuinely can't schedule) with a
compute / image-pull remedy, and cover it with a regression test.

Bugbot (PR #365): "Warn pod states claim ready".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): distinguish the two Pod-health warn sources in the readiness rollup

checkPods returns StatusWarn for BOTH stuck-Pending pods AND a failure to list
pods (e.g. RBAC). The previous fix gave the stuck-pending / Docker-Desktop remedy
to both, misdiagnosing the read-failure case. Split them: "could not list pods"
now rolls up to an honest can't-check (StatusUnknown, no compute remedy), while
stuck-Pending stays not-ready with the compute/image-pull remedy. Both covered.

Bugbot (PR #365): "Pod warn misclassified as pending".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(doctor): don't claim "everything looks good" when readiness is unverified

The RBAC / can't-list-pods case rolls readiness up to StatusUnknown ("couldn't
check your workloads"), but worseStatus treats Unknown as non-worsening, so the
closing verdict still printed "Everything looks good — you're ready to run
training" (exit 0), contradicting that line. Extract the decision into
doctorVerdict: "everything looks good" now requires BOTH connected and ready to
be genuinely StatusOK; a can't-check reports an honest partial result instead
(still exit 0, since nothing failed). Unit-tested.

Bugbot (PR #365): "Can't-check still claims ready".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@LukasWodka
LukasWodka deleted the fix/doctor-reachability-gate branch July 21, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant