ci(deploy): report the deploy that happened, not the cluster's worst pod (#835)

[lilyshen0722]

Closes #835.

Every Deploy Dev run on 2026-08-04 reported failure while deploying successfully — five consecutively, helm revisions 420–424 all recorded failed.

What was actually wrong

--wait waits for every resource in the release. Two workloads in the chart have been unready for days:

workload	state
litellm	CrashLoopBackOff, 561 restarts over 2d10h
cloud-codex-cody	Init:CrashLoopBackOff

Neither will ever become ready, so --wait burned its full 10m timeout and exited non-zero — while backend, frontend, clawdbot-gateway and commonly-bot had all rolled out correctly minutes earlier. Verified live: those four pass kubectl rollout status immediately; the other two never do.

The error is misleading, and I fell for it first. It surfaces as:

client rate limiter Wait returned an error: context deadline exceeded

That reads like throttling. It isn't — the release already runs --burst-limit 200 --qps 100 (#245, added long ago). It is the context deadline expiring while waiting on pods that cannot converge. Raising the limiter, which is the obvious fix and the first one I proposed on the issue, could never have worked. Worth recording because the next person will read that string the same way.

Why it mattered more than a red check

helm records a failed upgrade, so the newest revision helm considers deployed went stale by days. A bare helm rollback commonly-dev during an incident would have silently reverted every deploy since Aug 2 — including both auth fixes (#817/#821) and the heartbeat opt-in (#833). Rollback safety depends on this job telling the truth.

The secondary cost is desensitisation: five false failures in one day is enough to make a real one invisible.

The change

Drop --wait; verify exactly the four workloads whose image tags this job sets.

Scoping is the whole point. An unrelated unhealthy workload is a real problem, but it is not this job's failure, and conflating the two is what made the signal worthless. kubectl rollout status still returns non-zero on a genuinely stuck rollout, so a real regression fails as loudly as before — it just no longer fails on things the job never touched. Per-deployment timeouts, because clawdbot-gateway carries two init containers and legitimately takes minutes longer than the rest.

Unhealthy workloads outside the deployed set are surfaced as a ::notice:: rather than failed on. Silence is how litellm reached 561 restarts unnoticed; failing on it is how five good deploys got marked bad. A notice is the correct middle.

Deliberately not included

Setting litellm.enabled and agents.cloudCodex.enabled to false in values-dev.yaml. Both are enabled: true while dead, and turning them off would also fix this — but that is a decision about the native runtime and the cloud-agent tier, not a CI fix. It needs an owner's call, and it would be doing it for the wrong reason (to make CI green) rather than the right one (nobody wants them running). Flagged separately.

Verification

Both halves exercised against the live cluster before committing:

• the unhealthy-workload query returns exactly cloud-codex-cody litellm
• the four-deployment gate passes on all four, so the job would now correctly report success for the very deploy that reported failure

Workflow YAML re-parsed after editing; step order and the removal of --wait asserted programmatically.