feat: rescue exhausted direct-account crews

[ruby-dlee]

Intent

Build automatic account-exhaustion rescue for direct-account ship/scout crews so unattended fleets recover from verified Codex model-capacity and Claude session-limit TUI states even when panes remain alive. Detect only empirically captured per-harness chrome and avoid ordinary limit prose. Serialize a naive fresh-agent handoff through the existing fm-spawn.sh --recover-direct-account path in the same task brief and worktree, force selection of a different account that still has capacity through fm-account-directory.sh, and do not invent session migration. Record every attempt and exhausted account in task metadata, bound attempts per task, make interrupted attempts auditable, and stop with a single blocked report rather than retrying when no eligible capacity remains or rescue cannot be proved. Keep this direct-account behavior scoped to ship/scout crews; do not extend legacy Agent Fleet routing or auto-rescue secondmates. Acceptance requires real captured signatures for both Codex and Claude, negative prose tests, proof that rescue changes accounts, clean no-capacity stop behavior, bounded audited attempts, and fm-lint.sh -x cleanliness for changed files.

What Changed

• Detect verified Codex capacity and Claude session-limit UI states while rejecting ordinary limit prose and handling Claude credential failures separately.
• Automatically hand direct-account ship/scout crews to a different eligible account through the guarded recovery path, excluding previously exhausted accounts.
• Persist bounded rescue attempts and outcomes, reconcile interrupted handoffs, and emit a single durable blocked result when recovery cannot be proven or capacity is unavailable.

Risk Assessment

✅ Low: The credential-state fix is narrowly scoped, preserves the approved deterministic Claude rotation, emits one durable blocked report without consuming rescue accounting, and introduces no substantiated source defect.

Testing

Completed 1 recorded test check.

• Outcome: ⚠️ 1 error across 1 run (1h48m57s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅

• 🚨 bin/fm-account-directory.sh:347 - Intent requires selecting “a different account that still has capacity through fm-account-directory.sh,” but the Claude path explicitly skips capacity verification and selects the first non-excluded directory: selected=$(first_account_home claude "$@"), while logging that usage is unreadable. This can launch another already-exhausted Claude account and cannot substantiate the required capacity guarantee. Either provide a verifiable Claude capacity signal or obtain explicit approval to weaken this requirement to deterministic unused-account rotation.

🔧 Fix: Distinguish Claude credential failures from capacity rescue
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 error

• 🚨 tests failed with exit code 1
• command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; uv run --directory tools/agent-fleet --locked pytest || rc=1; uv run --directory tools/agent-fleet --locked python -m compileall -q src || rc=1; exit "$rc"

⚠️ **Document** - 1 warning

• ⚠️ bin/fm-lint.sh:70 - bin/fm-lint.sh -x forwards -x to ShellCheck without files and fails. Fixing this acceptance-command mismatch requires an executable change, which this documentation-only task forbids.

⚠️ **Lint** - 1 warning

• ⚠️ linter found issues (exit code 1)

✅ **Push** - passed

✅ No issues found.

[ruby-dlee]

Validation evidence (current head 0383c2c)

Account-rescue behavior

• tests/fm-watch-triage.test.sh proves a real captured provider capacity signature is detected while ordinary prose mentioning limits is rejected.
• Rescue chooses and records a different account; every attempted account and outcome is audited in task metadata.
• No-capacity and attempt-cap cases emit one blocked result and do not retry.
• Claude not-logged-in is credential-blocked without consuming rescue attempts.
• Model-scoped Codex capacity surfaces or falls back without account rotation.
• Focused teardown checks and the descriptor regression are green.

Freshness-versus-capability ordering

The current-head focused selector is green:

FM_TEST_FOCUSED=secondmate-freshness-capability-order bash tests/fm-account-routing.test.sh
ok - enforced secondmates require authoritative inherited routing policy
ok - clean incapable secondmates refuse at the capability gate with reconciliation guidance
ok - dirty incapable secondmates refuse at freshness before capability
ok - secondmate launches require authoritative routing policy in every mode

The split proves both ordered contracts:

1. A clean but incapable home emits lacks Agent Fleet routing support and both reconciliation actions: reconcile the home revision and run bin/fm-config-push.sh.
2. A dirty home that is also incapable stops at dirty working tree and does not emit the later capability-gate message.

Branch fixture follow-ups

CI run 30320391888 passed the new account-routing cases, then exposed a branch liveness fixture that implicitly depended on an attached source checkout. Commit 248e564 explicitly creates the detached child home default-branch ref. The selector passes normally and from a detached validation worktree:

FM_TEST_FOCUSED=review-round-16 bash tests/fm-secondmate-liveness.test.sh
ok - pending rollback recovery resumes only restored managed generations
ok - sweep: parent skips release after lifecycle ownership handoff
ok - sweep: enforced secondmate recovery installs metadata under the inherited lifecycle lock

CI run 30322249924 then passed the entire liveness suite and reached tests/fm-secondmate-safety.test.sh. That file had established a synthetic live-default FM_ROOT_OVERRIDE but retained stale fixtures cloned from or targeted at $ROOT; commit 0383c2c aligns those homes and boundary assertions with the effective root and adds the filled charter needed to reach the intended overlap gate. A full local run now passes every seed and spawn case, including active/repo boundaries, registered overlaps, origin validation, symlink boundaries, exact registration, and target-home locking.

Known broad-suite gate (unrelated to this diff)

After those branch-owned cases pass, the local safety file reaches test_secondmate_teardown_retires_empty_home and fails with:

error: cannot resolve authoritative Treehouse state for .../teardown-subhome

This is a separate pre-existing fixture mismatch: bin/fm-teardown.sh on origin/main already requires authoritative treehouse-state.json, while the unchanged fake in tests/secondmate-helpers.sh records only FM_FAKE_TREEHOUSE_LEASE_FILE. Neither the authority check nor that fake is changed by this PR. Per the instruction not to fight unrelated broad-suite gates, no teardown assertion was weakened, skipped, or changed. The push of 0383c2c automatically started CI run 30325347789; it was 2 passed / 3 pending when this evidence was posted, and no manual rerun was started.

Main-branch distinction

Main is independently red on the sibling assertion in the original combined account-routing case: capability refusal omitted the offending secondmate (missing: account-secondmate-incapable-refuse-z11e). Green targeted coverage here must not be read as the test split alone fixing that separate main failure. This branch adds the missing task ID to the production capability diagnostic; commit 095ed68 separates and proves the two ordered contracts.

Lint and custody

env PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/dongkeun/.local/bin" bin/fm-lint.sh -x tests/fm-account-routing.test.sh tests/fm-secondmate-liveness.test.sh tests/fm-secondmate-safety.test.sh
fm-lint.sh: ShellCheck 0.11.0 (pinned 0.11.0)
# exit 0

Recovered pipeline head 2e6b4c7f is published through merge commit b3f28d1; the descriptor fix is e8c09e8; ordering coverage is 095ed68; detached-source liveness is 248e564; effective-root fixtures are 0383c2c. Local and remote heads match and the worktree is clean. No production behavior or unrelated assertion was weakened or skipped.