Skip to content

fix(session): degrade fallback-unavailable 429s to in-turn retry - #744

Merged
code-yeongyu merged 3 commits into
mainfrom
fix/429-no-fallback-in-turn-retry
Aug 6, 2026
Merged

fix(session): degrade fallback-unavailable 429s to in-turn retry#744
code-yeongyu merged 3 commits into
mainfrom
fix/429-no-fallback-in-turn-retry

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Problem

A 429-class provider failure with no parseable retry-after hint routes to the no-hint-fast-fallback tier, which skips same-model retries and goes straight to the fallback chain. When no fallback candidate is usable (no chain configured for the model - the shipped defaults only cover anthropic/claude-fable-5 - chain exhausted, candidates cooling, or unauthenticated), the turn died instantly with auto_retry_end { attempt: 0 }:

Retry failed after 0 attempts: 429: {"message":"The model is temporarily at capacity. Please retry shortly.","type":"rate_limit_error",...,"code":"server_overloaded"}

Reproduced twice on 2026-08-06 against a wafer custom provider (request_ids 6992be6e959f, 94164e8af5fa). The provider literally asks for a retry and senpi performed zero. The same instant death existed in the tier2/tier3 hinted branches.

Design

Informed by comparative analysis of sst/opencode and openai/codex (fresh checkouts, file:line-cited reports):

  • opencode: session-level in-turn retry with visible countdown, no attempt cap, honors retry-after; no model fallback at all.
  • codex: bounded turn replays (default 5) including in-stream rate-limit errors with parsed "try again in Xs" delays; no model fallback.

senpi keeps its fallback-first advantage (unique among the three); this change makes fallback UNAVAILABILITY degrade to what both references always do - retry the same model in-turn:

  • no-hint 429 + no usable fallback -> in-turn exponential retries under the settings.retry budget
  • tier2 hint + no usable fallback -> in-turn wait clamped to hintedWaitCapMs
  • tier3 (>= probeBackMaxMs) -> still terminal, but the final error now names the provider-requested wait in seconds
  • budget exhaustion reports the TRUE attempt count
  • chain-available behavior is unchanged (fast-fallback stays the first move)

Changes

  • packages/coding-agent/src/core/retry-fallback/hint-policy.ts: pure degradeWithoutFallback policy
  • packages/coding-agent/src/core/agent-session.ts: _degradeRateLimitedWithoutFallback replaces both instant-death branches
  • packages/coding-agent/src/core/changes.md: fork-change entry
  • tests: test/suite/retry-fallback-no-chain-degrade.test.ts (4 session-level tests, captured RED on the old behavior first), pure policy tests in retry-fallback-hint-policy.test.ts (mutation-proofed)
  • senpi-qa: new mock-loop scenario no-hint-429-no-chain

Verification

  • RED -> GREEN: the new suite failed 4/4 against the old behavior, passes after the change
  • Mutation proof: the 3 pure policy tests fail under a policy mutation and pass after restore
  • 21 related test files / 212 tests green in a single run
  • root npm run check green
  • Real-CLI QA (senpi-qa mock-loop RPC channel): no-hint-429-no-chain 5/5 PASS, no-hint-429-fast-fallback 5/5 PASS (regression), hinted-429-in-turn 6/6 PASS; evidence under local-ignore/qa-evidence/20260806-retry-fallback/

Plan: .omo/plans/retry-fallback-degrade.md


Summary by cubic

Prevented “Retry failed after 0 attempts” on 429s with no usable fallback by retrying the same model in-turn. Tier2 hints are clamped, and tier3 stays terminal while naming the provider-requested wait.

  • Bug Fixes
    • No-hint 429 + no usable fallback now retries in-turn with exponential backoff under settings.retry; tier2 hinted waits retry in-turn clamped to hintedWaitCapMs; tier3 (>= probeBackMaxMs) stays terminal and names the requested wait in seconds.
    • Fast-fallback behavior is unchanged when a fallback chain exists.
    • Added degradeWithoutFallback policy in packages/coding-agent/src/core/retry-fallback/hint-policy.ts and routed both former instant-death branches through _degradeRateLimitedWithoutFallback in packages/coding-agent/src/core/agent-session.ts to report true attempt counts on exhaustion.
    • New tests packages/coding-agent/test/suite/retry-fallback-no-chain-degrade.test.ts and policy tests; QA scenario no-hint-429-no-chain added to mock loop; changelog updated in packages/coding-agent/CHANGELOG.md.

Written for commit b61b33a. Summary will update on new commits.

Review in cubic

A 429-class failure routed to fallback (no-hint, tier2, tier3) died with
auto_retry_end { attempt: 0 } when no fallback candidate was usable,
surfacing "Retry failed after 0 attempts" on the first hint-less 429 for
any model without a chain. No-hint failures now retry in-turn on the
ordinary exponential schedule, tier2 hinted waits retry in-turn clamped
to hintedWaitCapMs, and tier3 waits stay terminal but name the
provider-requested wait. Budget exhaustion reports the true attempt
count. Informed by sst/opencode (unbounded visible in-turn retry) and
openai/codex (bounded turn replays) retry analyses.
The mock-loop hint-429 channel only proved the chain-available paths, so
the instant-death no-chain path shipped unobserved. The new
no-hint-429-no-chain scenario runs the real CLI over RPC with no
fallback model and no chain, and asserts bounded exponential in-turn
retries recover the turn on the primary model.
@code-yeongyu
code-yeongyu merged commit c65abce into main Aug 6, 2026
15 checks passed
@code-yeongyu
code-yeongyu deleted the fix/429-no-fallback-in-turn-retry branch August 6, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant