feat(llm): shared status-error mapper + typed model-403s (ENG-598) - #236
Conversation
A gateway 403 for a tier-locked/disabled model surfaced in chat as
"An unexpected error occurred: Server returned 403 — the LLM endpoint may be
temporarily unavailable. Try again in a moment.. Please try again or rephrase
your request." — wrong cause, useless advice, double-wrapped. Two layers:
1) openai.py mapped every non-401/429 APIStatusError to the generic
"temporarily unavailable" copy, in four copy-pasted blocks (complete /
stream / _complete_via_responses / _stream_via_responses) that had already
drifted in wording.
2) session.py's plan_stream fallback swallowed the exception and yielded it
as assistant TEXT, so the turn "succeeded" and cowork-server's turn-error
mapping (the error-card machinery) never ran.
Fix:
- One shared _raise_for_status_error(exc, model) used by all four call paths
so the mapping can't drift. 401 and 429 behavior unchanged (same copy the
downstream provider_auth / token_limit detections key on; the "or to top
up" wording drift normalized).
- New ModelUnavailableError(ConnectionError) in provider.py carrying
{code, model} for the gateway's structured 403s: model_access_denied →
plan copy with upgrade/switch guidance; model_disabled → hedged copy
(can be tier lock or admin kill switch until ENG-596's config lands).
Detection is code-exact on error.code — BYOK OpenAI 403s, Anthropic
permission errors, and Cloudflare HTML 403s carry no such code and fall
through to the generic message, never the plan copy. Subclassing
ConnectionError keeps every legacy call site working unchanged.
- session.py: TokenLimitExceeded / ModelUnavailableError now PROPAGATE from
the plan_stream fallback instead of being wrapped into prose, so the turn
fails and the server can render an actionable card. All other exceptions
keep the existing text fallback.
Tests: 11 new (401 copy pin, 429 quota + precedence over model-403, both
gateway codes incl. attrs/copy, ConnectionError compatibility, BYOK/HTML/
codeless 403 fall-through, 500 generic). Suite: 1147 passed; the 2
test_chat_scratchpad failures pre-exist on clean staging (env-dependent).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…okenLimitExceeded A model-gate 403 is deterministic: the recovery loop would re-send the same doomed request twice (burning the retry budget + user-visible delay) before the fallback path finally propagated it. Short-circuit at the outer catch, exactly like TokenLimitExceeded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lucas-koontz
left a comment
There was a problem hiding this comment.
Reviewed as Part 1 of the ENG-598 chain (with cowork-server#168 and cowork#353). The core is solid: one shared _raise_for_status_error across all four call paths kills the copy-paste drift, code-exact 403 detection (never bare-status or string matching) is the right call, 429-before-403 precedence is correct, and the session.py change to propagate ModelUnavailableError/TokenLimitExceeded instead of wrapping them as assistant text is exactly what makes the downstream card possible. Test coverage is genuinely thorough (BYOK region 403, Cloudflare HTML body, no-code 403, quota-wins-over-403). Approving.
One thing worth fixing, and two nits — none blocking:
Should fix — the CLI now defaults tier-lock errors to "retry" (chat.py:1889, not in this diff). ModelUnavailableError subclasses ConnectionError, and the CLI turn handler does default="retry" if isinstance(exc, ConnectionError) else "setup". So a model_access_denied (a deterministic plan gate) now highlights retry as the default action — the user hits Enter, re-sends the identical doomed request, and fails the same way. This directly contradicts the comment you added in session.py ("deterministic — the auto-retry below would just re-send the same doomed request"). Since you're already importing the type, consider default="setup" if isinstance(exc, ModelUnavailableError) else ("retry" if isinstance(exc, ConnectionError) else "setup") so the interactive default steers toward switching models. (model_disabled is hedged, so retry-as-default there is more defensible.)
Nits: the 429 docstring bullet is grammatically broken (inline comment), and _raise_for_status_error is typed -> None but always raises — typing.NoReturn would document/enforce that the four except sites can rely on it. The NoReturn one is truly optional (this repo runs no type checker in CI, and the except sites do fall through to a raise today).
| permission errors, and Cloudflare HTML 403s carry no such code and | ||
| must fall through to the generic message, never the plan copy. | ||
| - 429 with a quota detail → TokenLimitExceeded (checked before the 403 | ||
| branch can't shadow it — quota keeps its own card downstream). |
There was a problem hiding this comment.
Grammar: "checked before the 403 branch can't shadow it" is malformed. The 429 and 403 branches gate on mutually-exclusive status_code values, so ordering can't cause shadowing between statuses anyway — the precedence that actually matters is only for a body carrying BOTH detail and error.code on the same status (your test_429_with_detail_wins_even_if_error_code_present pins exactly that). Suggest: "429 with a quota detail → TokenLimitExceeded, checked first so a body carrying both detail and error.code stays token_limit."
There was a problem hiding this comment.
Fixed in 9dd769b — reworded to your suggestion: "429 with a quota detail → TokenLimitExceeded, checked first so a body carrying both detail and error.code stays token_limit."
| ) | ||
|
|
||
|
|
||
| def _raise_for_status_error(exc: "openai.APIStatusError", model: str) -> None: |
There was a problem hiding this comment.
Optional: this always raises on every branch, so -> None slightly understates the contract that the four except APIStatusError: _raise_for_status_error(...) sites depend on (they dereference response right after). -> typing.NoReturn documents "never returns normally" and would let a strict checker catch a future non-raising branch that leaves response unbound. No functional change; skip if you'd rather not introduce NoReturn (it's used nowhere else here).
There was a problem hiding this comment.
Done — typed -> NoReturn in 9dd769b (imported from typing). Documents the contract the four except-sites rely on.
…pper doc/type nits Addresses lucas-koontz's review on PR #236 (all non-blocking; PR approved): - CLI turn handler (chat.py) defaulted to "retry" for any ConnectionError, and ModelUnavailableError subclasses ConnectionError — so a deterministic plan gate highlighted "retry" as the default, re-sending the same doomed request (contradicting the session.py comment). Now defaults to "setup" (switch model/provider) for ModelUnavailableError; transient ConnectionErrors keep retry. - Fixed the grammatically-broken 429 docstring bullet in _raise_for_status_error. - _raise_for_status_error typed -> NoReturn (it always raises); documents the contract the four except-sites rely on. Mapper + session tests: 11 passed. (Pre-existing F541s elsewhere in chat.py left untouched — not in scope.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks @lucas-koontz — pushed
Mapper/session tests green (11). |
|
Re-checked the polish — CLI default now branches |
Part 1 of 3 for ENG-598 (with cowork-server and cowork companion PRs).
Problem
A gateway 403 for a tier-locked/disabled model surfaced in chat as:
Wrong cause, useless advice, double-wrapped. Two layers: (1)
openai.pymapped every non-401/429APIStatusErrorto the generic copy, in four copy-pasted blocks that had already drifted in wording; (2)session.py's plan_stream fallback swallowed the exception into assistant text, so the turn "succeeded" and cowork-server's error-card machinery never ran.Fix
_raise_for_status_error(exc, model)used by all four call paths (chat/stream × completions/responses) — the mapping can't drift again. 401 and 429 behavior byte-compatible (same copy the downstreamprovider_auth/token_limitdetections key on).ModelUnavailableError(ConnectionError)carrying{code, model}for the gateway's structured 403s (error.codeofmodel_access_denied/model_disabled):model_access_denied→ "isn't included in your current MindsHub plan — upgrade or switch models in Settings"model_disabled→ hedged copy (can be tier lock or admin kill switch until mindshub-inference#335 / ENG-596 lands)ConnectionErrorkeeps every legacy call site working unchanged.session.py:TokenLimitExceeded/ModelUnavailableErrornow propagate from the plan_stream fallback instead of being wrapped into prose — the turn fails, so the server can render an actionable card. All other exceptions keep the existing text fallback.Tests
11 new in
tests/test_status_error_mapper.py: 401 copy pin, 429 quota (+ precedence over model-403), both gateway codes (attrs + copy), ConnectionError compatibility, BYOK/HTML/code-less 403 fall-through, 500 generic.Suite: 1147 passed. The 2
test_chat_scratchpadfailures pre-exist on clean staging (verified via stash — environment-dependent, fail identically without this diff).Ticket: ENG-535's sibling — ENG-598. Related: ENG-516 (this introduces the typed-exception pattern it asked for).
🤖 Generated with Claude Code