Skip to content

feat(llm): shared status-error mapper + typed model-403s (ENG-598) - #236

Merged
alecantu7 merged 3 commits into
stagingfrom
alejandrocantu/eng-598-model-403-mapper
Jul 9, 2026
Merged

feat(llm): shared status-error mapper + typed model-403s (ENG-598)#236
alecantu7 merged 3 commits into
stagingfrom
alejandrocantu/eng-598-model-403-mapper

Conversation

@alecantu7

Copy link
Copy Markdown
Contributor

Part 1 of 3 for ENG-598 (with cowork-server and cowork companion PRs).

Problem

A gateway 403 for a tier-locked/disabled model surfaced in chat as:

"An unexpected error occurred: Server returned 403 — the LLM endpoint may be temporarily unavailable. Try again in a moment.. Please try again or rephrase your request."

Wrong cause, useless advice, double-wrapped. Two layers: (1) openai.py mapped every non-401/429 APIStatusError to the generic copy, in four copy-pasted blocks that had already drifted in wording; (2) session.py's plan_stream fallback swallowed the exception into assistant text, so the turn "succeeded" and cowork-server's error-card machinery never ran.

Fix

  • One shared _raise_for_status_error(exc, model) used by all four call paths (chat/stream × completions/responses) — the mapping can't drift again. 401 and 429 behavior byte-compatible (same copy the downstream provider_auth / token_limit detections key on).
  • New ModelUnavailableError(ConnectionError) carrying {code, model} for the gateway's structured 403s (error.code of model_access_denied / model_disabled):
    • model_access_denied → "isn't included in your current MindsHub plan — upgrade or switch models in Settings"
    • model_disabled → hedged copy (can be tier lock or admin kill switch until mindshub-inference#335 / ENG-596 lands)
    • Detection is code-exact: BYOK OpenAI 403s (region blocks), Anthropic permission errors, and Cloudflare HTML 403s carry no such code and fall through to the generic message — never the plan copy. Subclassing ConnectionError keeps every legacy call site working unchanged.
  • session.py: TokenLimitExceeded / ModelUnavailableError now propagate from the plan_stream fallback instead of being wrapped into prose — the turn fails, so the server can render an actionable card. All other exceptions keep the existing text fallback.

Tests

11 new in tests/test_status_error_mapper.py: 401 copy pin, 429 quota (+ precedence over model-403), both gateway codes (attrs + copy), ConnectionError compatibility, BYOK/HTML/code-less 403 fall-through, 500 generic.

Suite: 1147 passed. The 2 test_chat_scratchpad failures pre-exist on clean staging (verified via stash — environment-dependent, fail identically without this diff).

Ticket: ENG-535's sibling — ENG-598. Related: ENG-516 (this introduces the typed-exception pattern it asked for).

🤖 Generated with Claude Code

A gateway 403 for a tier-locked/disabled model surfaced in chat as
"An unexpected error occurred: Server returned 403 — the LLM endpoint may be
temporarily unavailable. Try again in a moment.. Please try again or rephrase
your request." — wrong cause, useless advice, double-wrapped. Two layers:

1) openai.py mapped every non-401/429 APIStatusError to the generic
   "temporarily unavailable" copy, in four copy-pasted blocks (complete /
   stream / _complete_via_responses / _stream_via_responses) that had already
   drifted in wording.
2) session.py's plan_stream fallback swallowed the exception and yielded it
   as assistant TEXT, so the turn "succeeded" and cowork-server's turn-error
   mapping (the error-card machinery) never ran.

Fix:
- One shared _raise_for_status_error(exc, model) used by all four call paths
  so the mapping can't drift. 401 and 429 behavior unchanged (same copy the
  downstream provider_auth / token_limit detections key on; the "or to top
  up" wording drift normalized).
- New ModelUnavailableError(ConnectionError) in provider.py carrying
  {code, model} for the gateway's structured 403s: model_access_denied →
  plan copy with upgrade/switch guidance; model_disabled → hedged copy
  (can be tier lock or admin kill switch until ENG-596's config lands).
  Detection is code-exact on error.code — BYOK OpenAI 403s, Anthropic
  permission errors, and Cloudflare HTML 403s carry no such code and fall
  through to the generic message, never the plan copy. Subclassing
  ConnectionError keeps every legacy call site working unchanged.
- session.py: TokenLimitExceeded / ModelUnavailableError now PROPAGATE from
  the plan_stream fallback instead of being wrapped into prose, so the turn
  fails and the server can render an actionable card. All other exceptions
  keep the existing text fallback.

Tests: 11 new (401 copy pin, 429 quota + precedence over model-403, both
gateway codes incl. attrs/copy, ConnectionError compatibility, BYOK/HTML/
codeless 403 fall-through, 500 generic). Suite: 1147 passed; the 2
test_chat_scratchpad failures pre-exist on clean staging (env-dependent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…okenLimitExceeded

A model-gate 403 is deterministic: the recovery loop would re-send the same
doomed request twice (burning the retry budget + user-visible delay) before
the fallback path finally propagated it. Short-circuit at the outer catch,
exactly like TokenLimitExceeded.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@lucas-koontz lucas-koontz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed as Part 1 of the ENG-598 chain (with cowork-server#168 and cowork#353). The core is solid: one shared _raise_for_status_error across all four call paths kills the copy-paste drift, code-exact 403 detection (never bare-status or string matching) is the right call, 429-before-403 precedence is correct, and the session.py change to propagate ModelUnavailableError/TokenLimitExceeded instead of wrapping them as assistant text is exactly what makes the downstream card possible. Test coverage is genuinely thorough (BYOK region 403, Cloudflare HTML body, no-code 403, quota-wins-over-403). Approving.

One thing worth fixing, and two nits — none blocking:

Should fix — the CLI now defaults tier-lock errors to "retry" (chat.py:1889, not in this diff). ModelUnavailableError subclasses ConnectionError, and the CLI turn handler does default="retry" if isinstance(exc, ConnectionError) else "setup". So a model_access_denied (a deterministic plan gate) now highlights retry as the default action — the user hits Enter, re-sends the identical doomed request, and fails the same way. This directly contradicts the comment you added in session.py ("deterministic — the auto-retry below would just re-send the same doomed request"). Since you're already importing the type, consider default="setup" if isinstance(exc, ModelUnavailableError) else ("retry" if isinstance(exc, ConnectionError) else "setup") so the interactive default steers toward switching models. (model_disabled is hedged, so retry-as-default there is more defensible.)

Nits: the 429 docstring bullet is grammatically broken (inline comment), and _raise_for_status_error is typed -> None but always raises — typing.NoReturn would document/enforce that the four except sites can rely on it. The NoReturn one is truly optional (this repo runs no type checker in CI, and the except sites do fall through to a raise today).

Comment thread anton/core/llm/openai.py Outdated
permission errors, and Cloudflare HTML 403s carry no such code and
must fall through to the generic message, never the plan copy.
- 429 with a quota detail → TokenLimitExceeded (checked before the 403
branch can't shadow it — quota keeps its own card downstream).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar: "checked before the 403 branch can't shadow it" is malformed. The 429 and 403 branches gate on mutually-exclusive status_code values, so ordering can't cause shadowing between statuses anyway — the precedence that actually matters is only for a body carrying BOTH detail and error.code on the same status (your test_429_with_detail_wins_even_if_error_code_present pins exactly that). Suggest: "429 with a quota detail → TokenLimitExceeded, checked first so a body carrying both detail and error.code stays token_limit."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9dd769b — reworded to your suggestion: "429 with a quota detail → TokenLimitExceeded, checked first so a body carrying both detail and error.code stays token_limit."

Comment thread anton/core/llm/openai.py Outdated
)


def _raise_for_status_error(exc: "openai.APIStatusError", model: str) -> None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: this always raises on every branch, so -> None slightly understates the contract that the four except APIStatusError: _raise_for_status_error(...) sites depend on (they dereference response right after). -> typing.NoReturn documents "never returns normally" and would let a strict checker catch a future non-raising branch that leaves response unbound. No functional change; skip if you'd rather not introduce NoReturn (it's used nowhere else here).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — typed -> NoReturn in 9dd769b (imported from typing). Documents the contract the four except-sites rely on.

…pper doc/type nits

Addresses lucas-koontz's review on PR #236 (all non-blocking; PR approved):

- CLI turn handler (chat.py) defaulted to "retry" for any ConnectionError, and
  ModelUnavailableError subclasses ConnectionError — so a deterministic plan
  gate highlighted "retry" as the default, re-sending the same doomed request
  (contradicting the session.py comment). Now defaults to "setup" (switch
  model/provider) for ModelUnavailableError; transient ConnectionErrors keep
  retry.
- Fixed the grammatically-broken 429 docstring bullet in _raise_for_status_error.
- _raise_for_status_error typed -> NoReturn (it always raises); documents the
  contract the four except-sites rely on.

Mapper + session tests: 11 passed. (Pre-existing F541s elsewhere in chat.py
left untouched — not in scope.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alecantu7

Copy link
Copy Markdown
Contributor Author

Thanks @lucas-koontz — pushed 9dd769b:

  • Should-fix (CLI default): fixed. The handler now defaults to setup for ModelUnavailableError (deterministic gate → switching is the useful action), keeping retry as the default for other (transient) ConnectionErrors. Resolves the contradiction with the session.py comment.
  • 429 docstring reworded to your text; _raise_for_status_error now typed -> NoReturn.

Mapper/session tests green (11).

@alecantu7
alecantu7 requested a review from lucas-koontz July 7, 2026 23:14
@alecantu7
alecantu7 merged commit 8ba4232 into staging Jul 9, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 9, 2026
@lucas-koontz

Copy link
Copy Markdown
Contributor

Re-checked the polish — CLI default now branches setup for ModelUnavailableError (deterministic plan/kill-switch, retry can't help) and keeps retry only for genuine transient ConnectionErrors. Matches session.py's don't-retry intent. Approval stands.

@lucas-koontz
lucas-koontz deleted the alejandrocantu/eng-598-model-403-mapper branch July 9, 2026 16:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants