Report a failed turn as an error, not as the answer

[pathscale]

The problem

Ask Claude for a model that does not exist and it exits 0. Verbatim from the run:

{"type":"result","subtype":"success","is_error":true,"api_error_status":404,
 "result":"There's an issue with the selected model (bogus-model-xyz). It may not exist or you may not have access to it."}

subtype says success. is_error is the field that decides. Before this, that came back as Ok(Outcome) with the sentence in text, so a GUI checking Result::is_ok() rendered "There's an issue with the selected model" as the model's reply.

This is the same shape as auth and quota failures, both of which were already lifted out of Ok for exactly this reason. Error::AgentError covers the rest of that family.

What is here

Error::AgentError { agent, bin, status, message } — the agent's own wording plus the provider's status where one was reported. Classified after auth, quota and a rejected flag, so those more specific readings still win.

Codex needed a second step. It exits 0 too, but forwards the upstream response body as a JSON string:

{"type":"turn.failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"message\":\"The 'bogus-model-xyz' model is not supported when using Codex with a ChatGPT account.\"}}"}}

Left alone the caller gets JSON instead of a sentence, with the status buried inside it. It is unwrapped to the sentence plus the code; anything not in that shape passes through untouched. Copilot reports no explanation with a non-zero exitCode, so the code is named rather than leaving the failure blank.

A bug this turned up

The new live test passed alone and failed in the full suite. The reason was worth having:

Every Claude stream-json run prints a rate_limit_event, and on a healthy run it says "status":"allowed" and refuses nothing. Quota classification scanned the raw stream for the substring rate_limit, which that record contains by construction. So any Claude failure was reported as Error::RateLimited: the caller was told to wait out a limit that was never hit, and the actual cause never reached them. The record is not emitted on every single run, which is why running the test alone passed.

Prose heuristics applied to structured JSON match field names. Where the stream parsed, the parsed signal and the agent's own wording now decide; the raw scan stays as the fallback for output that produced neither, which is what it was written for.

Also

The 0.2.0 changelog entry credited that release with streaming by default and token-by-token Claude text. The commit is timestamped three hours after the 0.2.0 release commit, and the tree of 52db91c ("Release 0.2.0") has no --include-partial-messages.

Correction. This originally cited git show v0.2.0:src/agent.rs. This repository has no tags at all, so that command errored and proved nothing; the zero it appeared to return was grep -c counting an empty stream. Re-checked against the release commit itself, which is the evidence quoted above. The conclusion is unchanged, and the timestamps were always independent of it. Anyone reading the entry against the published 0.2.0 would find the feature missing. Corrected in place with a note rather than silently, since 0.2.0 is on crates.io.

The behaviour change folds into the unpublished 0.2.1 entry rather than taking a new version.

Verification

• fmt, clippy --all-targets, cargo doc, cargo package: clean
• 123 unit + 6 process + 7 doctests pass
• Full live suite: 16/16 against claude 2.1.205, codex-cli 0.145.0, Copilot CLI 1.0.75, including two new tests that drive a real unknown-model failure on Claude and Codex

Every payload in the new tests is copied from a real run, not constructed.