Bug: bare 400/413 (no body) mislabeled as context overflow for non-Cerebras providers

[jc77411]

Bug report: bare "400/413 (no body)" is mislabeled as context overflow for non-Cerebras providers

English

Summary

When a non-Cerebras OpenAI-compatible provider (e.g. SiliconFlow) rejects a request with a bare 400 status code (no body), dsh-llm-pi-ai's mapStopReason labels it CONTEXT_WINDOW_EXCEEDED. The harness then runs its overflow recovery — a compaction whose summarization request re-sends the same failing provider and fails the same way — collapsing the turn into a retry loop that surfaces a fabricated "context overflow" for a failure that was never about context length.

Root cause

mapStopReason delegates overflow detection to pi-ai's isContextOverflow, which matches a bare 400/413 (no body) against the pattern:

/^4(?:00|13)\s*(?:status code)?\s*\(no body\)/i

That pattern exists only for Cerebras, whose gateway reports overflow as an empty HTTP 400/413. But the harness applies the verdict provider-agnostically, so SiliconFlow's empty 400 (which can mean quota, auth, request guard, or a transient failure) is promoted to CONTEXT_WINDOW_EXCEEDED.

Proposed fix

Guard the pi-ai overflow promotion on the model's provider:

const errorText = message.errorMessage ?? ''
const bareHttpNoBody = /^4(?:00|13)\s*(?:status code)?\s*\(no body\)/i.test(errorText)
const skipPiAiOverflow = bareHttpNoBody && message.provider !== 'cerebras'
const piAiOverflow = !skipPiAiOverflow && isContextOverflow(message, contextWindow)

A bare 400/413 (no body) from a non-Cerebras provider then falls through to classifyPiAiError, which routes the 400 to INVALID_REQUEST. Cerebras keeps the overflow promotion, and descriptive overflow text (matched by isContextWindowExceededError via the separate harnessOverflow arm) is unaffected.

Ready-to-review implementation

A complete fix (source change + unit test + Agent Note + bilingual README update) is on my fork branch:

https://github.com/jc77411/deepseek-harness/tree/fix/pi-ai-bare-400-overflow-misclassification

The single behavioral change is in packages/llm/llm-pi-ai/src/stream.ts.

---

中文

摘要

当非 Cerebras 的 OpenAI 兼容提供方（如 SiliconFlow）以裸 400 status code (no body) 拒绝请求时，dsh-llm-pi-ai 的 mapStopReason 会把它标记为 CONTEXT_WINDOW_EXCEEDED。Harness 随后运行溢出恢复——compaction 的摘要请求向同一个失败的提供方重发并以同样方式失败——使回合陷入重试循环，把一个与上下文长度无关的失败呈现为编造的"上下文溢出"。

根因

mapStopReason 把溢出检测委托给 pi-ai 的 isContextOverflow，后者用正则匹配裸 400/413 (no body)：

/^4(?:00|13)\s*(?:status code)?\s*\(no body\)/i

这条正则仅为 Cerebras 存在——它的网关用空的 HTTP 400/413 表示溢出。但 harness 无差别地应用该判定，于是 SiliconFlow 的空 400（可能意味着配额、鉴权、请求守卫或瞬时故障）被升级为 CONTEXT_WINDOW_EXCEEDED。

建议的修复

按模型的 provider 守卫 pi-ai 的溢出升级：

const errorText = message.errorMessage ?? ''
const bareHttpNoBody = /^4(?:00|13)\s*(?:status code)?\s*\(no body\)/i.test(errorText)
const skipPiAiOverflow = bareHttpNoBody && message.provider !== 'cerebras'
const piAiOverflow = !skipPiAiOverflow && isContextOverflow(message, contextWindow)

非 Cerebras 提供方的裸 400/413（无 body）随后落入 classifyPiAiError，把 400 归类为 INVALID_REQUEST。Cerebras 保留溢出升级，描述性溢出文本（由独立的 harnessOverflow 分支经 isContextWindowExceededError 匹配）不受影响。

可直接评审的实现

完整的修复（源码改动 + 单元测试 + Agent Note + 双语文档更新）在我的 fork 分支：

https://github.com/jc77411/deepseek-harness/tree/fix/pi-ai-bare-400-overflow-misclassification

唯一的行为改动在 packages/llm/llm-pi-ai/src/stream.ts。

[tianhao8687]

Confirmed against current master: mapStopReason calls isContextOverflow(message, contextWindow) before the Harness classifier and without a provider guard, so the reported false promotion is real.

The proposed direction is good, with one refinement: exclude only the provider-specific bare no-body heuristic, while retaining all other pi-ai overflow signals for every provider. In other words:

• bare 400/413 (no body) + provider === 'cerebras' may promote to overflow;
• the same bare status from any other provider must not promote;
• descriptive overflow text and usage/context-window detection remain provider-independent.

Also define the non-Cerebras 413 fallback explicitly. Today classifyPiAiError recognizes 400 as INVALID_REQUEST, but a bare 413 would otherwise fall through to PI_AI_ERROR. If there is no dedicated request-too-large code, INVALID_REQUEST is still more honest than fabricated context overflow.

Regression cases should cover:

1. Cerebras bare 400 and 413 → context overflow;
2. SiliconFlow/custom-provider bare 400 and 413 → non-overflow error;
3. descriptive context-limit errors from any provider → overflow;
4. usage-based silent/length overflow remains unchanged;
5. rate-limit exclusions remain unchanged; and
6. the agent loop does not launch compaction for the non-overflow cases.

That last assembled-loop assertion matters because preventing the misleading recovery cycle is the actual user-visible fix, not only changing one error code.

[NovaRouteAI]

@tianhao8687 the ordering bug you pinned — status classifier running before the provider guard — is exactly the class of issue that bites multi-provider setups, because a bare 413 from one vendor means payload size while another means quota.

A status-code-only matcher will keep misrouting compaction triggers until each provider's empty-body semantics are tabled.

Have you considered shipping that table as data (provider to empty-body meaning) instead of code?

[jc77411]

Thank you for the thorough review against master — much appreciated, especially the 413 fallback catch, which I had missed.

Adopted your refinement:

• classifyPiAiError HTTP-status matcher widened from \b400\b to \b4(?:00|13)\b, so a bare 413 (no body) now routes to INVALID_REQUEST instead of falling through to PI_AI_ERROR.
• The provider guard stays narrow: only the bare no-body heuristic is gated on provider !== 'cerebras'; descriptive overflow text and usage/context-window detection remain provider-independent.

Updated the unit test to cover both bare 400 and bare 413 for a non-Cerebras provider. The fork branch is updated: https://github.com/jc77411/deepseek-harness/tree/fix/pi-ai-bare-400-overflow-misclassification

The remaining regression cases you listed (Cerebras bare 400/413 → overflow, descriptive errors → overflow, usage-based silent/length overflow, rate-limit exclusions, and the assembled-loop assertion that no compaction is launched for non-overflow cases) are the right coverage set. I cannot run the assembled-loop test locally here (the monorepo install fails on network access), so that last assertion is the one I would most welcome help validating.

[tianhao8687]

The repository already has the right assembled-loop harness: packages/compaction/compaction-basic/tests/compaction-loop-repro.spec.ts, especially the context-overflow recovery across the real loop and compaction-basic block.

I would add one sibling case with a tiny adapter that returns an in-band error finish carrying code: 'INVALID_REQUEST' and the bare non-Cerebras 400/413 message. Mount AgentLoop, TokenMeter, and BasicCompactionEngine exactly like the existing overflow case, then assert:

• one conversation request only;
• zero summary requests;
• no compaction/start, compaction/summary, or compaction/end events;
• no retry step; and
• the final turn error preserves INVALID_REQUEST and the original text.

The llm-pi-ai unit test proves the wire message maps to INVALID_REQUEST; this real-loop test proves that code does not enter overflow recovery. Those two tests meet at the public finish-reason seam, so the assembled test does not need a real provider or network access. A fully wired pi-ai mock stream would add cost without proving a different product behavior.

[jc77411]

Done — added the assembled-loop sibling case exactly as you described. It uses an in-band INVALID_REQUEST finish carrying the bare 400 status code (no body) text, mounted with AgentLoop, TokenMeter, and BasicCompactionEngine alongside the existing overflow case.

Assertions:

• one conversation request only;
• zero summary requests;
• no compaction/start, compaction/summary, or compaction/end events;
• no retry step;
• final turn/end preserves INVALID_REQUEST and the original text.

Verified locally against the full workspace install: the loop test (7 cases) and the llm-pi-ai adapter test (47 cases) both pass. The llm-pi-ai unit test and the real-loop test now meet at the finish-reason seam as you outlined.

Branch updated: https://github.com/jc77411/deepseek-harness/tree/fix/pi-ai-bare-400-overflow-misclassification

[tianhao8687]

Excellent — that closes the actual user-visible gap.

The coverage split is now appropriate: the llm-pi-ai unit cases prove both bare 400 and 413 map to INVALID_REQUEST, while one assembled-loop INVALID_REQUEST case proves that this public failure code never enters compaction recovery. The loop test does not need to duplicate every wire spelling because it consumes the normalized finish-reason seam.

With the 47 adapter cases and 7 real-loop cases passing, I would consider the behavior ready for maintainer review. The remaining useful handoff detail is an exact commit SHA plus the two focused test commands in the discussion, so maintainers can inspect/cherry-pick a stable revision even if the branch moves.

[jc77411]

Thank you — appreciate the thorough review throughout.

Stable revision

The tip of fix/pi-ai-bare-400-overflow-misclassification is:

• 3d7e36d fix(llm-pi-ai): stop mislabeling bare 400/413 (no body) as context overflow
• d9156cf fix(llm-pi-ai): route bare 413 (no body) to INVALID_REQUEST
• b2ef32e test(compaction-basic): assert INVALID_REQUEST does not enter overflow recovery ← HEAD

Full SHA of HEAD: b2ef32e45d61f374b0bd4dd78ae65d9f69bf8192

The two prior commits: 3d7e36da7bceeec67855d4650f4e856d899e332e, d9156cf59cbf729d7ca5f5803c875b768d44a964

Focused test commands

# wire mapping: bare 400 / 413 → INVALID_REQUEST, overflow text untouched
pnpm exec vitest run packages/llm/llm-pi-ai/tests/adapter.spec.ts

# assembled loop: INVALID_REQUEST never enters compaction recovery
pnpm exec vitest run packages/compaction/compaction-basic/tests/compaction-loop-repro.spec.ts

Both pass locally (47 adapter + 7 loop). The branch lives at jc77411/deepseek-harness if you want to cherry-pick a single commit — each one is self-contained.

[NovaRouteAI]

Thanks for the revision trail — the three-commit split (mislabel fix, routing fix, regression test) is exactly the shape that makes this safe to backport. Worth watching how the empty-body semantics hold up as more providers turn up; a data table for those meanings would age better than code per vendor.