[Bug] A length stop with one output token escapes overflow detection, so a dead session never recovers and continue is a no-op

[gorban]

Summary

When a provider truncates an oversized prompt it can return finish_reason: "length" with a tiny non-zero output. pi-ai's overflow sniff requires exactly zero output for that case, so dsh classifies the response as an ordinary max-tokens stop: agent/request-error never sees CONTEXT_WINDOW_EXCEEDED, no overflow-recovery compaction is spent, and the user's continue produces one token and ends the turn — repeatedly, with no way out.

Environment

• dsh 0.1.6-alpha.2, commit ddefc45fbc7f8e46dd73185e68295696d1297887
• routed model openrouter/deepseek/deepseek-v4.1-flash, harness-resolved window 262,144
• @earendil-works/pi-ai@0.85.1
• session session-2496881f-8d36-4ce6-93d9-1954a31930c7

Reproduction

The session's last five turns, each answering continue:

turn	input	cacheRead	output	total	turn/end
22	269,321	17,792	1	287,114	max-tokens
23	11	287,104	1	287,116	max-tokens
24	17	287,104	1	287,122	max-tokens
25	19	287,104	1	287,124	max-tokens
26	21	287,104	1	287,126	max-tokens

Each assistant message's entire content is a single reasoning block containing "The". The prompt is 287k tokens against a 262,144 window, so the truncation signal is unambiguous — the only difference from the two turns that did raise CONTEXT_WINDOW_EXCEEDED is that their output was 0.

Current behavior

@earendil-works/pi-ai's isContextOverflow handles the length-stop case with:

if (contextWindow && message.stopReason === "length" && message.usage.output === 0) {
      const inputTokens = message.usage.input + message.usage.cacheRead
      if (inputTokens >= contextWindow * 0.99) return true
}

One output token and the check misses. packages/llm/llm-pi-ai/src/stream.ts then falls through to:

case 'length': return { kind: 'max-tokens' }

so the turn ends normally from the harness's point of view. Nothing records that the model produced a single token on a full prompt, and the agent/request-error overflow listener in compaction-basic never runs. A user-visible dead end: every continue costs a full request and returns nothing.

Expected behavior

A length stop whose prompt already fills the resolved window is context pressure, not an output cap. Suggested direction:

• Treat stopReason === 'length' as overflow when usage.input + usage.cacheRead >= contextWindow × ~0.99, regardless of whether the output was 0 or 1–2 tokens; or
• consult pi-ai's isRecoverableLength(message, desiredMaxOutput), which already exists for "a length stop below the caller's intended output limit" and is currently unused by dsh (grep -rn isRecoverableLength packages finds only node_modules).

Either way the turn should reach the existing overflow-recovery path, which is exactly what the two output = 0 turns did.

Relation to existing reports

• Overflow recovery produced no compaction attempt after CONTEXT_WINDOW_EXCEEDED (no compaction/start recorded; session stays dead) #6672 and Bug: context-window-exceeded errors aren't recovered when the provider is self-hosted vLLM #4956 describe overflow recovery not running when the provider does report an error. This is the opposite direction: the provider reports length, the harness does not recognise it as overflow, and no recovery is even attempted.
• Auto-compaction destroys history on a bodyless HTTP 400 (misclassified as CONTEXT_WINDOW_EXCEEDED, bypasses thresholdRatio) #5263 is the inverse misclassification (a bodyless 400 treated as overflow). The pair suggests the length/overflow classification needs a single documented rule.

Evidence bundle

https://gist.github.com/gorban/4725c87390cb391047a2f14b56b98d28

• session-2496881f-degenerate-length-stops.json — the five stops with full usage
• EVIDENCE.md section 3
• session-2496881f-sanitized.jsonl — full session log

[argszero]

Verified the mechanism end to end, and I think it moves the fix site: the one-token output is not the provider truncating anything — dsh asked for one token.

Where the 1 comes from

packages/llm/llm-pi-ai/src/adapter.ts:380 streams through pi-ai's simple entry point:

const events = snapshot.models.streamSimple(model, context, {
  ...profileOptions(profile, reasoning, apiKey),
  ...
  ...options.maxTokens === undefined ? {} : { maxTokens: options.maxTokens },

streamSimple builds its request through buildBaseOptions, whose output budget is not the caller's value but a context-clamped one (@earendil-works/pi-ai@0.85.1, dist/api/simple-options.js:2-9):

const CONTEXT_SAFETY_TOKENS = 4096;
const MIN_MAX_TOKENS = 1;
export function clampMaxTokensToContext(model, context, maxTokens) {
    if (model.contextWindow <= 0) return Math.max(MIN_MAX_TOKENS, maxTokens);
    const available = model.contextWindow - estimateContextTokens(context).tokens - CONTEXT_SAFETY_TOKENS;
    return Math.min(maxTokens, Math.max(MIN_MAX_TOKENS, available));
}

Both constants are module-private, so nothing on the dsh side can observe the reserve it is subject to. The clamp is not specific to the completions protocol: every api implementation reaches it (anthropic-messages.js:680 and bedrock-converse-stream.js:402 call it directly; the rest through buildBaseOptions), so all three protocols dsh can name are subject to it. With the window this session resolved (#7213: 262,144 while the provider advertises 1,048,576), available goes negative long before the provider would ever refuse, and Math.max(MIN_MAX_TOKENS, available) is exactly 1. The provider is handed max_tokens: 1, generates one token, and reports finish_reason: "length" — correctly, for the request it received.

Arithmetic against your own numbers

estimateContextTokens (dist/utils/estimate.js:91) anchors on the newest assistant usage that is not aborted/error (:56-57) — it takes usage.totalTokens and adds only the messages after it. A length stop qualifies, so each degenerate answer becomes the prefix estimate for the next request. Replaying your five turns through the pinned build with model = { contextWindow: 262_144, maxTokens: 32_768 } and a single trailing continue (2 estimated tokens):

turn	reported prompt	pi-ai estimate	available	max_tokens sent
22	287,113	287,116	−29,068	1
23	287,115	287,118	−29,070	1
24	287,121	287,124	−29,076	1
25	287,123	287,126	−29,078	1
26	287,125	287,128	−29,080	1

Same shape, swept: prompt 250,000 → 8,046 output tokens; 258,000 → 46; ≥ 258,048 (= 262,144 − 4,096) → 1. The MIN_MAX_TOKENS = 1 floor is why the output is one token rather than zero — and therefore why pi-ai's own output === 0 gate can never fire on it.

Scope of that check: I did not replay your session file; I ran your published usage numbers through pi-ai's own two exported functions from the pinned 0.85.1 build. The one assumption is the prefix-anchoring above, which your turn table satisfies.

Why this makes the loop unbreakable

Each 1-token answer is a valid prefix for the next estimate (it is neither aborted nor an error), so continue adds tokens to a prompt that is already over the believed window and reproduces the same negative headroom. The turn ends max-tokens, which is a sticky terminal TurnEndReason with no recovery hook — the only overflow-recovery entry point is agent/request-error carrying CONTEXT_WINDOW_EXCEEDED, and nothing in this path produces it. Every continue costs a full request and returns nothing, forever.

Your option 2 is unsafe on its own

isRecoverableLength(message, desiredMaxOutput) is stopReason === 'length' && desiredMaxOutput > 0 && usage.output < desiredMaxOutput (dist/utils/overflow.js:163-164) — true for every ordinary output-cap truncation, including a healthy one where the model simply hit its own ceiling with most of the window still free. Routing those into compactIfNeeded replaces the surface with no context pressure behind it. The docstring frames it as permission for "one bounded compact-and-retry attempt", not as a predicate that identifies overflow. (It is exported but never called anywhere in pi-ai 0.85.1's dist either, so "currently unused by dsh" is right — it is unused by pi-ai too.)

Where I would put the fix

The sniff is downstream of the defect. Math.max(MIN_MAX_TOKENS, available) converts "the prompt no longer fits the window we believe in" into a successful response that no layer classifies as anything but a normal max-tokens stop — while dsh already has a recovery path for exactly that condition (agent/request-error + CONTEXT_WINDOW_EXCEEDED, consumed by compaction-basic). The request should surface as CONTEXT_WINDOW_EXCEEDED when its prompt already exceeds the resolved window, instead of being rewritten into one nobody can win. That check has to be dsh-side, since neither the reserve nor the floor is exported, and it fixes #7213 and #7214 at one site.

Worth noting the asymmetry if the gate is merely relaxed: the clamp collapse fires at any resolved window — it is driven by the harness's own estimate, not by the provider — whereas "prompt fills ≥99% of contextWindow" is only as good as contextWindow. With #7213's documented per-model contextWindow: 1048576 workaround the same 287k prompt carries 384,000 tokens of room, so the clamp never collapses; but a route whose backend serves less than it advertises then hits the opposite problem — the fill test compares against 1,048,576 and can never fire, so the dead loop comes back with no clamp left to explain it. The window-independent trigger is the better primary; the fill test then becomes the confirmation.

So the single documented rule your "Relation to existing reports" asks for is roughly: a length stop whose prompt already exceeds the resolved window is context pressure; a prompt that exceeds the window must never reach the wire as a clamped request.

Related, but distinct from #5123 / #6671

Those cover dsh's own ratio math against a per-request reserve. This is pi-ai's hidden CONTEXT_SAFETY_TOKENS reserve feeding the same quantity from the other side, and the band between them is where requests are silently degraded: with the resolved window, pressure compaction starts at 0.8 × 262_144 = 209_715 while the clamp floor sits at 258_048, so between those two numbers every request's output budget shrinks (32,768 → 46 → 1) with no event, no error, and nothing in the UI.

Silence

RequestContext (packages/core/session/src/types.ts:242-251) records contextWindow but no requested output budget, so the collapse leaves no durable trace — your session log shows an ordinary max-tokens turn end. Recording the sent max_tokens would make this class visible, but that is a request/context shape change and belongs in its own thread.

[argszero]

Update: this is now a published plugin — @argszero/cordis-plugin-length-stop-overflow.

Since the analysis above, I built the fix I described as mountable and verified it end to end against real shipped code. Posting it here so the gap has a plugin-shaped answer, and so anyone hitting the same dead session has something to install today.

npm i @argszero/cordis-plugin-length-stop-overflow

• npm: https://www.npmjs.com/package/@argszero/cordis-plugin-length-stop-overflow
• source: https://github.com/argszero/cordis-plugin-length-stop-overflow (MIT)

Mount it with a bundle patch (no config needed):

- insert:
    - id: length-stop-overflow
      name: '@argszero/cordis-plugin-length-stop-overflow'

What it does

It observes the public llm/stream waterfall and rewrites the truncating finish into the failure the harness already has a recovery path for:

{ kind: 'max-tokens' }   ->   { kind: 'error', failure: { code: 'CONTEXT_WINDOW_EXCEEDED', … } }

The synthesized failure carries the harness' own CONTEXT_WINDOW_EXCEEDED_CODE, imported by value from @deepseek-ai/dsh-llm, so the plugin and the core cannot drift apart about which failure the recovery path keys on. @deepseek-ai/dsh-compaction-basic (mounted in bundle/base and bundle/web-app) then compacts and returns { kind: 'retry' }, bounded by its own maxOverflowRetries (default 1) — so no retry loop.

The verdict is window-independent, which matters because this thread's root cause is a window the harness believes and the provider does not (see #7213): a max-tokens finish with a negligible output count is a truncation of our request by construction — the real output cap is the model's own, tens of thousands of tokens in the catalog. No context-window number is read anywhere in the plugin. A finish carrying no usage chunk is left alone (fail-closed).

Verified end to end, with control arms

48 tests. The interesting ones run against the real downstream rather than the plugin's idea of it:

• hop 1 — real LlmRuntime + the plugin mounted in a real Cordis context: the chunk the caller sees ends as CONTEXT_WINDOW_EXCEEDED. Unmounted, the same scripted turn is an ordinary max-tokens end (the defect).
• hop 2 — real @deepseek-ai/dsh-compaction-basic mounted with the services it injects (llm, tokenMeter, sessions, sessionProjections), driven through agent/request-error exactly as agent-loop/src/agent.ts:445-464 drives it: the plugin's failure passes compaction-basic/src/index.ts:180's code gate, reaches compactIfNeeded, and comes back { kind: 'retry' }. A foreign code (INVALID_REQUEST — the code the sibling report 自动压缩在中文密集会话与大陆中转站下失效：两类根因 + 实测证据 #6361 lands on) compacts nothing and requests no retry; and a second overflow failure is terminal, honouring maxOverflowRetries.

Two properties I checked because they decide whether the rewrite is exact or merely plausible:

• On an error finish the loop settles only assistant/attempt (agent-loop/src/agent.ts:445-448) and never appends assistant/message, so the useless one-token answer does not enter the derived conversation — compaction summarizes a history without it. The alternative (retrying with a degenerate answer appended each turn) would grow the log while getting nowhere.
• CONTEXT_WINDOW_EXCEEDED is not in DEFAULT_RETRYABLE_CODES (llm/src/retry-policy.ts:17-23) and llm-retry's normal mode hands anything not in its route's set straight to next() (llm-retry/src/index.ts:215-217), so the two never fight over the same failure.

Configuration, and why you may want warn first

- set:
    - id: length-stop-overflow
      config:
        mode: error             # error | warn | off  (default: error)
        atMostOutputTokens: 2   # default: 2

atMostOutputTokens is the whole judgement: a length stop after ≤ N output tokens is a truncation. The reported case is exactly 1, because pi-ai's clamp floor (MIN_MAX_TOKENS = 1) is what keeps the count off the zero its own overflow sniff requires. Raise it when your backend clamps to a small positive remainder instead: with a believed window of 262,144 and pi-ai's 4,096-token reserve, a prompt anywhere in [250_000, 258_047] yields a budget in [8_046, 1], all of it context pressure — atMostOutputTokens: 64 covers that band. That band is also why this is worth measuring on your own traffic: mode: warn reports the detection and streams unchanged.

Honest boundaries

• This restores the recovery, it does not correct the window. The root causes in this thread (the catalog miss and the silent degradation band) stay upstream — with this mounted, a session in that state compacts and keeps working instead of answering continue with one token forever.
• It is a mitigation at the stream boundary, so it treats the symptom the provider reports, not the request dsh built. A max_tokens: 1 that the provider answers with an empty-but-successful response, rather than length, is not covered here.
• Related but separate: @argszero/cordis-plugin-overflow-classifier-guard covers the other half of the same recovery path — a provider error that names an input-token budget in wording the classifier misses (自动压缩在中文密集会话与大陆中转站下失效：两类根因 + 实测证据 #6361). The two are safe to mount together; one handles a wrong classification of an error, this one handles the absence of an error.