A provider can truncate a response mid-stream and still send finish=stop — mapStopReason() records the turn as completed when content is non-empty but usage is 0/0/0

[EvenZ2001]

Summary

A provider can truncate a response mid-stream and still signal a clean completion. When it does, DSH records the turn as completed and the assistant's half-finished message is treated as a finished answer. The existing guard in mapStopReason() only catches the fully-empty case, so a truncated-but-non-empty response slips through silently.

Filed as a Discussion per CONTRIBUTING (issues/PRs are closed on this repo).

Observed behavior

In a real durable session, the final step of a turn ended like this:

15:31:31.661  block-start (reasoning)
15:32:13.207  block-end          (+41.5s)
15:32:13.207  text-chunks (1)
15:32:13.208  usage    { inputTokens: 0, outputTokens: 0, totalTokens: 0 }
15:32:13.208  finish   reason={"kind":"stop"}  stopReason="stop"

Then:

turn/end  {"turn": 1, "reason": {"kind": "completed"}}

The user-visible result: my reply stopped in the middle of a sentence ("The spawn in runner-launch has " — literally trailing off mid-clause). The system considered the turn successfully completed, so there was no retry and no error surfaced. The only reason I could explain it afterwards was by reading the journal.

Why the existing guard doesn't catch it

packages/llm/llm-pi-ai/src/stream.ts — mapStopReason() (line ~208):

case "stop":
  if (message.content.length === 0) return {
    kind: "error",
    failure: { message: `model "${message.model}" returned a completed response with no content`, code: EMPTY_RESPONSE_CODE }
  };
  return { kind: "stop" };   // ← any non-empty content is accepted as a clean stop

The guard is content.length === 0. In this incident the step carried 1033 characters of reasoning plus 1033 characters of text (the text block was a duplicate of the reasoning), so content.length !== 0 and the response was accepted as a legitimate stop.

The discriminating signal: zero usage

Across all 57 steps of that turn, this was the only step reporting usage = 0/0/0:

	step 56	step 57 (terminal)
usage (in/out/total)	3365 / 1574 / 150859	0 / 0 / 0
finish reason	{"kind":"tool-calls"}	{"kind":"stop"}
stopReason	toolUse	stop
blocks	reasoning, text, tool-call, tool-call	reasoning, text

Steps 1–56 all carried non-zero usage (thousands to 226k). A genuinely completed generation cannot report zero tokens in and zero out. Compare the other two completed turns in the same session, whose terminal steps had normal accounting:

turn	terminal step	usage (in/out/total)
1	57	0 / 0 / 0 ← truncated
2	58	444 / 905 / 227653
3	16	139 / 664 / 246435

Contributing factor: a flaky relay

The provider route (deepseek/deepseek-v4.1-flash via a third-party OpenAI-compatible relay) was returning 502 with no body earlier in the same turn:

turn=1 step=10 retry=1 :: OpenAI API error (502): 502 status code (no body)
turn=1 step=11 retry=1 :: 502
turn=1 step=11 retry=2 :: 502
turn=1 step=13 retry=1 :: 502

Those were retried and recovered (SERVER is retryable by default). The terminal truncation is the same infrastructure degrading, but it arrives as a successful stream with a fabricated terminal event, so the retry policy never engages — retry covers failed requests, not truncated-but-"successful" responses.

Relationship to existing discussions

This is adjacent to but distinct from #6743 (which is excellent and I read carefully before filing):

	#6743 item 2	This report
content	empty	non-empty but cut mid-sentence
current mapping	EMPTY_RESPONSE (mislabeled as model behavior)	{kind:"stop"} → turn completed
requested change	relabel to infra/transport	detect and treat as retryable failure

#6743 asks to relabel the empty case. My case is the gap next to it: the guard is keyed on emptiness, so a truncated response that produced some content is indistinguishable from a clean finish and never even reaches EMPTY_RESPONSE.

Also related, though about length truncation rather than a fabricated stop: #2143, #1703, #1263 (all INVALID_REPLAY_STATE after length truncation), and #6918 (client discards already-streamed body when the server drops). None of these covers "stop + non-empty + zero usage".

Proposed approach

Treat a stop with zero usage as a transport fault rather than a model outcome:

1. In mapStopReason(), extend the stop case: if content.length === 0 or the usage is all-zero (inputTokens === 0 && outputTokens === 0), map to an error.
2. Classify it as SERVER (or a dedicated code such as TRUNCATED_RESPONSE) so the existing dsh-llm-retry policy retries it automatically — SERVER is already in the default retryableCodes.
3. Keep a one-line human message that names the real cause, e.g. "provider ended the stream with zero usage — response was likely truncated, not completed".

This is deliberately narrow: it only fires on the combination (stop + zero usage), so it cannot affect normal completions, and since usage totals are already recorded per step it needs no new bookkeeping.

A secondary, complementary idea: when a terminal step's finish is stop and neither text nor a tool call ended on a sentence/pre-block boundary, that is suspicious — but I'd treat that as much weaker evidence than the zero-usage signal and would not gate on it.

Environment

• DSH: DSH Desktop 2.0.10 (also reproducible against the npm packages @deepseek-ai/dsh-llm-pi-ai@0.1.5-rc.2)
• OS: Windows 11 10.0.26200
• Provider: OpenAI-compatible third-party relay (openai-responses API), model deepseek/deepseek-v4.1-flash
• Relevant config: retryPolicy: { mode: normal, maxRetries: 5, retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT] }, streamIdleTimeoutMs: 300000

I can provide a redacted session.v3.jsonl.zstd extract with the exact step-57 records (stream chunks, usage, finish, turn/end) if that's useful.

Impact

This is worse than it first looks. A half-sentence is obvious, but the same mapping applies to any long turn: if the relay truncates while the model is mid-way through editing files or running a multi-step task, the turn is recorded as completed, no retry happens, and the work silently stops halfway with the agent believing it finished. In this session the truncation was cosmetically obvious; on a code-editing turn it would not be.

Thanks for the harness — the append-only journal is what made this diagnosable at all.

[argszero]

Thanks — the detection half of this is exactly as you describe, and I checked it against the tree. The retry half is in better shape than the report implies, which changes what a fix has to touch.

The guard is keyed on content, not on accounting. mapStopReason (packages/llm/llm-pi-ai/src/stream.ts:80) takes the degenerate-completion branch on message.content.length === 0 (:99), and the terminal event is produced at case 'done' (:208), which yields usage and then finish. Any content block at all — a reasoning block counts — defeats it, so a truncated-but-non-empty response is indistinguishable from a clean stop at that site. Your reading is correct.

But the failure path never asks whether content was streamed. packages/core/agent-loop/src/agent.ts:443 branches on finish.kind === 'error' || 'aborted', settles the stream as assistant/attempt (an attempt record — not an assistant/message, so the half-finished text is not committed as an answer), and dispatches agent/request-error. llm-retry then decides purely on the failure code: packages/llm/llm-retry/src/index.ts:215 returns {kind:'retry'} iff policy.retryableCodes.includes(failure.code), and DEFAULT_RETRYABLE_CODES (packages/llm/llm/src/retry-policy.ts:18) contains EMPTY_RESPONSE_CODE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, bounded by maxRetries (default 5).

So the shape you ask for already exists end to end — detect → error finish → retryable code → retry — and the retry is not blocked by the partial text. The missing half is purely the detection.

And the detection is reachable from a plugin. The llm/stream waterfall lets a listener observe the terminal chunk and replace it, and packages/llm/llm/src/invariant.ts:75 sanctions exactly this shape: an error/aborted finish is allowed with blocks still open, while a non-error finish requires every block closed. Re-labelling a stop as a retryable error is therefore a legal transformation at that seam.

One caution for whoever implements the zero-usage discriminator. Zero usage is only evidence against a control: a route that never reports usage returns 0/0/0 for every clean stop, and treating all of them as failures converts a silent truncation into up to maxRetries extra requests per step. I would make it self-calibrating — zero usage counts as evidence only on a route/model that has previously reported non-zero usage — and cap the number of reclassifications per session. Note also that the trailing-off text itself is not a usable signal: nothing machine-checkable separates "cut mid-clause" from "answered briefly", so the accounting field is the only handle, exactly as your table shows.

I maintain a small plugin for the sibling case you link (discussion 6743 item 2 / #6218 — a completion with no visible content reported as a successful turn); it emits the same retryable error finish at the same seam: https://github.com/argszero/cordis-plugin-empty-response-guard. Your case is the same defect one branch further out, so I plan to extend that guard rather than publish a second plugin for it, and will follow up here with the version once it lands.

[argszero]

Follow-up: the version that ships this is published.

When I replied above I said I would extend this guard rather than publish a second plugin at the same seam, and follow up here with the version once it landed. It landed:

• npm: @argszero/cordis-plugin-empty-response-guard@0.2.0
• repo: https://github.com/argszero/cordis-plugin-empty-response-guard (be71911)

npm install @argszero/cordis-plugin-empty-response-guard

- insert:
    - id: empty-response-guard
      name: '@argszero/cordis-plugin-empty-response-guard'

Mounting needs no configuration. For the shape you described — a stream that ends with finish=stop on a route that has previously reported usage in this process, whose usage chunk accounts for zero tokens — the terminal chunk is re-emitted as an error finish with code TRANSPORT instead of stop. TRANSPORT is retryable under the default policy, so the retry machinery gets its turn (with one correction to the framing in the original report, noted below).

The control is the part worth reviewing. Zero usage is only evidence if the route reports usage at all: a gateway that never sends a usage chunk would otherwise have every clean stop read as fabricated, trading one silent truncation for up to maxRetries extra requests per step. So the guard calibrates per route — a route's zero is trusted only after that route has reported non-zero usage at least once in the process, and the arming is logged once so that "declined because uncalibrated" is distinguishable from "saw nothing" — and bounds shape-2 corrections per session (fabricatedStopMaxPerSession, default 3), past which it logs the evidence it had and leaves the stop alone. A stream that carries no usage chunk at all is never touched.

Three details that decided the implementation, in case they are useful for the upstream fix:

• The retry machinery was never the missing half. agent.ts:443 settles assistant/attempt and dispatches agent/request-error without being gated on already-streamed content, and llm-retry decides purely on the failure code (retry-policy.ts:18). Both are already true today. Only the detection is missing, which is why this can be fixed at the stream seam at all.
• It is reported as TRANSPORT, not EMPTY_RESPONSE. The caller did receive content, so naming the failure after an absent answer would misreport it; what failed is the transport's claim to have delivered a whole one.
• Text-level "it stopped mid-sentence" is not decidable, and guessing wrong costs real requests. Usage is the only handle the provider gives us — which is exactly why it needs the control above.

The earlier shape (the reasoning-only stop from #6218) is unchanged and still wins when both could apply; detectFabricatedStop: false restores exactly the 0.1.0 behaviour.

45 tests pass (npm test), including one that drives a real cordis Context and a real llm/stream waterfall end to end, and one that fails if the guard ever starts buffering. This remains a seam fix, not the core fix — the repair that would make it unnecessary belongs in mapStopReason.