Bug: reasoning-only completions are reported as successful - the EMPTY_RESPONSE guard tests order.length, which counts the reasoning block

[MutaLucem]

Summary

A provider completion that contains only reasoning_content and ends with stop is reported as a successful turn. No assistant text block and no tool call are produced, so the turn terminates with no visible output, no error, and no retry.

From the user side this looks like the model "thinking halfway and then stopping". The session log records turn/end with reason.kind = "completed".

Root cause

@deepseek-ai/dsh-llm-deepseek/lib/index.js — the translate() consumer:

// L1216-1224 — every opened block is pushed into `order`
function open(kind) {
    const block = { index: nextIndex++, kind, text: "" };
    order.push(block);
    return block;
}

// L1239 — the degeneracy guard tests order.length
reason: reason.kind === "stop" && order.length === 0 ? {
    kind: "error",
    failure: {
        message: "model returned a completed response with no content",
        code: EMPTY_RESPONSE_CODE
    }
} : reason

// L1258-1260 — a reasoning block is opened through the very same open()
const reasoning = delta?.reasoning_content;
if (typeof reasoning === "string" && reasoning.length > 0) {
    if (!reasoningBlock) {
        reasoningBlock = open("reasoning");   // <- order.push(), so order.length >= 1

So as soon as the model emits any reasoning_content, order.length >= 1 for the remainder of the stream, and the EMPTY_RESPONSE branch at L1239 can never be taken. A genuine degenerate completion — reasoning only, no text, no tool call — is forwarded as a normal stop.

The JSDoc immediately above states the intended contract:

A stop (or absent) finish with no opened blocks is a degenerate provider completion and maps to an EMPTY_RESPONSE error finish instead of a successful empty message.

The intent is correct; the predicate merely counts a block kind that must not count as content.

A correct predicate already exists elsewhere in the codebase

@deepseek-ai/dsh-llm/lib/index.js:

• L1351 — function assistantStreamHasVisibleText(stream)
• L1378 (JSDoc) — "Every streamed text-delta fragment joined in stream order; reasoning and Tool-call fragments are excluded."

dsh-llm therefore already defines "visible" as text-only and explicitly excludes reasoning. The DeepSeek adapter's degeneracy guard uses order.length instead, so the two definitions disagree on whether a reasoning block is content.

Suggested fix

Test for the absence of visible content rather than the absence of any block:

const hasVisible = order.some((b) => b.kind === "text" || b.kind === "tool");
reason: reason.kind === "stop" && !hasVisible ? { /* EMPTY_RESPONSE */ } : reason

or reuse the assistantStreamHasVisibleText definition. Note that reasoning followed by a tool call is a legitimate completion and must not be classified as empty.

Evidence from a local session log

Read-only inspection of session.jsonl.zstd (5.12 MB compressed → 21 MB, 4564 events); metadata only, no prompts, tool arguments, or tool outputs.

• The failure rate is not uniform within the session, but the split is by model, not by reasoning effort. The session's 27 request/header snapshots show reasoningEffort: max and maxTokens: 65536 on every request — neither value ever changed:

  model	turns	affected turns
  deepseek-v4.1-flash-expires-on-0910	1-28	0
  deepseek-flash	29-51	7 (30.4%)

  The model was swapped at 21:44 when the older, date-limited experimental model reached its expiry date. Two model/selection records three seconds apart (21:43:51 high, 21:43:54 max) are easy to misread as a settings change, but they are the new model's default being written and then immediately overridden: the first request on the new model already carries max.

  All 7 affected turns fall in the latter part of the new model's run (37-51); turns 29-36 on the same model were clean. So the model change is an aggravating factor, not the defect itself — it raises how often the model emits reasoning-only completions, while the guard predicate is what converts that into a silent success. The defect is upstream of any particular model or effort setting.

• For each of the 7 affected turns, per-turn output_tokens equals reasoning_tokens. No text tokens were billed at all — consistent with reasoning-only output.

• All 7 turns ended with reason.kind = "stop". There is no chunk.reason === "length" anywhere in the session, and the largest single output was 11,214 tokens against a 65,536 budget — output truncation is excluded.

• No llm/retry event was emitted for any of the 7 turns. Because the finish surfaced as stop rather than EMPTY_RESPONSE, the retry policy never saw an empty-response failure it could act on.

• In at least one affected turn the model had already composed the next tool invocation in its reasoning but stopped before emitting it; the reasoning text is present in the log while the tool call is absent.

Attempted workarounds

Attempts were made at two levels before the defect was isolated. None resolved the symptom, which is itself consistent with the root cause: no caller-side setting can help, because the degenerate completion is never classified as a failure in the first place.

Configuration level

attempt	change	observed effect
raise the output budget, in case completions were being cut off	maxTokens 32768 -> 65536	no change. The largest observed completion in the session is 11,214 tokens, so the budget was never the binding constraint
switch provider / model preset	provider and model preset changed	no change to the failure mode
swap the default model	deepseek-v4.1-flash-expires-on-0910 -> deepseek-flash at 21:44, because the former carries its expiry date in its own name	0 / 28 turns -> 7 / 23 turns. The replacement model emits reasoning-only completions far more often, so the defect went from invisible to frequent

A note on reasoningEffort, since it is the obvious suspect: it was not varied. It is max on every request/header snapshot in the session. Raising it is not a fix, and the model swap above is not evidence of a model-specific bug either — the guard predicate is wrong for any model that can open a reasoning block and then stop.

Process level

The affected work consisted of long, multi-step tool chains, which is the shape that triggers the symptom most often. These measures were adopted while the defect was still unexplained:

measure	intent	observed effect
externalise the plan into a standing execution document - roadmap, a single progress ledger, enumerated decision points, and per-step confirmation	cut one long autonomous chain into short, explicitly confirmed segments	fewer long chains, hence fewer opportunities to hit the defect; it never removed it
register the work as a tracked goal object with a round cap	bound how far a single run can drift before control returns to the user	same
drive progress through short follow-up turns	re-enter after each step rather than one long run	each re-entry is a fresh completion, which sidesteps a stalled one

These reduced exposure but did not eliminate it — every affected turn (37 through 51) occurred while the process measures were already in place. That is the strongest argument for fixing the classification in the adapter rather than expecting callers to work around it.

Also ruled out by direct inspection of the session log (4,564 events):

• output truncation - no chunk.reason === "length" anywhere in the session; the largest single output was 11,214 tokens against a 65,536 budget.
• transport / rate-limit failures - the only three llm/retry events in the session predate the affected range and belong to an unrelated older model configuration.
• context-window exhaustion - the window is 1,000,000 tokens and 41 compaction/prune events ran normally.
• a second, concurrent client instance - process inspection found exactly one client process running.

Local workaround

A small out-of-tree plugin (dsh-llm-empty-guard) resolves the symptom on the affected installation without patching the provider package. It is included here as evidence that the defect is addressable at the adapter boundary, and in case it helps anyone else hitting the same symptom.

Design - delegate, do not reimplement

The plugin deliberately does not implement retry logic. EMPTY_RESPONSE is already the first entry of DEFAULT_RETRYABLE_CODES in @deepseek-ai/dsh-llm, and the stock policy (5 attempts, exponential backoff) already handles it correctly. The only thing missing is that the failure is never raised.

So the plugin does exactly one thing - in the llm/stream waterfall it rewrites a reasoning-only stop finish into the EMPTY_RESPONSE finish the adapter should have emitted, yields the rewritten chunk, and lets the stock retry path do the rest:

if (shouldRewrite(chunk, productive, reasoningChars, config.minReasoningChars)) {
    yield emptyResponseFinish(chunk);   // the only behavioural change
    continue;
}
yield chunk;

Why it does not misfire

• The decision is taken on the terminal chunk only, so a stream that emits reasoning and then text or a tool call is never touched.
• Visibility is decided with the harness's own chunkHasVisibleText rather than a third, private notion of "content" - deliberately avoiding a repeat of the two-definitions disagreement described above.
• Non-agent-loop streams pass through untouched.

Verification - 27/27 assertions pass across unit and waterfall-integration tests. The integration harness drives the real waterfall against a stubbed downstream and asserts the four critical paths: reasoning-only -> rewritten to EMPTY_RESPONSE; reasoning-then-text -> untouched; reasoning-then-tool-call -> untouched; non-agent-loop stream -> untouched.

Expected behavior

Either:

1. a reasoning-only stop maps to EMPTY_RESPONSE and therefore enters the normal empty-response retry path, or
2. the turn is resumed/continued rather than recorded as a successful completion with no visible output.

Environment

• OS: Windows 10 Home (Chinese), 22H2, build 10.0.19045 - this corrects the original report, which said Windows 11. The defect reproduces on Windows 10.
• Node.js: v24.18.1
• CLI: @deepseek-ai/dsh 0.1.5-rc.1
• provider / model: deepseek-official / deepseek-flash
• reasoningEffort: max when observed

[argszero]

你这份报告的分析是准确的，我在 0.1.5 线的 master（c291e7961a）上逐行复核后全部确证，并且有三处可以加强你的结论。

一、比"错误地报告成功"更严重：它同时关掉了重试

EMPTY_RESPONSE 在 DEFAULT_RETRYABLE_CODES 里（packages/llm/llm/src/retry-policy.ts:18-24）：

const DEFAULT_RETRYABLE_CODES = Object.freeze([
  EMPTY_RESPONSE_CODE, 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT',
])

而 @deepseek-ai/dsh-llm-retry（bundle/base/cordis.patch.yml:84、bundle/sdk-minimal:97 都挂了）是监听 agent/request-error 并只看 failure.code（llm-retry/src/index.ts:195/:215/:243）；agent/request-error 的触发点正是 finish.kind === 'error' || 'aborted'（core/agent-loop/src/agent.ts:443-453）。

所以 order.length === 0 这个谓词失效，代价不只是"少一条错误信息"——它让本该重试的退化完成被当成成功接受。这道守卫存在的意义（触发 retry）被它自己的谓词取消了。你日志里那 7 个 turn 本应各有一次重试机会。

二、你建议的谓词已在同一棵树里，而且没有任何消费者

packages/llm/llm/src/assistant-stream.ts:290-293 就是那个"visible"定义：

export function chunkHasVisibleText(chunk: StreamChunk): boolean {
  if (chunk.type === 'text-delta') return hasNonWhitespace(chunk.text)
  return chunk.type === 'block-end' && chunk.block.type === 'text' && hasNonWhitespace(chunk.block.text)
}

它比"排除 reasoning"更严：连纯空白的 text-delta 也不算内容（:361 的 assistantStreamHasVisibleText 按 JSDoc 把 reasoning 与 tool-call 一并排除）。但全仓库 grep 之后，这两个函数没有任何调用点——唯一的消费者是它自己和测试。

也就是说：修 translate.ts:135 不需要新造谓词，甚至不需要新造 helper，只需要让那处 order.length === 0 改用它。顺带一提，pi-ai 适配器也定义了 EMPTY_RESPONSE 谓词（llm-pi-ai/src/stream.ts:99），用的是 message.content.length === 0。

三、同一个形状在 pi-ai 里也存在（顺带修正一处）

llm-pi-ai/src/stream.ts:165-172 把 thinking_start/delta/end 映射成 reasoning 块，而 contentIndex 是与 text/tool-call 同一套块索引（:148 注释原文：两者都从 0 计数块）。所以 message.content 里同样会有一个 reasoning 块，message.content.length === 0 同样不成立。

如果你要对上游提补充，值得把这条一起带上：它不是 DeepSeek 适配器独有的笔误，而是两个适配器共享的同一个谓词错误（"块数"当"内容"用），修法也是同一个。

四、一个可用的止血方案（已发布）

在你等 core 修复期间，我已经把这个缺口做成了插件形态，刚发布到 npm：

npm install @argszero/cordis-plugin-empty-response-guard

- insert:
    - id: empty-response-guard
      name: '@argszero/cordis-plugin-empty-response-guard'

在公开的 llm/stream waterfall 上观察，把"stop 且无可见内容"的 finish 换成 EMPTY_RESPONSE，于是 §1 那条链路重新打通（agent/request-error → llm-retry 会重试）。实现上有三点是刻意的：

• 复用 harness 自己的谓词（chunkHasVisibleText、EMPTY_RESPONSE_CODE），而不是重新推导一份——所以它和 core 对"什么算内容"的定义不可能漂移；
• 不缓冲：判定只需要 finish，而 finish 永远是最后一个 chunk，所以每个 chunk 到达即转发（流式与落盘完全不受影响）；我为此写了一条"若开始缓冲就失败"的测试；
• reasoning 一个字都不动，只改这一轮的分类；reasoning 后面跟了 tool call 属于正常进展，原样放行。

仓库：https://github.com/argszero/cordis-plugin-empty-response-guard（20 条测试，含一条把"order.length 说成功 / harness 谓词说没内容"这个分歧本身钉死的回归）。

五、边界（说清楚）

这是接缝侧的止血，不是 core 修复 —— 真正的修法仍在 translate.ts:135 与 llm-pi-ai/src/stream.ts:99（谓词换成"无可见内容"）。插件只是让失败可见且可重试，不会让退化完成变成一次好回答。