[bug]「只有 reasoning、无可见正文也无工具调用」的响应被判为成功 ⇒ 静默空回复

[luoyukun3-oss]

[Bug] 只含 reasoning 的响应被判为成功完成 ⇒ 静默空回复 / Response with only reasoning blocks is reported as completed ⇒ silent empty reply

摘要

当模型产出一条只含 reasoning block、既没有 text 也没有 tool-call 的最终消息时，Harness 把它当作正常完成：turn/end 记 completed、不报错、不重试、日志里没有任何异常痕迹。用户侧表现为「思考区滚完，正文一个字都没有」——看起来像客户端卡死或断流，而实际请求被判为"成功"。

同一判据缺陷在两个适配器里各出现一次（见「同一缺陷的第二个适配器」）。

环境

• DSH Desktop 0.1.5-rc.2，Windows 11
• provider deepseek-official，model deepseek-v4-flash
• reasoningEffort: high，maxTokens: 256000
• 复现会话：单会话 46 轮，其中 6 条符合下述退化特征

证据

退化消息的共同铁证是 outputTokens 恰好等于 reasoningTokens ⇒ 整个输出预算都花在 reasoning 上，可见正文零 token：

本地时间	turn/step	outputTokens	reasoningTokens	输入侧上下文
16:46:11	39/9	259	259	≈627k
17:10:24	42/1	1007	1007	≈634k
17:11:17	43/1	976	976	≈635k
17:14:09	44/1	537	537	≈636k
17:15:09	45/1	991	991	≈637k
17:17:01	46/1	928	928	≈642k

推理内容本身呈自我催促循环（复读），例如结尾形如：

好。**Go.** 好。（执行。）好。**（输出。）**好。—— 立即。
好。**（我停止循环，输出工具调用。）**好。写。好。**Go.** 好。（我立即输出。）

与「流中断」的区别（同一现象，两个成因）

审计同一批日志时还发现另外 2 条「只有 reasoning」的消息，但它们是另一种成因，混为一谈会开错处方：

成因	判据（可机械区分）	处置
推理退化（本文将报的）	有 usage，且 outputTokens === reasoningTokens；推理文本呈低熵复读	需要模型侧/框架侧干预
流中断	data.interrupted === true，或整条没有 usage；推理文本正常，只是被硬切断（本次两条的思考停在半条 shell 命令上）	传输问题，与上下文体积无关

相关因素（统计，非因果断言）

扫描 18 个会话日志、共 2584 条 assistant/message：

输入侧上下文	消息数	其中「只有 reasoning」	占比
0 – 50k	278	0	0%
50k – 100k	274	0	0%
100k – 200k	538	0	0%
200k – 400k	798	0	0%
400k – 600k	414	0	0%
≥ 600k	280	6	2.14%

• 6 条退化全部落在同一个「上下文 ≥ 600k」的会话里；
• 但大上下文不是充分条件：另有两个会话最大上下文达 791k / 793k，退化 0 条（轮数 56 / 115）；
• 一旦出现就会连续：turn 42 → 43 → 44 → 45 → 46 连中五轮 ⇒ 疑似「空消息被写回上下文后自我强化」（模型看到自己上一轮"只想了没说"，更容易重复）。

根因定位（源码）

1. @deepseek-ai/dsh-llm-deepseek

lib/index.js 的 translate() 处理 [DONE] 哨兵处：

const reason = pendingFinish ?? { kind: "stop" };
yield {
  type: "finish",
  reason: reason.kind === "stop" && order.length === 0 ? {
    kind: "error",
    failure: { message: "model returned a completed response with no content",
               code: EMPTY_RESPONSE_CODE }
  } : reason
};

判据是 order.length === 0（一个 block 都没打开）。而 reasoning block 会先 open("reasoning") 并进入 order，因此**「只有 reasoning」的消息 order.length === 1** ⇒ 直接走 reason（stop）⇒ 被判为成功。

另外，@deepseek-ai/dsh-llm 已导出 assistantStreamHasVisibleContent / assistantStreamHasVisibleText（见该包 lib/index.js 导出表），但本 adapter 未使用它们。

2. @deepseek-ai/dsh-llm-pi-ai（同一缺陷的第二个适配器）

lib/index.js 的 mapStopReason()：

case "stop":
  if (message.content.length === 0) return {
    kind: "error",
    failure: { message: `model "${message.model}" returned a completed response with no content`,
               code: EMPTY_RESPONSE_CODE }
  };
  return { kind: "stop" };

同样是只看「块数为 0」。而该 adapter 的 content 里有思考块（pi-ai 原生 message 用 type: "thinking"；completeTerminalMessage() 从流块恢复时用 DSH 风格的 type: "reasoning"）⇒ 只要模型思考过，content.length ≥ 1 ⇒ 判 stop ⇒ 同样静默成功。

修判据时需注意：这里两种类型名都要认（工具块为 toolCall 或 tool-call，正文块两条路径都是 text）。

影响

• 用户侧完全无感于"出错"：没有报错、没有重试、日志无痕（turn/end 仍是 completed），无法自查；重发也常继续退化（见上表连续五轮）；
• 与另一种故障形态可区分：真·断流（服务端未发 [DONE]）会抛 STREAM_CLOSED，那条路径至少有错误。

建议

1. 主建议：把两个适配器的判据都从「块数为 0」放宽为「没有可见正文且没有工具调用」（dsh-llm-deepseek 可直接改用现成的 assistantStreamHasVisibleContent），并映射为 EMPTY_RESPONSE —— 该错误码已在默认重试白名单内（@deepseek-ai/dsh-llm/lib/types/retry-policy.js 的 DEFAULT_RETRYABLE_CODES），于是这类空回复能被自动重试救回，而不是静默失败。
2. 可选：重试时不要把这条"空消息"原样写回上下文（针对上面的自我强化现象）。
3. 可选（框架层）：目前内置的 guard/ 家族（timeout-policy、repeat-tool-reminder）都以工具调用为中心，而退化的典型形态恰恰是不发工具调用。若在 agent-loop 层增加一个「连续 N 步无可见产出」的守卫，可在重试之外更早地打断（这一层的事前干预比事后重试更省 token）。
4. 附带一问：STREAM_CLOSED（真正的流截断）不在 DEFAULT_RETRYABLE_CODES 里，因此真断流不会自动重试（对比：TRANSPORT 在内，故传输类失败会自动重试）。是否有意如此？若纳入，用户侧"说一半没了"的问题也能显著减少。

自查方法

扫描会话日志（多帧 zstd，需逐帧解压）中所有 assistant/message 记录，判定：

const blocks  = rec.data.message.content;             // 数组
const hasText = blocks.some(b => b.type === 'text'   && b.text?.trim());
const hasTool = blocks.some(b => b.type === 'tool-call');
const hasReason = blocks.some(b => b.type === 'reasoning' && b.text?.length);
const suspect = hasReason && !hasText && !hasTool;    // ← 候选；还需按下表分类

// 分类（两者成因不同，处置也不同）
const u = rec.data.usage || {};
const degenerate = suspect && u.reasoningTokens > 0
  && u.outputTokens === u.reasoningTokens;            // ← 本文所述退化
const interrupted = suspect && (rec.data.interrupted === true
  || Object.keys(u).length === 0);                    // ← 流中断，非本文所报

配合 inputTokens + cacheReadTokens（上下文体积）即可定位高发区间。
2026-09-18-dsh-reasoning-only-empty-message.md

[argszero]

Thanks for the careful report — the classification table (推理退化 vs 流中断) and the outputTokens === reasoningTokens evidence are what made this quick to check. Three things, all verified against the real published packages on 0.1.5-rc.2 (your line) and re-run identically on 0.1.6-alpha.2.

1. This shape already has a shipped plugin fix (it is the same gap as #6218)

The first report of exactly this shape was discussion #6218 (reasoning-only stop judged a success), and a second was #6948 (truncated reply, stop, usage 0/0/0). Both are covered by a published plugin:

@argszero/cordis-plugin-empty-response-guard@0.2.1 — npm · source

I installed that artifact from the registry into a clean project next to dsh-llm@0.1.5-rc.2 and drove your degenerate stream through it (reasoning deltas, then usage with outputTokens === reasoningTokens === 259, then stop):

step	finish	failure	what the loop does
plugin not mounted	stop	—	success path: assistant/message assembled, turn/end = completed — your bug
0.2.1 mounted	error	EMPTY_RESPONSE	real @deepseek-ai/dsh-llm-retry returns {"kind":"retry"} and appends llm/retry + llm/retry-started

Also checked in the same run: the reasoning bytes pass through untouched (the rewrite replaces the terminal chunk only, stream length unchanged, nothing is buffered), a healthy attempt (reasoning + visible text) is byte-identical, a tool-call attempt is untouched, and the shape-2 half from #6948 still fires.

One structural point that saves you the second patch: the plugin sits at the llm/stream waterfall, which lives on LlmRuntime itself — declared at packages/llm/llm/src/index.ts:72, dispatched at :1120. Every adapter's stream passes through it, so one mount covers both dsh-llm-deepseek and dsh-llm-pi-ai; the two per-adapter predicate fixes are not required for the behaviour you are asking for (they are still the right upstream fix, for the classification to be correct without the plugin).

2. Correction: suggestion 1 would not change anything

I measured the two helpers you named (assistantStreamHasVisibleContent / assistantStreamHasVisibleText, real dsh-llm@0.1.5-rc.2, compact records):

case	HasVisibleContent	HasVisibleText
empty (usage + finish only)	false	false
reasoning only — your case	true	false
reasoning + visible text	true	true
tool call only	true	false
visible text only	true	true

assistantStreamHasVisibleContent answers true for precisely the completion you want flagged, so swapping order.length === 0 for it is a no-op. The reason is the predicate underneath it: isVisibleChunk counts reasoning as content by contract — its JSDoc reads "Text and reasoning count only with non-whitespace content" (dsh-llm/lib/index.js:1270-1285). The narrow sibling chunkHasVisibleText is the one that excludes it ("Reasoning, Tool calls, and other block kinds never count", :1287-1295) — but it also returns false for a legitimate tool-call-only completion, so it cannot be the whole predicate either. What distinguishes them at that call site is the finish kind: a tool call terminates with tool-calls, not stop.

Second, independent problem: both assistantStream* helpers read the compact records of an already-settled durable Assistant message ({type:'chunk',chunk} records and packed text/tool-call runs), not the live chunk sequence your order.length predicate sees. Handed the live sequence, assistantStreamHasVisibleContent throws (TypeError: Cannot read properties of undefined (reading 'length')) and assistantStreamHasVisibleText silently returns false. So neither is callable where the predicate lives. The shape that belongs there is: this stream carried no non-whitespace visible text and no tool call.

3. Suggestion 2 is already satisfied by the retry path

The retry decision happens before the assistant message is assembled, so a corrected degenerate completion is never written back into the conversation:

• packages/core/agent-loop/src/agent.ts:444 — an error/aborted finish dispatches agent/request-error (:450);
• :461 — on anything other than {kind:'retry'} the loop throws; on retry it continues at :464;
• createAssistantMessage is only reached at :467 on the non-retry path.

What a retried attempt does persist is assistant/attempt — and that is not a message: SurfaceEventType is exactly system/message | user/message | assistant/message | tool/result (packages/core/session/src/types.ts:417-421), and deriveMessages() builds the request history from surface nodes only (packages/core/session/src/index.ts:841-858). So the empty turn cannot be echoed back to the model, and your 自我强化 hypothesis is not reachable through the retried attempt. If retries are exhausted the loop throws instead, so the turn ends visibly rather than silently.

(Where your hypothesis would be observable: the plugin's warn mode deliberately leaves the finish as stop for diagnosis — that is the one configuration where a degenerate completion still enters the transcript.)

Two boundaries worth knowing

• The guard fires for a reasoning-only stop regardless of usage — a stop with nothing to show is degenerate however the provider accounts for it. So a hard-cut stream that still ends in stop is also classified and retried: the outcome matches your 流中断 处置, but the code name (EMPTY_RESPONSE) reads as "answer missing" rather than as a transport failure. If your logs show interrupted markers, those belong on the transport side, which is where A provider can truncate a response mid-stream and still send finish=stop — mapStopReason() records the turn as completed when content is non-empty but usage is 0/0/0 #6948's shape-2 code (TRANSPORT) sits.
• Nothing in the guard depends on your ≥600k correlation, so it cannot mis-fire on the long-context sessions that stay healthy — the verdict is per completion.

If you would rather not mount a plugin, the upstream fix you propose still stands and is the better long-term shape; the plugin is the version that works today. Its 0.2.1 peer range claims the 0.1.3-alpha.2, 0.1.5.x and 0.1.6.x lines, and I re-ran the two probes above against one release per line (0.1.3-alpha.2, 0.1.5-rc.2, 0.1.6-alpha.2) — 18/18 expectations each, with the same helper table.

[PerryLink]

是有意的，而且被测试锁住了：STREAM_CLOSED 不在默认重试集合里，并且有一条专门为这个行为命名的用例。

• 默认集合：packages/llm/llm/src/retry-policy.ts:18-24 定义 DEFAULT_RETRYABLE_CODES = [EMPTY_RESPONSE_CODE, 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT']，没有 STREAM_CLOSED。
• 用例：packages/llm/llm-retry/tests/transport-recovery.spec.ts:178 的标题就是 "exposes a clean partial EOF as non-default-retryable STREAM_CLOSED"；里面只发生一次请求（:192）、没有任何 llm/retry 事件（:196）、turn/end 直接带上 { message: 'SSE stream ended without [DONE]', code: 'STREAM_CLOSED' }（:199）。

为什么这个默认值说得通

关键在"干净的 EOF"这四个字。同一个用例喂的是 partial_eof + partialText: 'discarded clean eof'（:179-183），也就是服务端已经吐出过一部分正文再收流；框架记下了这条 attempt 的 3 个 chunk（:194），但因为没有 [DONE]，它选择不提交任何 assistant message（:195）。这种情况下"流被截断"和"服务端答完但丢了终止符"无法区分，而两种成因的代价完全不同：

• TRANSPORT 是响应之前的传输失败（packages/llm/llm-deepseek/README.md:112），什么都没产生，重发是安全的，所以它在默认集合里。
• STREAM_CLOSED 出现在流的尾部：packages/llm/llm-deepseek/src/protocols/chat-completions/sse.ts:20-23 的契约写的是 "truncated response — the model call cannot be trusted"，抛出点在 :39；pi-ai 侧同义（packages/llm/llm-pi-ai/src/stream.ts:233）。此时自动重发意味着再付一次完整请求（你报的场景上下文已经很大），而且无法排除把一次已经产生过部分输出的尝试整个重来。

对比之下，你主报的那类空回复能进白名单，是因为它的判定落在终止块上、结论是"这次没有产出可见内容"，重试方向明确；STREAM_CLOSED 的结论是"不确定产出了什么"。把默认值放在保守一侧，代价是断流要用户自己重发，收益是不会替你发出一个可能重复的请求。

想让它自动重试

改 provider 路由上的 retryPolicy——它是 @deepseek-ai/dsh-llm-deepseek 配置块的字段（章节标题见 docs/config-catalog.md:1148，retryPolicy 声明在 :1201；该配置块就是 cordis.yml 条目可设的 config:，见 docs/config-catalog.md:6；默认值一行在 packages/llm/llm-deepseek/README.md:71）。形状大致是：

retryPolicy:
  mode: normal
  retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, STREAM_CLOSED]

两点要注意：

1. 这是替换而不是追加。只有省略 retryableCodes 时才取默认数组，给了值就整份用你给的（packages/llm/llm/src/retry-policy.ts:166；schema 默认在 :90）。所以原来的 5 个要一起写上，否则会顺手关掉 EMPTY_RESPONSE 的重试（该码的字面值就是 EMPTY_RESPONSE，见 packages/llm/llm-retry/tests/transport-recovery.spec.ts:167 的断言）。校验规则是非空、只含非空字符串、不能重复（retry-policy.ts:170-178）。
2. 另一个更宽的开关是 mode: 'always'：它重试每一个模型请求失败（retry-policy.ts:48-54），决策点在 packages/llm/llm-retry/src/index.ts:199-217，并且 maxRetries 只在 normal 模式下生效（:223）。它当然会覆盖 STREAM_CLOSED，但也会同样去重试 AUTH、INVALID_REQUEST 这类重试没有意义的失败——为了这一个错误码换过去，我不建议。

[sprainJinyu]

补充本机量化证据（与本帖同族：接受侧把"无实质正文"的结束当成功）：

环境：macOS 自研 WKWebView 桌面壳 + dsh web（:3080），0.1.6-alpha.2 系 fork。
方法：全量解码本地会话日志（多帧 zstd 逐帧 decompress），扫描 5 个 workspace、5008 条 assistant/message、319 个 turn/end。

1. "纯 reasoning、无可见正文、无工具调用"形态：本地 0 例（该静默空回复形态在本机未复现）。
2. 但同族变体命中 1/319（≈0.3%）：末条 assistant 为过程旁白体（"让我…/首先…/Let me …"）、无 tool-call，直接 turn/end 被判成功；用户须发"继续"才继跑，最终结果正确。即"有可见文本但无实质答案"也会穿过成功判定。
3. 持久化层无串道/损坏（reasoning 错合并进 text 0 例、工具被跳过即 turn/end 0 例），可排除 [Bug] rc.7: reasoning (thinking) intermittently stored and rendered as text blocks - thinking leaks into the visible transcript #3021 存储型泄露；问题集中在结束判定。

建议：成功/停止守卫按"实质答案"判定而非"是否有可见文本"：末条为旁白体或纯 reasoning 且本回合存在未完成工具意图时，判 incomplete 并自动续跑一次或提示重试；否则长任务中模型提前 stop 会把过程旁白留作"最终答案"。