【实践笔记】插件里调模型做结构化输出，请默认把推理关掉——推理 token 和正文共用一份预算

[wangzhanchao883]

这个坑我踩过两次，写出来省得别人再绕一遍。

插件里调模型做翻译、抽取、出题这类结构化输出的时候，默认是把推理开着的（DeepSeek 适配器上
默认 high）。模型会先在心里想一大段，而这段思考跟正文共用一个输出预算——预算不够的时候，
全花在思考上了，正文一个字都没写出来。

最坑的不是它出错，是它不出错。请求正常返回，不抛异常，你那边就是拿不到 JSON。
程序一看"没报错"，也就不重试了，最后表现成"模型没返回这个单词"，你还以为是提示词写坏了。

改起来很简单：插件里所有调模型的地方，显式传 reasoningEffort: "off"，别靠宿主默认。
翻译、抽取、打标签、出题、分类、写短文案，这些都是照规则干活的活。真正需要多步推演的
（规划、难题求解）才留着。

还有一条，别拿"我本地测过了"当证据。本地那个模型不思考、宿主路由思考，这类失败你根本测不出来。
要测就用宿主实际路由的模型，或者至少强制开着推理跑一轮。

--- 以下是AI助手提供的技术细节，不关心的可以跳过 ---

定位说明：这是插件作者的实践笔记，不是 bug 报告。下面这个根因，针对 harness 自身辅助调用的部分
官方已被通知过（#3468 会话标题、#6797 压缩摘要）。我没找到的是另一半——第三方插件该怎么做。
这篇补的是这一半。

---

一句话结论

插件里调 ctx.llm.stream() 产出结构化输出（翻译、抽取、打标签、出题、分类、短文案）时，
显式传 reasoningEffort: "off"。DeepSeek 适配器上默认是 high，而推理 token 和你的正文
共用一个 maxTokens 预算。

失败是静默的：请求成功、不抛异常，你就是拿不到可用输出。

---

与既有帖子的关系（以及差异）

• Session titles silently fail to generate on reasoning (thinking) models — no config way to disable reasoning for the auxiliary title call #3468 —— 会话标题辅助调用：报的是核心辅助调用没有关推理的配置口。同模式，对象是核心组件。
• [Bug] Compaction summarization always fails with thinking models — default maxTokens=8192 is consumed by reasoning tokens #6797 —— 压缩摘要：「default maxTokens=8192 is consumed by reasoning tokens」。同根因，对象是核心组件。
• Automatically disable thinking during context compression #6510 —— 功能请求：压缩时自动关思考。同思路，仅限压缩。
• Bug: compaction drops same-route reasoning effort and loses cache compatibility #3002 —— 压缩丢失 reasoningEffort；评论区指出 pi-ai 路径上推理上限正是按 reasoningEffort 门控的。
• [dsh-llm-deepseek] A completed response with no visible content passes as a legal `stop` — the empty-response guard can never fire #6532 / Bug: reasoning-only completions are reported as successful - the EMPTY_RESPONSE guard tests order.length, which counts the reasoning block #6218 —— 空响应守卫不触发，因为一个流式 reasoning token 就会打开一个块。
  这正是把「输出被饿死」变成「静默成功」的机制。

以上都不是讲插件自己的模型调用。如果维护者认为已被覆盖，说一声，我把它并进相关帖。

---

我们遇到的现象

插件：dsh-word-vault（生词记忆卡生成）。任务：为一批 6 个单词生成结构化 JSON 卡片，maxTokens = 4500。

结果：6 个全部返回「模型未返回该词」。

• 没有抛异常。
• 解析器兼容了我们能想到的所有 JSON 形状。
• 同一提示词直连 API、用无推理模型（deepseek-chat）跑，6/6 成功。

也就是说：请求没问题，解析没问题——模型压根没吐出 JSON。

---

根因

来自 @deepseek-ai/dsh-llm-deepseek README 原文：

「生成的 token 遵循请求中记录的推理强度与 maxTokens」

以及

reasoningEffort: off | low | high | max      # 模型默认 high

推理 token 从和可见正文同一个 maxTokens 预算里扣。我们宿主路由是 DeepSeek-V4-Flash High，
推理阶段把 4500 的预算先花光，JSON 还没开始写就停了。

顺带解释了我们一直归因到别处的一个现象：关掉推理后，同一批调用明显变快。

---

最小复现

同一宿主路由、同一模型、同一提示词、同一 maxTokens：

A) reasoningEffort: "high"（或直接不传——在该适配器上等价）
B) reasoningEffort: "off"

看组装后的 assistant 消息，不要只看「有没有抛错」。A 组应该能看到推理块吃掉预算、
正文被截断或为空。我们 A 侧的表现是解析为空，而不是报错——这正是它难查的原因。

也可以直接观察：每次请求实际生效的调用配置是被记录的——推理强度属于请求信封，不是连接级设置。

---

最费时间的那个坑

我们用一个无推理模型验了提示词，6/6 通过，于是判断插件没问题——「我这儿能跑、宿主那边不行」。
这个差异本身就是 bug。

如果你的开发期模型不思考、而用户的宿主路由思考，你的测试根本看不见这类失败。
请用宿主实际路由的模型验证，至少跑一轮强制开着推理。

---

改法

把插件里所有模型调用收敛到一个函数，默认值只在这一处注入：

export async function collectText(llm, options, signal) {
  // 翻译 / 抽取 / 出卡 / 出题都是「照规则干活」，不是多步推演。
  // 2026-09-16 实测：宿主 reasoningEffort=high 时，推理吃光整个 maxTokens 预算，
  // JSON 从未被输出。
  const opts = options?.reasoningEffort
    ? options
    : { ...options, reasoningEffort: "off" };
  let out = "";
  for await (const chunk of llm.stream({ ...opts, signal })) { ... }
}

调用方仍可显式传 reasoningEffort 覆盖。

在我们这个插件里 cardgen / examgen / translate 都走这一个函数，所以改一处覆盖全插件。
补了测试：默认是 "off"，显式传值被尊重。

---

判断标准

照规则干活的结构化输出 -> 关
（翻译、抽取、打标签、拆词、出题、分类、短文案）

真正多步推演的任务 -> low / high
（规划、难题求解）

推理并不能让「翻译这个单词」做得更好。它只是让你为它付费，并且在预算紧时悄悄拿走你的答案。

---

同一次事故带来的两条配套纪律

1. 即使关了推理也要留余量。不要按预期 JSON 的大小去卡 maxTokens。
   我们用 min(16000, 2500 * 批大小 + 2000)。

2. 「解析为空」必须当失败处理并重试。只 catch 异常不够。
   上面那个响应，按我们当时所有的检查都算成功——没抛错、正常结束——只是里面什么都没有。
   于是代码逐项记了「模型未返回该词」，并且从不重试，因为它认为没出错。
   现在解析为空即失败：批次对半拆开重试。

---

环境

• @deepseek-ai/dsh-llm-deepseek，deepseek-official 路由
• dsh 0.1.5-alpha / rc 线，插件侧 API（ctx.llm.stream）
• 宿主模型 DeepSeek-V4-Flash，reasoningEffort=high
• 修复版本 dsh-word-vault v0.5.3（commit 75eef23）

---

两个插件，相隔八天

值得明说：这个坑我们踩了两次。

• 2026-09-08，另一个插件（简历筛选，deepseek-v4-pro）：「模型把全部 token 预算烧在推理链上，
  无可见输出」。同样方式修好了，但结论只留在那个插件自己的笔记里。
• 2026-09-16，生词库插件：同一根因，又从头查了一遍。

这就是为什么这条经验该写在一个通用的地方，而不是某个项目的笔记里——
也是为什么每个产出结构化输出的插件，都该默认传 reasoningEffort: "off"。

如果你也在写会调模型的 DSH 插件，想知道你有没有遇到、怎么处理的；
以及你认为 "off" 是工具类调用的正确默认值，还是说单开一条不思考的专用路由更干净。

---

[Field note] Structured-output calls in plugins should default to
reasoningEffort: "off" — reasoning tokens share the maxTokens budget

Context first: this is a plugin-author field note, not a bug report. The root
cause below has already been reported against harness-internal auxiliary calls
(#3468 session titles, #6797 compaction). What I did not find anywhere is the
plugin-side half: what a third-party plugin should do about it. That is what
this post is.

---

TL;DR

If your plugin calls ctx.llm.stream() to produce STRUCTURED OUTPUT — translation,
extraction, tagging, question generation, classification, short copy — pass
reasoningEffort: "off" explicitly. Do not rely on the host's default. On the
DeepSeek adapter the default is high, and reasoning tokens are counted against
the same maxTokens budget as your answer.

The failure is silent: the request succeeds, nothing throws, and you simply get
no usable output.

---

Related, and how this differs

• Session titles silently fail to generate on reasoning (thinking) models — no config way to disable reasoning for the auxiliary title call #3468 — session-title auxiliary call: reports no config way to disable
  reasoning for a core auxiliary call. Same pattern, core component.
• [Bug] Compaction summarization always fails with thinking models — default maxTokens=8192 is consumed by reasoning tokens #6797 — compaction summarizer: "default maxTokens=8192 is consumed by
  reasoning tokens". Same root cause, core component.
• Automatically disable thinking during context compression #6510 — feature request to auto-disable thinking during context compression.
  Same idea, compression only.
• Bug: compaction drops same-route reasoning effort and loses cache compatibility #3002 — compaction drops reasoningEffort; the comments point out that on the
  pi-ai path the reasoning cap is gated on reasoningEffort.
• [dsh-llm-deepseek] A completed response with no visible content passes as a legal `stop` — the empty-response guard can never fire #6532 / Bug: reasoning-only completions are reported as successful - the EMPTY_RESPONSE guard tests order.length, which counts the reasoning block #6218 — the empty-response guard never fires because a streamed
  reasoning token opens a block. This is the mechanism that turns "starved
  output" into "silent success".

None of these are about the plugin's own LLM calls. If a maintainer considers
this covered, tell me and I will fold it into the relevant thread.

---

What we saw

Plugin: dsh-word-vault (word-vault / flashcard generator for a DSH setup).
Task: generate structured JSON cards for a batch of 6 words. maxTokens = 4500.

Result: all 6 came back as "the model did not return this word".

• No exception was raised.
• The parser handled every JSON shape we tested.
• The same prompt against the API directly, with a non-reasoning model
  (deepseek-chat), returned 6/6.

So the request was fine, the parser was fine — the model just never emitted the
JSON.

---

Root cause

From the @deepseek-ai/dsh-llm-deepseek README:

"Generated tokens follow the reasoning effort recorded in the request and
 maxTokens."

and

reasoningEffort: off | low | high | max      # model default is high

Reasoning tokens are drawn from the same maxTokens budget as the visible answer.
Our host route was DeepSeek-V4-Flash at High. The reasoning pass consumed the
4500-token budget before the JSON ever started, so generation stopped with
nothing visible.

This also explains the latency we had been blaming on something else — the same
calls got noticeably faster once reasoning was off.

---

Repro recipe

On the same host route, same model, same prompt, same maxTokens:

A) reasoningEffort: "high" (or omitted — same thing on this adapter)
B) reasoningEffort: "off"

Read the assembled assistant message, not just "did it throw". In case A you
should see a reasoning block eating the budget and a truncated / empty answer.
Our A-side showed up as an empty parse, not as an error, which is exactly why it
took a while to find.

You can also watch it directly: every request's effective call config is
recorded — the reasoning effort is part of the request envelope, not a
connection setting.

---

The trap that cost us the most time

We validated our prompt with a non-reasoning model, got 6/6, and concluded the
plugin was fine — "works on my machine, fails on the host". That difference WAS
the bug.

If your dev-time model does not think and your user's route does, your tests
cannot see this class of failure at all. Validate against the model the host
actually routes, or at minimum run one pass with reasoning forced on.

---

Fix

Collapse every model call in the plugin into one function and inject the default
there, so you only have to get it right once:

export async function collectText(llm, options, signal) {
  // Translation / extraction / card generation / question generation are
  // rule-following tasks, not multi-step reasoning.
  // Measured 2026-09-16: with the host at reasoningEffort=high, reasoning
  // consumed the whole maxTokens budget and the JSON was never emitted.
  const opts = options?.reasoningEffort
    ? options
    : { ...options, reasoningEffort: "off" };
  let out = "";
  for await (const chunk of llm.stream({ ...opts, signal })) { ... }
}

Callers can still opt back in by passing reasoningEffort explicitly.

In our case cardgen / examgen / translate all went through this one function, so
one edit covered every model call in the plugin. Added tests: default is "off",
and an explicit value is respected.

---

Decision rule

Structured output from a rule-following task -> off
(translation, extraction, tagging, splitting, question gen, classification,
short copy)

Genuinely multi-step work -> low / high
(planning, hard problem solving)

Reasoning does not make "translate this word" better. It just bills you for it
and, on a tight budget, silently removes your answer.

---

Two companion disciplines from the same incident

1. Leave headroom even with reasoning off. Do not size maxTokens to the expected
   JSON. We use min(16000, 2500 * batchSize + 2000).

2. Treat "parsed to empty" as a failure and retry. Catching exceptions is not
   enough. The response above succeeded by every check we had — no throw, valid
   finish — it just contained nothing. Our code recorded a per-item failure
   ("model did not return this word") and never retried, because from its point
   of view nothing had gone wrong. An empty parse is now a failure: the batch is
   split in half and retried.

---

Environment

• @deepseek-ai/dsh-llm-deepseek, deepseek-official route
• dsh 0.1.5-alpha / rc line, harness-side plugin API (ctx.llm.stream)
• Host model DeepSeek-V4-Flash at reasoningEffort=high
• Fixed in dsh-word-vault v0.5.3 (commit 75eef23)

---

Two plugins, eight days apart

Worth saying out loud: we hit this twice.

• 2026-09-08, a different plugin (resume screening, deepseek-v4-pro): "the model
  burned the entire token budget on the reasoning chain with no visible output".
  Fixed the same way, but the conclusion stayed inside that one plugin's notes.
• 2026-09-16, the word-vault plugin: same root cause, found from scratch again.

That is the argument for writing it down somewhere general rather than in a
single project's notes — and for passing reasoningEffort: "off" by default in
every plugin that produces structured output.

If you build DSH plugins that call the model, I would like to know whether you
hit this and how you handled it — and whether you think "off" is the right
default for utility calls, or whether a separate dedicated route with no
thinking would be cleaner.