[bug] Compaction sends the tool schemas with no tool choice, so a tool-capable model answers with a tool call and the summary is lost silently

[mattafaak]

Summary

compaction-basic sends the conversation's tool schemas on the summarization
call and sets no tool choice. A tool-capable model, deep in a session where
every turn has been a tool call, answers the "compaction engine" instruction with
another tool call. summaryText() keeps only text blocks, finds none, and
throws summarization produced no text summary content — and the raw output is
never logged, so nothing downstream can say why.

This is a different failure from the token-cap one in #3201 / #3465 (that one is
reasoning eating the 8,192-token output budget, and Yunado's analysis there
matches what I measured too). Both can happen on the same model.

Evidence

74 compactions across 432 sessions, qwen3.8-27b (Q5_K_XL) via llama.cpp b10751
on an OpenAI-compatible endpoint, dsh 0.1.2-alpha.5. 25 produced no summary:

n	compaction/end error	duration	note
15	summarization produced no text summary content	3–17 s	the duration of a tool call, not a summary. 12 of them in one session
5	400 status code (no body)	0.1 s	a first prompt over the window, unrelated
2	summarization truncated at the token cap	266 s, 397 s	the #3201 / #3465 class, on a thinking lane
1	summary is not smaller than the shadowed content	—	correct refusal
1	pi-ai stream idle timeout after 300000ms	300 s	lane wedged

I put a capture proxy in front of the endpoint to see the request itself. The
summarization call:

POST /v1/chat/completions
  messages: 108   (system + the shadowed region + the compaction instruction)
  tools:    28    <-- the conversation's full tool set
  tool_choice: absent
  max_tokens: 8192

The short durations are the tell: 3–17 s is a tool call being emitted, not 2,900
tokens of summary being written (a successful one on the same lane takes 131 s).

Where it comes from

buildSummarizationInput() reconstructs "the last routed request's cacheable
prefix: its system prompt and tool schemas", and summarizeWithLlm() streams
with tools: [...input.tools]. That is deliberate and reasonable — it keeps the
call a genuine prefix so the provider's KV cache is reused. The gap is only that
nothing tells the model not to use them.

Why it is easy to misread

It retries at every step, so the context keeps growing until the provider rejects
the request outright. From the outside it looks like a context-length problem.

Suggested fix

Either would do:

1. toolChoice: "none" on the summarization call. Best outcome: the prompt
   stays a cache prefix (no re-prefill) and a text answer is certain. It needs
   plumbing — GenerateOptions has no toolChoice and dsh-llm-pi-ai builds
   the pi-ai options field by field — though pi-ai itself already maps
   options.toolChoice onto tool_choice.
2. Log the raw blocks on the failure path. Even without a fix, one line
   naming what came back instead of text turns weeks of silent loss into an
   obvious diagnosis.

Workaround, if it helps anyone before then

A dependency-free plugin on the llm/stream waterfall that deletes tools from
calls whose purpose is "compaction":

export const inject = ['llm']
export function apply (ctx) {
  ctx.on('llm/stream', function (options, next) {
    if (options?.purpose === 'compaction' && options.tools?.length) delete options.tools
    return next()
  }, { global: true })
}

Loaded by file:// URL from a cordis.patch.yml row. It works — 100% of
compactions have produced summaries since — but it costs a full re-prefill of
the condensed region, because the schemas sit inside the system region of Qwen's
template: about two minutes per compaction here. toolChoice: "none" upstream
would avoid that cost entirely.

[argszero]

已在 repo 当前 master（76fda72979）复核，你的根因判定成立 —— 这是一个真实的、可复现的数据丢失缺陷，而非 token-cap 那类。

根因确认（packages/compaction/compaction-basic/src/summarizer.ts）：

• 第 158 行把对话的工具 schema 透传给汇总调用：

  ...input.tools === undefined ? {} : { tools: [...input.tools] },

• 但 GenerateOptions（packages/llm/llm/src/types.ts:393）没有 toolChoice / tool_choice 字段，更没有任何"禁止工具调用"的开关。于是工具能力强的模型可以把「compaction engine 指令」当成一次工具调用作答；summaryText()（summarizer.ts:169）只保留 text 块，找不到 text → 抛出 summarization produced no text summary content。

关键点：这不是"把 tool_choice: none 一行加上"就能修的。 因为 GenerateOptions 类型面根本没有该字段，要落到方言层需要：

1. 给 GenerateOptions 加一个（例如 toolChoice?: 'none' | 'auto' | 'required' | { type: 'function' } 之类的）字段；
2. 在各个 provider adapter 里把它映射到自己的 tool_choice 方言；
3. 汇总调用设 toolChoice: 'none'。

或者更窄的修法：汇总调用不传 tools（第 158 行整个省略）——但这也丢掉了"保留 warm-prefix cache"的初衷（注释 111-114 说明 replay 工具 schema 是为了复用 prefix cache）。所以这个 bug 牵涉跨包（dsh-llm 类型面 + 各 adapter），比表面看着重。

顺手补充一个你能对照的独立证据：与 token-cap（#3201/#3465，Yunado 的分析）确实正交 —— 那个是 output budget 被 reasoning 吃掉、summaryText 仍能找到部分 text；这个则是"模型根本没产出 text"，直接命中第 171 行的抛错分支。你的 74/432 出现 25 次、其中 15 次是"3–17s 的工具调用时长"这个取证非常到位，刚好打在这条分支上（而非 400/截断那两条）。

可能的另一层追问（如果你要报 upstream）：第 171 行 throw 前，rawOutput 从未落日志 —— 所以真出了「模型答成工具调用」时，用户看到的只是这条笼统 error，不知道它其实是"模型选错了行为"。这放大了排查难度。是否值得在抛错前把 rawOutput 或 pending tool_call 记到 compaction/raw 之类可观测事件里，是个可讨论的点（与你贴的 #5524 里"compaction 进行中无信号"是同一家族）。

一句话：缺陷确认、根因在 summarizer.ts:158 + GenerateOptions 无 tool_choice 字段、修复面跨包；建议在回复里把我上面这条 GenerateOptions 缺字段的取证一起列上，upstream 处理时能绕过"为什么不能直接一行加 tool_choice"的来回。

[mattafaak]

Status: CONFIRMED upstream. argszero: "this is a real, reproducible data
loss defect, rather than the token-cap class", and named two fixes:

1. add toolChoice to GenerateOptions, map it across the provider adapters,
   and set toolChoice: 'none' on summarization calls; or
2. omit the tools entirely from the summarization request — "but sacrificing
   KV cache reuse".

We have been running option 2 in production since 2026-09-02, so the cache
cost that makes it look unattractive is measurable rather than hypothetical.

What we did

A llm/stream waterfall plugin drops the tool schemas from any call carrying
purpose: 'compaction'. It needs no preset fork and no patched vendor code —
the seam is enough, which may itself be worth knowing: a host-plane fix reaches
compaction under --profile headless, but under dsh web the app disables the
host-plane compaction-basic and the standard preset mounts its own inside
an isolated group, so a config-only fix cannot reach the acting instance there.
The llm/stream waterfall reaches both.

The numbers, on one box

Lane: qwen3.8-27b-vl on llama.cpp, 131k window, one slot, prefill ~815 t/s.

compactions with no summary, before the fix	25 of 64 (39%)
of those, this defect's signature	15 of 25 (12 in a single session)
compaction trigger, median input	65k tokens of a 131k window
cost of one compaction	~208 s, once per ~54k tokens of growth
— of which summarizing	131 s (decode-bound, ~22 t/s)
— of which re-prefill after the drop	77 s

So option 2 costs about 77 s per compaction on this hardware, roughly a
third of the total. That is the price of the cache miss, in full.

It is cheaper than it looks, because the tool RESULTS are the bulk

Measured on a real compaction here, 87% of the summarization request was tool
results, not schemas or assistant text. Capping oversized results inside the
summarized region at 2,000 characters (the default vpimshin's
dsh-qwen38-local-qol uses, adopted here) makes the request 43% smaller and
takes that re-prefill from 88 s to 50 s.

Two things follow for the fix choice:

• The KV-cache argument against option 2 is real but bounded and shrinking —
  and it is a one-off per compaction, against a defect that silently loses the
  summary 39% of the time.
• Option 1 is still strictly better, because it keeps the prefix AND fixes the
  defect. We would rather delete our shim than keep it.

The failure is silent, which is the part that hurt

summaryText() keeps only text blocks. When the model answers the "compaction
engine" instruction with a tool call there is no text, the plugin throws "no
text summary content", and it retries until the provider returns 400. Nothing
in the session log distinguishes that from a slow compaction. If toolChoice
lands, logging the raw blocks on the failure path would still be worth it —
that is the difference between diagnosing this in an afternoon and in a week.

Related: #3201, #3465 (token-cap class — different, as you say), #5524
(observability).

Still present in 0.1.2-rc.1 (checked 2026-09-03)

This box upgraded alpha.5 → rc.1 and the defect is unchanged. Checked in the
shipped packages, not inferred:

	alpha.5	rc.1
toolChoice in @deepseek-ai/dsh-llm	0	0
tool_choice occurrences	2	2 (both in one doc comment)
@deepseek-ai/dsh-compaction size	92,340 B	92,283 B

And rc.1's own source states the cause, which is the neatest confirmation of
argszero's diagnosis available:

GenerateOptions sampling is temperature/maxTokens/stop only — no
tool_choice, top_p, or penalty fields; the vocabulary grows when a
producer la[nds]…

So the field genuinely is absent by design today, the summarization call has no
way to say "do not call a tool", and the shim stays for now.

[argszero]

你这个确认非常有价值——production 数据（39% 静默丢摘要、单会话 12 次、87% 请求是 tool results）把 #5521 从"我觉得是缺陷"钉成了"可量化、且被实际运维承受的缺陷"。几点补充：

1. 你的 llm/stream waterfall 插件方案本身就是一个重要的插件化 seam 证据。 你说的一句很关键："the seam is enough"——这正好印证了这个缺陷可以（且正在）被插件形态兜底，而不用改 harness 源码。这正是插件化贡献的价值：llm/stream 是个 waterfall 事件，第三方插件可以在那里把 purpose === 'compaction' 调用的工具 schema 剥掉。这比"等官方加 toolChoice"更快缓解，且不需要 preset fork。我注意到它还绕开了 dsh web 下 host-plane compaction-basic 被禁用/standard preset 在隔离组里挂载的问题——这让 config-only 修复够不到 acting instance，而插件瀑布能。这个洞察值得单独展开（它说明插件能触及 core 够不到的运行时路径）。

2. Option 1（toolChoice）根子上对，但确实是跨包改动。 你在 rc.1 里查到的证据（tool_choice 0 处、GenerateOptions 无该字段）和我两周前在 master 源码层看到的一致。所以"加 toolChoice 到 GenerateOptions + 各 adapter 映射 + compaction 调用设 none"是三个文件的类型面 + 各 provider adapter 方言的改动，不是一行。这也是为什么插件瀑布（option 2 的 shim）在官方修复落地前是合理且必要的中间态——它正确地 trade-off 了 77s/compaction 的 cache miss 去换 39% 的静默数据丢失。

3. 你"failure path 应记录 raw blocks"的建议我完全同意，且和我最初回复里"抛错前把 rawOutput 记进 compaction/raw 可观测事件"是同一诉求。 现在抛错时 rawOutput 被直接丢弃，用户看到的就是一句笼统的 no text summary content，完全无法区分"模型选错行为"和"真没文本"。这条如果 upstream 采纳，能把这个缺陷的 next-failure 诊断成本从"一周"降到"一下午"（你说的）。

4. 一个可讨论的延伸：tool-choice 缺字段不只影响 compaction——任何"带 tools 但不应调工具"的合成调用（如 #5521、以及可能的 prompt-diagnostics/评估类调用）都有同类风险。如果 upstream 决定加 toolChoice，建议它是 GenerateOptions 上的通用可选字段，而不是为 compaction 特设——这样能一次覆盖所有同类调用。

一句话：Option 2 是当下正确的权衡, Option 1 是长期正确的修复, 且你的 waterfall 插件方案本身证明了这是插件化可挂载面。如果你愿意, 这个 llm/stream 瀑布剥 compaction tools 的做法, 正好可以做成一个可发布到 npm 的独立插件（dsh-compaction-tools-gate 之类的名字, 挂 llm/stream 事件）——需要的话我可以帮你评估发布形态（独立 repo + dsh.bundle + publish）。这也符合社区"讨论 → 插件 → 提及"的循环。

[mattafaak]

Thank you — the summarizer.ts:158 / types.ts:393 pinpointing is exactly
what the report was missing, and your point 4 lands on a real gap in our
mitigation, not just upstream's.

Our shim is compaction-only, by construction. Its gate is literally

if (!options || options.purpose !== COMPACTION_PURPOSE) return { action: 'pass' }

so it does nothing for any other synthetic call that carries tools and must
not use one. We narrowed it deliberately — a llm/stream waterfall sees every
request on the box, and a broad tool-stripping rule there is a footgun — but
that narrowness is precisely why it cannot substitute for the field. A
general optional toolChoice on GenerateOptions is the right shape, and if
it lands compaction-specific we would still be carrying this plugin for the
next call of the same class.

Confirming your master reading against the shipped rc.1, so the evidence
is not only from source: in @deepseek-ai/dsh-llm at 0.1.2-rc.1, toolChoice
occurs 0 times, and the only two occurrences of tool_choice are inside a
doc comment that states the constraint outright —

GenerateOptions sampling is temperature/maxTokens/stop only — no
tool_choice, top_p, or penalty fields

— and @deepseek-ai/dsh-compaction-basic still ships the
summarization produced no text summary content throw. So alpha.5 → rc.1
changed nothing here, and the type surface documents its own gap.

On logging rawOutput before the throw: strongly agree, and it is the
cheaper half of the fix. The toolChoice change is cross-package and will
take a release; the log line is local to summarizer.ts and would have
collapsed our diagnosis from days to one session. Right now
no text summary content is indistinguishable from a genuinely empty
completion, a truncation, and a model that chose to call a tool — three
different bugs behind one string. If only one of the two lands first, we would
rather it were this one.

On packaging the shim as a plugin — done, and thank you for the nudge.

• npm: dsh-compaction-tools-gate (0.1.0, MIT)
• source: mattafaak/dsh-compaction-tools-gate

Zero dependencies, nothing imported from @deepseek-ai, 36 checks that need
no network and no model. It ships a dsh.bundle.patch so the llm/stream
insert row comes with it. Releases go out by OIDC trusted publishing from a
tagged CI run, so there is no long-lived npm token anywhere.

The README says three things I want to be explicit about here too, because a
workaround that oversells itself is worse than none:

1. It is a stopgap and should be deleted when toolChoice lands. That is
   in the README's first paragraph, not a footnote.
2. It prices its own cost. Dropping the tools costs the prefix cache the
   summarizer replays them for — 77 s of re-prefill per compaction on our
   hardware. Anyone on a slower lane should do that arithmetic before
   installing it.
3. It is compaction-only and cannot substitute for the field, per your
   point 4 above.

If any of the framing there misrepresents the harness's behaviour, tell me and
I will fix it — you have read this code far more recently than I have.

One correction to my own earlier numbers, in the interest of not leaving a bad
figure in the thread: the "87% of the request was tool results" and the
88 s → 50 s prefill figures are from one captured compaction, not a
distribution. The 25-of-64 failure count and the 208 s per-compaction cost are
over 74 compactions and are the sturdier of the two.