[Bug] Compaction drops reasoning effort and busts KV cache; idle watchdog cancels active prefill / 压缩丢失推理强度并破坏 KV 缓存

[ohcedar]

English

Summary

dsh-compaction-basic can invalidate the provider's warm KV prefix when the active session uses a per-request reasoning effort different from the provider default. The compaction summarizer carries provider and model, but not reasoningEffort. For providers whose chat template encodes reasoning effort, the auxiliary compaction prompt diverges near the beginning and the intended prefix-cache reuse is lost.

A second problem makes this especially painful for local models: dsh-llm-pi-ai cancels a stream after 300,000 ms without a parsed model chunk. A local server can still be actively prefilling a large prompt, but transport keepalives or prompt-progress events do not appear to pulse this watchdog. The Harness cancels healthy work before the server's longer request timeout.

Environment

• DeepSeek Harness 0.1.7-rc.1, Web profile
• dsh-llm-pi-ai, OpenAI-compatible local endpoint
• 256K-context Qwen model served by Splash on Apple Silicon
• Active session reasoning effort: xhigh
• Provider profile default reasoning effort: medium
• Provider server request timeout: 3,600 seconds
• Harness streamIdleTimeoutMs: default 300,000 ms

Reproduction and evidence

The normal conversation used xhigh. Automatic compaction started around 149K input tokens. Four compaction attempts were observed:

Attempt	Input tokens	Cached tokens	Approx. uncached input	Result
1	111,303	0	111,303	Harness cancelled after 300 s
2	112,925	53,248	59,677	Harness cancelled after 300 s
3	112,925	55,296	57,629	Harness cancelled after 300 s
4	112,925	94,208	18,717	TTFT 152.4 s; generated 5,541 tokens; succeeded

The successful request took about 342 seconds in total, but it survived because output began after 152.4 seconds and subsequent chunks reset the idle watchdog. The cancelled attempts appear to have progressively populated a cache for the compaction prompt, eventually allowing attempt 4 to reach its first token in time.

A normal request immediately after compaction showed the same independent watchdog behavior: a 57,413-token cold request was cancelled after 300 seconds; its retry reused 8,384 tokens and succeeded with TTFT 279.4 seconds.

Code path

The current summarizeWithLlm() chooses a target containing only provider and model, and its GenerateOptions omits reasoningEffort:

https://github.com/deepseek-ai/deepseek-harness/blob/master/packages/compaction/compaction-basic/src/summarizer.ts

The Pi AI adapter then resolves:

options.reasoningEffort ?? profile.reasoning

The repository's own compaction design note says the replayed request should reproduce the warm prefix verbatim. Omitting a template-affecting request option violates that invariant:

https://github.com/deepseek-ai/deepseek-harness/blob/master/.agents/notes/archived/bug-fix/2026-07-21-compaction-summary-prefix-cache-reuse.md

For this model, medium and xhigh produce different system-template text almost immediately, so the whole prior prefix becomes unusable.

Expected behavior

1. A same-provider/model automatic compaction should inherit the effective reasoning effort of the routed request whose prefix it replays.
2. Long active prefills should not be mistaken for dead streams merely because no generated token has arrived yet.

Suggested fixes

• Carry the effective reasoningEffort from the routed request header into the summarization call. More generally, preserve every request option that can alter prompt rendering.
• Add a regression test where session effort differs from provider default and assert that the summarization call preserves it.
• Let provider progress events or transport activity pulse the idle watchdog, or distinguish prompt-processing idle from a dead connection.
• Consider a separate/default-longer idle bound for purpose: "compaction", since these requests are predictably the largest.

Workaround

Users can set a larger streamIdleTimeoutMs in their provider profile, but that only prevents premature cancellation. It does not repair the cache-breaking reasoning mismatch.

---

中文

摘要

当当前会话使用的逐请求推理强度与 provider 默认值不同时，dsh-compaction-basic 可能会使 provider 已预热的 KV 前缀缓存失效。压缩摘要请求只继承了 provider 和 model，没有继承 reasoningEffort。对于把推理强度写入聊天模板的 provider，辅助压缩请求会从提示词开头附近就发生变化，因此无法实现预期的前缀缓存复用。

第二个问题会让本地模型尤其容易失败：如果连续 300,000 ms 没有收到已解析的模型数据块，dsh-llm-pi-ai 就会取消流。此时本地服务器可能仍在正常处理一个很长的输入，但传输 keepalive 或提示词处理进度似乎不会重置该 watchdog。因此 Harness 会在服务器自身更长的请求超时之前取消仍然健康的工作。

环境

• DeepSeek Harness 0.1.7-rc.1，Web profile
• dsh-llm-pi-ai，本地 OpenAI 兼容端点
• Apple Silicon 上由 Splash 运行的 256K 上下文 Qwen 模型
• 当前会话 reasoning effort：xhigh
• Provider profile 默认 reasoning effort：medium
• Provider 服务器请求超时：3,600 秒
• Harness streamIdleTimeoutMs：默认 300,000 ms

复现与证据

正常会话使用 xhigh。输入约达到 149K token 时触发自动压缩。观察到四次压缩请求：

尝试	输入 token	缓存 token	约需新处理的输入	结果
1	111,303	0	111,303	300 秒后被 Harness 取消
2	112,925	53,248	59,677	300 秒后被 Harness 取消
3	112,925	55,296	57,629	300 秒后被 Harness 取消
4	112,925	94,208	18,717	TTFT 152.4 秒；生成 5,541 token；成功

成功的第四次请求总共约耗时 342 秒，但因为 152.4 秒后已经开始输出，后续数据块不断重置 idle watchdog，所以能够完成。前三次被取消的请求似乎逐步建立了压缩请求自己的缓存，最终使第四次请求能及时产生首 token。

压缩成功后的普通请求也独立复现了 watchdog 问题：一个 57,413-token 的冷请求在 300 秒后被取消；重试复用了 8,384 token，并以 279.4 秒 TTFT 成功。

代码路径

当前 summarizeWithLlm() 选择的 target 只包含 provider 和 model，构造的 GenerateOptions 没有 reasoningEffort：

https://github.com/deepseek-ai/deepseek-harness/blob/master/packages/compaction/compaction-basic/src/summarizer.ts

随后 Pi AI adapter 使用以下逻辑：

options.reasoningEffort ?? profile.reasoning

仓库自己的压缩设计说明强调，重放请求应逐字复现已预热的前缀。遗漏会改变模板的请求选项违反了这一约束：

https://github.com/deepseek-ai/deepseek-harness/blob/master/.agents/notes/archived/bug-fix/2026-07-21-compaction-summary-prefix-cache-reuse.md

对于本次使用的模型，medium 和 xhigh 几乎从 system 模板开头就不同，因此之前的整个前缀缓存无法使用。

预期行为

1. 使用相同 provider/model 的自动压缩应继承被重放请求的有效 reasoning effort。
2. 很长但仍在正常进行的 prefill 不应仅因尚未生成首 token 就被判断为死流。

建议修复

• 将路由请求头中的有效 reasoningEffort 传给摘要请求。更一般地说，应保留所有会改变提示词渲染的请求选项。
• 增加回归测试：会话 effort 与 provider 默认值不同，并断言摘要请求仍保留会话 effort。
• 让 provider 进度事件或传输活动重置 idle watchdog，或者区分“仍在处理提示词”与“连接已失效”。
• 可考虑为 purpose: "compaction" 设置单独或更长的默认 idle 限制，因为压缩请求通常就是最大的请求。

临时解决方法

用户可以在 provider profile 中增大 streamIdleTimeoutMs，这只能避免过早取消，不能修复由 reasoning 不一致导致的缓存失效。

[ohcedar]

Workaround (profile-local plugin) / 可用的临时方案（配置本地插件）

Until the header-inheritance fix lands, the cache-breaking half of this bug can be worked around without forking any package: a small profile-local Cordis plugin that listens on the global llm/stream waterfall and injects the session's folded request-header reasoningEffort (and temperature) into any call with purpose: "compaction" that did not set them explicitly. It mirrors dsh-agent-loop's rule of dropping the effort when adapterDefaults.reasoningEffort === true.

在修复合入之前，本 bug 中「破坏缓存」的那一半可以不动任何包源码、用一个配置本地的小型 Cordis 插件绕过：监听全局 llm/stream waterfall，对带有 purpose: "compaction" 且未显式设置这两个字段的调用，注入当前会话已固化的请求头中的 reasoningEffort（以及 temperature）。当 adapterDefaults.reasoningEffort === true 时，它与 dsh-agent-loop 的做法一致：不注入。

How to install / 安装方法 (tested on 0.1.7-rc.1, web profile):

1. Save this as ~/.dsh/profiles/web/plugins/compaction-reasoning/lib/index.js:

const name = "compaction-reasoning";
const inject = ["sessions"];

function apply(ctx) {
	ctx.on("llm/stream", (options, next) => {
		if (options?.purpose !== "compaction") return next();
		if (options.reasoningEffort !== void 0 && options.temperature !== void 0) return next();
		if (options.sessionId === void 0) return next();
		const session = ctx.sessions.get(options.sessionId);
		if (session === void 0) return next();
		const header = session.requestHeader();
		const config = header?.config;
		if (config === void 0) return next();
		if (options.reasoningEffort === void 0 && config.reasoningEffort !== void 0 && header?.adapterDefaults?.reasoningEffort !== true) {
			options.reasoningEffort = config.reasoningEffort;
		}
		if (options.temperature === void 0 && config.temperature !== void 0) {
			options.temperature = config.temperature;
		}
		return next();
	}, { global: true, prepend: true });
}

export { apply, inject, name };

2. Append to ~/.dsh/profiles/web/cordis.patch.yml — the entry schema supports relative plugin paths anchored beside the patch file, so no package manager is needed:

- insert:
    - id: compaction-reasoning
      name: './plugins/compaction-reasoning/lib/index.js'

3. Restart dsh web. Verify with dsh --profile web --dump-config (the entry appears with its resolved file:// URL).

4. 将其追加到 ~/.dsh/profiles/web/cordis.patch.yml —— 条目 schema 本身支持以补丁文件为锚点的相对插件路径，因此无需任何包管理器（见第 3 步的 YAML）。3. 重启 dsh web，并可用 dsh --profile web --dump-config 验证（条目会显示解析后的 file:// 路径）。

Scope / 适用范围: the listener is preset-independent (registered globally), so it covers every preset and both automatic and /compact-triggered compaction. It does not address the second half of this report (the idle watchdog cancelling active prefills) — that still needs streamIdleTimeoutMs tuning or an upstream fix.

该监听器全局注册、与预设无关，自动压缩与 /compact 手动压缩都会生效。但它不能解决本帖的第二个问题（idle watchdog 取消仍在预填充的请求）——那部分目前仍需调大 streamIdleTimeoutMs 或等待上游修复。

Long-term / 长期方案: the proper fix is still the one proposed above — carry the effective reasoningEffort (and any other template-affecting option) from the routed request header into summarizeWithLlm()'s options, plus a regression test where session effort differs from the provider default.

长期方案仍是楼上建议的做法：把路由请求头中生效的 reasoningEffort（以及其他所有会影响模板渲染的请求选项）带入 summarizeWithLlm() 的调用参数，并补充「会话强度 ≠ provider 默认值」场景的回归测试。