Repository navigation
【中文】Compaction 中两处独立缓存失效(补丁 + 回归测试)— Two independent cache breaks in compaction on master @ 687f282e86: patch and regression specs #10573
Replies: 1 comment
|
Fix for both causes above, as a patch on 🔧 FixEnvelope. Note
Prune order (
Pruning the region saves nothing when a summary runs, because the checkpoint shadows it anyway; the summarizer now also reads full-fidelity tool output. The retained tail is re-prefilled after the checkpoint regardless (its position shifts), so pruning it costs nothing extra. 📊 MeasuredReal agent loop, token meter, pruner and compaction-basic; only the provider is simulated. The simulated cache keys a request family on (provider, model, effort, temperature, stop) and reuses the longest byte prefix of any earlier request in that family;
Each cause in isolation: effort only puts every fold at 0–0.8% on At a 512k prefix on V4-Pro off-peak pricing, one fold goes from about $0.22 to about $0.008. A miss costs 30× (Pro) to 50× (Flash) a cache hit per token, so the gap grows linearly with every token of context. Tip The simulator is a worst case for cause 1: an effort mismatch misses everything, as on @bulhenvaster's server. DeepSeek's own endpoint still served the ~18k static head in @deepseekv41flash's data. Same conclusion: the conversation body misses. ✅ Verification
¹ The structural guard passes on both trees by design; see below. Patch: With the patch the compaction suite is green (236 passed), The window-scaling case, verbatim from the new spec ( it.each([32_000, 128_000, 512_000])('bounds the uncached summary prompt independently of a %i-token window', async (window) => {
const run = await runWorkload({ window, steps: 60, effort: 'max', pruner: 'rewrite-only' })
const summaries = run.cache.compactions()
expect(summaries.length).toBeGreaterThanOrEqual(2)
for (const summary of summaries) {
expect(hitRatio(summary)).toBeGreaterThanOrEqual(0.95)
expect(uncached(summary)).toBeLessThanOrEqual(MAX_UNCACHED_SUMMARY_TOKENS)
}
assertNoSilentRewrite(run)
}, 60_000)
🧭 Scope
📦 Patch and reproduce# save the full patch below as compaction-cache-integrity.patch
git checkout 687f282e86 && git apply compaction-cache-integrity.patch
# a full `pnpm install` currently fails behind proxies on cdn.sheetjs.com;
# the filtered install is enough:
pnpm install --frozen-lockfile --filter "@deepseek-ai/dsh-compaction-basic..." --filter "@deepseek-ai/dsh-compaction-tool-result-pruner..."
npx vitest run packages/compaction/Full patch:
|
Uh oh!
There was an error while loading. Please reload this page.
master@687f282e86上仍可复现。原因有两个,彼此独立:① 摘要请求没有继承请求头中的reasoningEffort/temperature/stop;② 新发现:tool-result 修剪器在摘要之前改写了摘要请求要重放的历史,从第一个被改写的字节起全部重新计算。只修复 ① 解决不了 ②。master上 6 个失败,打补丁后全部通过(见下方回答);packages/compaction/共 236 个测试通过,tsc -b与 oxlint 无报错。git apply到687f282e86(即当前master),见下方回答。Important
TL;DR: The summary call misses the prefix cache for two independent reasons. Effort parity, as discussed in #1944 and #3002, fixes the first; the second survives it. The answer below has a patch that fixes both, with regression specs. Through the real agent loop on a simulated prefix cache, the uncached part of every summary call drops to the ~470-token instruction, at 32k, 128k and 512k windows alike (
master: about 22k / 84k / 332k).🔍 Two independent causes
masterreasoningEffort,temperatureandstopfrom the routed headersummarizer.ts#L152-L162vs. the loop'sagent.ts#L695-L702highvsnone(@bulhenvaster)index.ts#L321-L326Both break a guarantee the package README states twice: the replayed prefix "match[es] the conversation's last routed request byte-for-byte" (README#L108, README#L233).
Cause 1: the request envelope
The loop builds every request by spreading the durable request header,
...header.config(agent.ts#L696).summarizeWithLlmreads the very same header (summarizer.ts#L127) but keeps onlyproviderandmodel, sodsh-llmsubstitutes the adapter's default effort. The package's own doc names exactly these fields as cache-relevant (call-config.ts#L1-L7), and the DeepSeek adapter serializes effort asthinkingplusoutput_config.effort(serialize.ts#L147-L157).masterprovider,modelreasoningEffortmaxhightemperature,stoptoolsmaxTokensCause 2: the prune pass rewrites the replayed region
The base bundle mounts the tool-result pruner by default (cordis.patch.yml#L425-L430, 8,192 chars). Once pressure qualifies,
compactIfNeededlandspruneSession()over the whole surface and only then builds the summary input, from the rewritten surface. Oversized tool results admitted since the last checkpoint are still verbatim in the cached prefix, so the first one rewritten becomes the first divergent byte. Effort parity cannot touch this.How a prefix cache bills: a request is charged at the cache-hit price up to the first byte that differs from an earlier request, and at the full price from that byte on.
%%{init: {"flowchart": {"rankSpacing": 28, "nodeSpacing": 24, "padding": 8}}}%% flowchart TB subgraph A ["① cached: last request"] direction LR a1["tools +<br/>system prompt"] --> a2["turns<br/>1 … k"] --> a3["tool result j<br/>original"] --> a4["turns<br/>j+1 … n"] ~~~ a5[" "] end subgraph B ["② master: summary call"] direction LR b1["tools +<br/>system prompt"] --> b2["turns<br/>1 … k"] --> b3["tool result j<br/>rewritten ✗"] --> b4["turns<br/>j+1 … n"] --> b5["summary<br/>instruction"] end subgraph C ["③ patched: summary call"] direction LR c1["tools +<br/>system prompt"] --> c2["turns<br/>1 … k"] --> c3["tool result j<br/>original ✓"] --> c4["turns<br/>j+1 … n"] --> c5["summary<br/>instruction"] end A ~~~ B ~~~ C classDef ref fill:#f6f8fa,stroke:#8c959f,color:#1f2328 classDef hit fill:#dafbe1,stroke:#1a7f37,color:#0f3d1c classDef miss fill:#ffebe9,stroke:#cf222e,color:#6e0b12 classDef fresh fill:#ddf4ff,stroke:#0969da,color:#0a3069 classDef ghost fill:transparent,stroke:transparent,color:transparent class a1,a2,a3,a4 ref class b1,b2,c1,c2,c3,c4 hit class b3,b4 miss class b5,c5 fresh class a5 ghost style A fill:transparent,stroke:#8c959f style B fill:transparent,stroke:#8c959f style C fill:transparent,stroke:#8c959f🟩 served from the cache · 🟥 computed again at full price · 🟦 new input, unavoidable · ⬜ what the cache already holds
In the 128k run measured in the answer below, with the effort already aligned, the red part of the first summary call was 73,241 of 83,945 tokens (87%). Patched, only the 469-token instruction is new.
Fix, measurements, verification and the full patch follow in the answer below.
All reactions