compaction-basic: a session fed periodic user prompts grows its checkpoint every fold until the summary hits maxTokens (no length budget in COMPACTION_INSTRUCTION)

[avpanasyuk]

Version: @deepseek-ai/dsh-compaction-basic 0.2.0-rc.2. The same instruction is on master at 687f282 (packages/compaction/compaction-basic/src/summarizer.ts:32-67).
Model: deepseek/deepseek-v4.1-flash via OpenRouter. Window 262144, policy maxTokens 40960.

What happens

One long-running session receives a short automated user prompt every few minutes, a reminder listing due items, on top of its normal work. Each compaction's checkpoint ends up larger than the last, until the summary call hits maxTokens. From then on, a large share of compactions fail closed (finishError → MAX_TOKENS). The session keeps its full context and the pre-step tries again on the next step.

Measured over that session's 378 successful folds (outputTokens of compaction/summary):

	folds	median first third	median last third	max	least-squares slope
this session	378	9,726	34,059	40,860	88.5 tok/fold
same model, same host, other long sessions	146-203	3,363-4,143	4,688-4,946	7,948	7.8-8.9 tok/fold

• Successive checkpoints' sizes correlate at 0.97.
• 32 of the last 82 compactions failed, 29 of them at the 40960 cap. The other sessions on the same model had 0 failures in 105 compactions.
• Breaking the latest checkpoint down by section: ## Primary Request and Intent is 56,009 of 117,031 chars (48 %), up 29x in 12 days, against 6x for the whole checkpoint. Critical Context grew 9.6x and Pending Jobs 7.7x. The technical sections stayed roughly flat; the model does condense those.

Why (from the source)

COMPACTION_INSTRUCTION (rc.2 lib/index.js:243-278) has no length budget. Two of its rules pull against condensation for this kind of session:

• the first section asks for "the user's original and evolving goals; quote verbatim where the exact wording matters";
• the last rule says a prior <compacted-summary> must not be copied verbatim, but tells the model to "preserve still-true facts". A standing reminder is always still true, so every one of them is kept, fold after fold.

The prior checkpoint is an ordinary message in the next span (buildSummarizationInput, :695-704), so whatever the last summary kept is input to the next one, and there is no upper bound on it. The instruction is a module constant, so a deployment cannot add a budget through config. maxTokens only moves the point where the call fails. Raising it lets the checkpoint keep growing, and each fold then spends more of what it frees on carrying the previous checkpoint: here, about a quarter of the 83-126k tokens freed per fold.

Two smaller observations from the same failures:

• On a max-tokens failure, the default compaction/summary-error waterfall returns false (:1013-1022), so the same range is summarised again on the next step, with the same input and the same outcome. The README documents the recovery hook, but with nothing mounted there, the failure repeats until the turn ends.
• The compaction/summary event records usage, but not the size of the prior checkpoint that went in. Spotting this growth meant decompressing session stores and diffing successive summaries.

Suggestions

1. A length budget in the instruction: a configurable target, or a fixed one scaled to maxTokens, plus a rule that each section is bounded, not only "condensed".
2. The prior-checkpoint rule says what to do with standing or repeated user prompts: record their current effect once, never every occurrence.
3. Put the carried checkpoint's size, and the produced one's, in the compaction/summary event.
4. A default recovery for MAX_TOKENS, such as retrying over a smaller span or with a tighter budget, rather than repeating the identical request.

Workaround that works on rc.2

An llm/stream listener that acts only when options.purpose === "compaction". It mutates options.messages in place, since the waterfall's next() takes no arguments and adapterStream reads options at dispatch time. It:

• appends a per-section budget to the final instruction message, replacing that message because it is deep-frozen;
• bounds the carried checkpoint per section by whole items, so an existing backlog drains in one fold.

The checkpoint is identified by frameSummary's framing, not by the tag text, because a user can paste the tag. On a stub-backed home, a carried 25,466-char checkpoint went out at 1,632 chars, and every summary call carried the budget. It is not yet measured on the real session; I will add those numbers here.

---

Found and verified against the installed 0.2.0-rc.2 source and master 687f282 with the help of an AI coding agent.

[argszero]

Your source reading is right on every point I checked against master at 687f282e86, and two of your observations connect to a structural gap that was reported separately. Details below, with the line numbers I re-derived on that tree.

Confirmed: no length budget exists, and two rules push the other way

packages/compaction/compaction-basic/src/summarizer.ts:

• :32-67 COMPACTION_INSTRUCTION is a module-local const, joined into one string, delivered as the final user message (:149). It is not exported and not reachable from config, so "a deployment cannot add a budget" is exact.
• :43 "quote verbatim where the exact wording matters" — the pull toward retention in ## Primary Request and Intent.
• :66 the prior-checkpoint rule: "preserve still-true facts, drop stale ones, and merge newer information into a single consolidated summary under the same structure". Your reading is the correct one: a standing reminder is always still true, so nothing in the instruction ever authorises dropping it, and the only bound is the model's own judgement.

And the prior checkpoint is not special-cased anywhere in the next span — it is an ordinary message, exactly as you traced.

So the growth is not a model quirk on this session; it is what the instruction asks for. Two sessions on the same model diverge only because one of them has something that repeats.

Your observation 1 — the repeat is structural, and it is the second report of this gap

You are right that a MAX_TOKENS failure re-summarises the same range on the next step. The mechanism:

• index.ts:442-447 the default recovery is () => false; with nothing mounted on compaction/summary-error, a failed summary has no recovery at all.
• packages/compaction/compaction-basic/README.md:123 documents the contract that makes that deliberate: "A recovery listener must record a durable input change before requesting retry. The backend re-derives the selected messages and refreshes their token prices and shrink baseline." A listener that returns true without recording anything would produce the identical request, so the contract forbids an in-step spin.
• index.ts:158-176 the pressure trigger is agent/pre-step; its in-step budget is the compactionRetries loop at :330-341; and the failure lands in the catch at :166-173, which logs step compaction failed: …; continuing the turn and falls through to next().

The consequence is what you measured: the retry does not spin inside one step, it relocates to the next step, and nothing counts it there. The only attempt accounting in the module is overflowRetries (:134), which belongs to the overflow path (agent/request-error, :199, :217, :232) and is cleared on idle (:179) and on a successful assistant/message (:187). The pressure path has no analogue — no counter, no backoff, no give-up state. It retries once per step for as long as the turn lasts, with byte-identical input.

This is the second independent report of that same absence: #9888 covered the case where a compaction succeeds but does not fit, so the loop ran inside one step; your report covers the case where the summary fails at the cap, so it runs across steps. A cross-step attempt bound on the pressure path would address both, and it is the piece that is missing rather than the recovery hook.

Your suggestion 4 has one hard constraint worth naming

"A default recovery for MAX_TOKENS, retrying over a smaller span" is the right instinct, but the recovery contract blocks the obvious implementation. The backend re-derives the selection from sourceEventSeqs and rejects cancellation and unrelated selection changes (compaction/README.md:84, compaction-basic/README.md:123). A different range chosen inside a listener is a selection change, so it is rejected rather than honoured. A narrower span therefore has to be expressed as a durable input change the backend can re-derive — the shape compaction-image-offload uses, where each retry records an omission and the backend re-prices the region. A default for the MAX_TOKENS case would have to take that form, not the form of a range swap.

Your suggestion 3 is cheap and I would put it first

Agreed, and the event is the natural place: the carried checkpoint's size is computable exactly where the input is built, so it can ride on compaction/summary next to the usage you are already reading. Right now the only way to see this growth is what you did — decompress the stores and diff successive summaries — which is why a 12-day, 29x section-level drift went unnoticed until the call started failing. Recording both sizes (carried in, produced out) would have made it a single query.

On the workaround

The llm/stream seam with a purpose === 'compaction' predicate is the right one, and the two constraints you hit are structural rather than incidental: the waterfall's next() takes no arguments, and adapterStream reads options at dispatch time, so in-place mutation of options.messages is the only lever. Replacing the deep-frozen instruction message instead of mutating it is correct.

Identifying the carried checkpoint by frameSummary's framing rather than by the tag text is the right call, and worth stating in the discussion because the tags are plain string constants (summarizer.ts:22-23, <compacted-summary> / </compacted-summary>) that any user can type into a message. Matching on the tag alone would bound a user who pasted it.

If you do add the real-session numbers, the ratio worth reporting alongside them is the one you already implied: what fraction of the freed tokens each fold spends carrying the previous checkpoint. That is the number that says whether a budget is the fix or whether the carry itself needs to shrink.

[avpanasyuk]

Thanks. I checked your points against the 0.2.0-rc.2 package and they hold:

• the pressure path's failure lands in the catch that logs step compaction failed: …; continuing the turn, and the only attempt counter in the module is overflowRetries;
• the default compaction/summary-error recovery is () => false;
• compaction-basic/README.md:123 requires a durable input change before a retry.

So a cap failure repeats once per step with identical input, and I agree a cross-step attempt bound on the pressure path is the missing piece, for this case and for #9888.

Suggestion 4, corrected. You are right that a narrower span cannot be a range chosen inside a listener; the backend would reject it as a selection change. It would have to be recorded as an omission the backend re-derives, the way compaction-image-offload does it. I withdraw the range-swap form. Suggestion 3 (carried-in and produced sizes on compaction/summary) is the cheap one and should come first.

Early numbers from the workaround on the affected session. These are character counts of the carried checkpoint, from the plugin's own log; five folds is a small sample.

fold	carried in	after the trim
1	92,459	41,075
2	43,460	36,322
3	31,622	30,842
4	35,491	34,698
5	36,433	35,251

• The "carried in" figure from fold 2 on is what the model produced at the previous fold.
• A length budget in the instruction alone did not hold the model to it. Asked for about 23,500 chars, it wrote 43,460 at the first fold, then settled at 31-36k. The snowball stopped, at about 1.5x the budget.
• Two trim lessons that cost content. A per-section item-count cap cut by age: the newest items went first. A char trim that stops at the first item that does not fit let one large item take every newer item after it: 5,148 chars dropped where 1,083 were over. The trim is now char-only, skips an item that does not fit, and marks any item it keeps only in part.
• We have just added a line to the instruction that names each section still over its target, with its size. I'll report whether that moves the produced size after about ten folds.
• With that batch I'll include the ratio you suggested: the carried checkpoint's tokens over the tokens each fold frees.

---

Checked against the installed 0.2.0-rc.2 source with the help of an AI coding agent.

[avpanasyuk]

The numbers I promised, from the same session. These are metadata only: sizes from the plugin's log and the server's fold records.

Two arms after the first five folds.

arm	folds	carried checkpoint (chars)	"Critical Context" section (chars)	summary output tokens	cap hits
budget + per-section line, old trim	4	21,491-25,874	8,255-10,186	6,553-8,422	0
same + skip-and-fill trim	34	10,956-26,788 (median 14,895)	1,801-12,645 (last 5: 2,024-2,895)	3,339-8,501 (median 4,646)	0

• The per-section line alone did not move the largest section. In the first arm, Critical Context stayed at 8-10k against a 6,000 target, though the line named it and its size every fold.
• In the second arm it fell once the trim had actually cut it. The trim dropped 666 and then 3,658 chars of that section at folds 1 and 3. At fold 4 the model wrote 5.3k, and from fold 6 on it stayed at 1.8-4.5k. I read that as the model taking the trimmed section as the new norm, with the line steering it. It is one session, and the two effects are confounded.
• It now undershoots. The checkpoint is about 60% of the ~23.5k budget and that section about half its target. I have not yet measured whether recall paid for that.

Carry fraction (carried-checkpoint tokens over the tokens each fold frees):

• Every fold triggered at 190-204k and left 59-67k, freeing a median of 134k tokens.
• With the checkpoint at a steady ~4.6k tokens, the fraction is ~3.4% (2.4-4.2% from fold 6 on).
• Before the workaround, the snowballed checkpoint reached ~38k tokens, about 28% of what a fold frees.

For the fix upstream, this argues for the order in my first post: a length budget in COMPACTION_INSTRUCTION, plus carried-in and produced sizes on compaction/summary so the budget can be checked. A budget alone was ignored at ~1.5x. A budget the model can see enforced took hold within three folds.

---

Measured and verified against the installed 0.2.0-rc.2 source with the help of an AI coding agent.