Compaction with Qwen3.8-27B (local model via llama.cpp): two root causes and a working fix in compaction-basic

[Yunado]

Running DSH against a local thinking model via llama.cpp (Qwen3.8-27B, ~150K-token sessions), I ran into two compaction failures in long sessions: manual /compact → "summarization truncated at the token cap (incomplete checkpoint)", and auto-compaction never completing. After digging into packages/compaction/compaction-basic, I traced both to distinct root causes in the package itself. The fixes are small and let me compact 150K-token sessions reliably — sharing in case they help.

Root cause 1: summarization inherits the conversation's reasoning effort

Why the token cap is where it breaks (maxTokens vs. context length): the summarization call is a plain completion — its input is the region to summarize (scales with session length), while its output is capped by a fixed maxTokens (default 8192) sized only to hold the structured checkpoint, independently of the model's context window. On a non-thinking client the checkpoint is short and fits with room to spare. On a thinking model, the output budget is shared between reasoning and checkpoint — and the longer the session, the more the model plans before writing — so past a certain session size the thinking exhausts maxTokens and the checkpoint is truncated mid-structure, no matter how much context headroom remains. Bigger context windows do not fix this; the fix is to stop thinking on this one call (optionally with maxTokens raised to ~16384 for extra headroom).

summarizeWithLlm() in src/summarizer.ts builds its completion request from the conversation's current reasoning settings. With a thinking model at a non-off effort, the model spends the fixed output budget (maxTokens, default 8192) thinking before it ever emits the structured checkpoint — so the response hits the token cap mid-thought and the run ends with "summarization truncated at the token cap (incomplete checkpoint)".

This also explains why auto-compaction "never engages" in practice: when the threshold finally fires, the same truncated summary fails, so compaction can never complete.

Fix: the summarization request should not think. Add reasoningEffort: ReasoningEffortId("off") to the request in summarizeWithLlm. (For llama.cpp with --reasoning-budget, this also stops the server-side budget from being spent on the summary call.)

Root cause 2: the replayed region is not reduced before summarization

The region sent for summarization still contains the model's own reasoning, images, and multi-hundred-KB tool results. On a slow local model, that large input idles out under the stream watchdog before the summary can start streaming, which fails the whole compaction.

Fix: add prepareMessagesForSummary, which runs on a copy of the region and applies three new config options (sane defaults, per-model policy supported, validated via zod):

option	default	effect
summarizeReasoningKeepTurns	5	keep reasoning content only in the last N turns, strip it from older turns
summarizeImages	"strip"	remove image blocks before summarizing
summarizeToolResultMaxChars	2000	truncate oversized tool results with a marker

Gotcha found while writing this: when stripping reasoning from a model-produced turn, you must also drop that turn's stored replay projection (the source field) — otherwise replay validation fails with INVALID_REPLAY_STATE. The projection exists to make the model's own reasoning replayable; it is inert once the reasoning text is gone, and leaving it in place is what trips validation.

Result

With both changes, compaction works reliably on the local model across the session sizes I've tested — 128K, ~150K, and 256K tokens (all well past the trigger threshold). Both manual /compact and auto-compaction complete: the summary fits inside the token cap, there is no watchdog idle-out, and the session continues normally afterwards.

Full diff (8 files in packages/compaction/compaction-basic, +501/−15, including tests):

• Branch fix/compaction-local-model on my fork: https://github.com/Yunado/deepseek-harness/tree/fix/compaction-local-model — diff vs upstream: 47f9438...Yunado:fix/compaction-local-model
• Raw diff file: https://gist.github.com/Yunado/583e53fff8dd217f76024eaf787364df — git apply compaction-fix.diff from the repo root

Applies cleanly on 0.1.0-rc.5 (commit 47f9438).

Not an upstream contributor — happy to rework the shape (single boolean instead of three options, different defaults, etc.) if maintainers prefer. The INVALID_REPLAY_STATE interaction in particular might warrant a doc note even if the option design changes.

[ain-soph]

This is certainly an important feature to support👍