Context Compaction Issue with llama.cpp

[cjdell]

Not sure where to share this but this has a happened a few times using my local LLM running Qwen 3.8. I can verify the compaction happened successfully by looking at the llama.cpp logs however I would still see Compaction could not produce a useful summary. The conversation is unchanged; the attempt is recorded in the session log..

{"type":"compaction/start","seq":108936,"time":1787078871014,"data":{"compactionId":"cb609cd3-10e1-4e19-bbd5-36dce71c98e9","sourceCommandId":"cmd-6a258e3c-5","turn":null}}
{"type":"compaction/end","seq":108937,"time":1787079314647,"data":{"compactionId":"cb609cd3-10e1-4e19-bbd5-36dce71c98e9","sourceCommandId":"cmd-6a258e3c-5","turn":null,"error":"summarization truncated at the token cap (incomplete checkpoint)"}}
{"type":"command/done","seq":108938,"time":1787079314656,"data":{"commandId":"cmd-6a258e3c-5","kind":"error","text":"Compaction could not produce a useful summary. The conversation is unchanged; the attempt is recorded in the session log."}}

Perhaps another issue. This was a manual compaction operation as I didn't ever see the harness attempt to do it automatically (it would just hit the context limit and give up).

[argszero]

Source-verified against master (99f6f02) — this is one root cause producing both of your observations, plus a llama.cpp-specific detail that explains the confusing logs.

1. The error chain (verified)

Your compaction/end error "summarization truncated at the token cap (incomplete checkpoint)" is produced by packages/compaction/compaction-basic/src/summarizer.ts:198-214:

case 'max-tokens': {
  const error = new Error('summarization truncated at the token cap (incomplete checkpoint)') as Error & { code?: string }
  error.code = 'MAX_TOKENS'
  return error
}

The summarization is a normal ctx.llm.stream() call with maxTokens: config.maxTokens (summarizer.ts:159), and the default is 8192 (compaction-basic/src/index.ts:91: maxTokens: config.maxTokens ?? 8192). The local model generated until the cap, the stream finished with max-tokens, and the compaction fails closed — a truncated checkpoint is missing the closing tag, so resuming from it would reconstruct a gutted session (that strictness is the corruption-family backstop, deliberate). command-compact then maps the summary error code to exactly the message you saw (command-compact/src/index.ts:39-45).

2. The llama.cpp logs are a different mechanism

What llama.cpp logs as "compaction" is server-side KV-cache compression / context shift — the server squeezes its cached state to fit the incoming prompt. That is not dsh's compaction. dsh's compaction is a durable checkpoint: an LLM call that writes a structured summary (the COMPACTION_INSTRUCTION at summarizer.ts:31-60 demands a 6-section Markdown checkpoint with verbatim quotes) into the session log, replacing the shadowed history. The two share a name and nothing else.

In your sequence, the llama.cpp logs almost certainly show the server compressing its KV cache to fit the summarization request itself — and here's the second mechanism worth knowing:

3. Why auto-compaction "never fires" (and it's the same root cause)

auto: true is the default (index.ts:95) and auto-compaction has two triggers (index.ts:129, 133-221): between-step pressure, and context-overflow recovery — when the model request overflows, dsh compacts and retries (maxOverflowRetries ?? 1). But with llama.cpp's default context shift, the server silently compresses instead of returning a context-length error — dsh never sees an overflow, so the context-overflow trigger never fires, and the session log just keeps growing. Then your manual /compact sends the entire shadowed region as the summarization input (region.ts:498-510 builds messages from every shadowed seq — no input trimming), the server compresses to fit, and the weak output budget (8192) caps the summary. One root cause, two symptoms.

4. What to change (config, not code)

1. Give summarization a bigger output budget: maxTokens: 16384 (or higher) in the compaction config. For an 8B-class local model producing a 6-section checkpoint with verbatim quotes, 8192 output tokens is genuinely small.
2. Point summarization at a stronger model: set summarizationProvider / summarizationModel explicitly (summarizer.ts:128-138 — the default target is the agent's own provider/model, i.e. your Qwen 3.8). A remote capable model for the checkpoint while the agent keeps the local model is the standard split and dramatically raises summary quality — which also lowers output length variance.
3. Decide how you want auto-compaction to engage: if you want dsh's context-overflow trigger to actually fire, disable llama.cpp's context shift for the harness endpoint (--no-context-shift) so overflow errors surface to dsh — then auto-compaction runs at the right moment instead of after the log has grown unboundedly. The trade-off is that without context shift, an oversized request errors instead of degrading — which is precisely the signal dsh's trigger is built on.
4. If a summary still fails: compactionRetries ?? 1 (index.ts:92) allows one retry; the fail-closed behavior is intentional and should stay — the fix is budget/model, not loosening the checkpoint contract.

5. A note on the README/UX gap

This is the second report where "compaction" ambiguity (server KV compression vs durable checkpoint) produced a confusing failure (the first being the general "could not produce a useful summary" phrasing). If the maintainers want a cheap win: the command-compact summary error could append the underlying error code (MAX_TOKENS) so users immediately know it's a budget problem, and the compaction docs could note the llama.cpp context-shift interaction. Happy to expand on either if useful.

[LeonidasTMT]

Getting the same problem even though from LM Studio it appears to be successful

{"type":"command/run","seq":126232,"time":1787151834532,"data":{"commandId":"cmd-171e9870-3","name":"compact","args":"","source":{"kind":"user"}}}
{"type":"compaction/start","seq":126233,"time":1787151834533,"data":{"compactionId":"5e079ec0-2ece-4ca8-a54a-e22a34b9b9d0","sourceCommandId":"cmd-171e9870-3","turn":null}}
{"type":"compaction/end","seq":126234,"time":1787152074976,"data":{"compactionId":"5e079ec0-2ece-4ca8-a54a-e22a34b9b9d0","sourceCommandId":"cmd-171e9870-3","turn":null,"error":"summarization truncated at the token cap (incomplete checkpoint)"}}
{"type":"command/done","seq":126235,"time":1787152074978,"data":{"commandId":"cmd-171e9870-3","kind":"error","text":"Compaction could not produce a useful summary. The conversation is unchanged; the attempt is recorded in the session log."}}

2026-08-19 23:07:54 [DEBUG]
19.04.219.327 I slot print_timing: id 2 | task 16759 | prompt eval time = 548.97 ms / 401 tokens ( 1.37 ms per token, 730.46 tokens per second)
19.04.219.333 I slot print_timing: id 2 | task 16759 | eval time = 238254.95 ms / 8192 tokens ( 29.08 ms per token, 34.38 tokens per second)
19.04.219.334 I slot print_timing: id 2 | task 16759 | total time = 238803.91 ms / 8593 tokens
19.04.219.338 I slot print_timing: id 2 | task 16759 | graphs reused = 24786
2026-08-19 23:07:54 [DEBUG]
19.04.220.386 I slot release: id 2 | task 16759 | stop processing: n_tokens = 37697, truncated = 0

[denial123789]

The LM Studio trace makes the boundary conclusive: eval ... 8192 tokens exactly matches compaction-basic’s default summary maxTokens. The server completed that generation, but Harness received max-tokens, converted it to MAX_TOKENS, and rejected the incomplete checkpoint before committing any replacement. truncated = 0 in the server log does not mean the Harness checkpoint contract was satisfied.

One source-level correction to the earlier recommendation: compactionRetries is used after a successful replacement when measured pressure is still above threshold. A first summary that throws because it reached maxTokens exits that attempt; increasing compactionRetries does not make the truncated checkpoint valid.

The bounded routes are:

1. inspect dsh --profile web --dump-config, then raise the existing compaction-basic row’s maxTokens incrementally (for example, 8192 → 16384) and measure one /compact attempt;
2. set summarizationProvider and summarizationModel together to a route with enough input context, output capacity, and instruction following;
3. decide whether the local server or Harness owns overflow recovery—server-side context shift can hide the canonical context-overflow error that triggers Harness recovery;
4. if the model repeatedly cannot produce a complete checkpoint, carry a bounded handoff into a fresh Session instead of looping /compact.

A successful durable attempt must show compaction/start → compaction/summary → compaction/end without error. I documented the exact rc.7 transaction, config, and acceptance gates here: https://sandbaseai.github.io/deepseek-harness-handbook/compaction-summary-truncated.html

[Yunado]

Same symptoms here, qwen3.8-27b via Llama.cpp, long sessions: manual /compact → "truncated at the token cap", auto-compaction never completing. I traced it to two distinct root causes in compaction-basic and wrote it up with a tested fix + full diff: #3465

[fan56]

The "compaction ran but the summary is useless" family is inherent to LLM summarization — a backend that never calls the model for compaction sidesteps it entirely. dsh-dcp swaps the summary step for deterministic extraction (verbatim paths/commands/errors/todos/user lines), inheriting the official trigger/retention/lock semantics. Zero model calls per compaction, stable output for identical input, CJK-aware token accounting.

dsh plugin add @aiwayds/dsh-dcp

Checkpoint format is compatible with the official compaction-basic one, so it is also a try-and-revert if you want to compare on your llama.cpp setup.

[mattafaak]

Corroborating the token-cap half from another Qwen3.8-27B / llama.cpp deployment,
and adding a taxonomy, because I found the same error to be a minority of my
compaction failures.

@Yunado's root cause 1 matches what I measured. On my box the two
truncated at the token cap cases were both on the thinking lane
(reasoning_effort=medium at the server), 266 s and 397 s — the output budget
being spent on reasoning before the checkpoint starts, exactly as described in
#3465. The non-thinking sibling of the same weights never hit it.

I mitigated it differently, in case the shape is useful: rather than raising
maxTokens, I route the summarization call only to the non-thinking lane, on
a plugin sitting on the llm/stream waterfall keyed on
options.purpose === 'compaction'. Same weights, no reasoning, whole budget on
the checkpoint. That costs two lane loads and needs a second lane, so it is not
a general fix — but it does confirm the diagnosis from the other direction:
remove the reasoning and the cap stops being reached.

The taxonomy, in case others land here with a different error. 74
compactions across 432 sessions, 25 with no summary:

n	compaction/end error	duration
15	summarization produced no text summary content	3–17 s
5	400 status code (no body)	0.1 s
2	summarization truncated at the token cap	266 s, 397 s
1	summary is not smaller than the shadowed content	—
1	pi-ai stream idle timeout after 300000ms	300 s

So the cap class was 2 of 25 here. The 15 are a different cause: the
summarization call carries the conversation's tool schemas with no tool choice,
and a tool-capable model answers the compaction instruction with another tool
call, which summaryText() discards. I wrote that up separately in #5521 rather
than crowd this thread.

Useful tell for anyone triaging their own logs: read the duration. A
successful summary on this lane takes ~131 s. 3–17 s is a tool call. 266–397 s
is the cap being reached the slow way. The error text alone does not separate
them.