Bug: compaction drops same-route reasoning effort and loses cache compatibility

[orziz]

Summary

In dsh-v0.1.0-rc.7 (99f6f02), BasicCompactionEngine reconstructs the conversation prefix for its summary call but drops the effective reasoningEffort from the latest durable request. The compaction request therefore falls back to the adapter default even when it uses the same provider/model as the request whose warm prefix it replays.

This makes the auxiliary request envelope differ from the conversation route and, for adapters that serialize reasoning effort, can put the otherwise matching prefix on a different server-side path. The DeepSeek adapter serializes this as thinking / reasoning_effort.

Source

summarizeWithLlm reads the latest header only to choose provider and model, then constructs GenerateOptions without reasoningEffort. The normal request header persists effective call configuration, and DSH documents model-owned reasoning effort as request-envelope state: LLM streaming documentation.

Minimal proposed change

Forward the latest effective effort only when the selected summarization target exactly matches the latest durable provider/model route:

const reasoningEffort = latest?.provider === target.provider && latest.model === target.model
  ? latest.reasoningEffort
  : undefined

const options: GenerateOptions = {
  provider: target.provider,
  model: target.model,
  ...reasoningEffort === undefined ? {} : { reasoningEffort },
  // existing replayed messages/system/tools and compaction maxTokens
}

The exact route check is important: an explicit summarization provider/model that differs from the conversation route must resolve its own adapter defaults rather than receive an opaque effort ID from the old route. The compaction-specific maxTokens cap remains unchanged.

Reproduction and validation

• Add a latest request/header with { provider, model, reasoningEffort: "high" }.
• Run same-route summarization and observe that the pre-fix ctx.llm.stream() options omit the effort.
• After the change, assert that same-route options carry "high"; assert that an explicit different summarization target has no inherited effort.
• pnpm exec vitest run packages/compaction/compaction-basic/tests: 122/122 passed.
• pnpm run typecheck: passed.
• The paired English/Chinese package documentation was checked with verify-translation-pairing.

No live provider cache probe was run, because cache hits remain provider-controlled and require billable credentials. This report establishes the local request-envelope defect; it does not claim that any provider must cache a request. DeepSeek documents context caching as requiring matching request prefix semantics: https://api-docs.deepseek.com/guides/kv_cache/.

Contribution channel

The repository CONTRIBUTING.md says external pull requests are not currently accepted, so I am posting the minimal, tested change here instead of opening a PR.

[gavenma]

Confirming this defect in a live deployment (dsh-compaction-basic@0.1.2-rc.1) and adding one consequence that makes the impact larger than cache compatibility alone: on the pi-ai OpenAI-compatible path, dropping reasoningEffort changes what maxTokens means for the checkpoint.

In @earendil-works/pi-ai (dist/api/openai-completions.js), the answer-room guard that caps reasoning is gated on exactly the field this call drops:

// "Reasoning and the answer share max_tokens here, so an uncapped reasoning
//  phase can consume the whole response and leave no answer and no tool call."
function resolveClampedThinkingBudget(model, options, params) {
  if (!options?.reasoningEffort || !model.reasoning) return undefined
  ...
}

and for the Qwen thinking dialect:

params.enable_thinking = !!options?.reasoningEffort

So on OpenAI-compatible routes the summarize call falls into one of two silent failure modes:

1. Shared pool (routes where the model default leaves thinking on): reasoning tokens consume BasicCompactionConfig.maxTokens, and the default 8192 cap truncates the checkpoint. Observed in production with local Qwen at xhigh effort: every checkpoint in the final stretch of a long run came back as an "incomplete checkpoint", losing context exactly when it matters most.
2. Thinking disabled (Qwen dialect, no effort on the request): the checkpoint is generated without reasoning at all — no truncation, but a silently degraded summary, because enable_thinking is false.

Native DeepSeek has no budget split at all — the adapter sends one shared max_tokens and resolveThinking() falls back to the deployment default effort. Identical BasicCompactionConfig therefore produces three different semantics depending on the route, and none of them is "summary text ≤ maxTokens with bounded reasoning".

The current workaround is blunt: raise the policy maxTokens to 32768 so the shared pool is large enough for CoT + summary. That stops the truncation but accepts oversized checkpoints and less context headroom per cycle.

Beyond the minimal same-route inheritance proposed above, two small extensions would be worth folding in while this is being touched:

1. Make the summarization reasoning effort explicitly configurable (a reasoningEffort field on the compaction policy, defaulting to the same-route inheritance), so operators can say "compact with low reasoning" regardless of the conversation's effort.
2. Record the effective effort on the compaction/summary event. That event already carries provider, model, maxTokens, and usage (which includes reasoningTokens on the DeepSeek route) — adding the effort would make cross-route divergence auditable instead of inferred.

Happy to provide the production compaction/summary receipts if useful.

[orziz]

Thanks for confirming this in production and for tracing the pi-ai behavior. This establishes that the missing reasoningEffort is not only a cache-compatibility issue: it changes compaction semantics and can either consume the shared max_tokens budget or silently disable thinking.

The redacted receipts would be useful. Could you share one example from the shared-pool truncation path and, if available, one from the Qwen thinking-disabled path, including provider/model, compaction maxTokens, requested/effective effort, usage/finish reason, and the incomplete-checkpoint marker?

I agree that same-route inheritance should remain the baseline fix. An explicit compaction reasoningEffort override and recording the effective effort on compaction/summary both look useful. For an explicitly different provider/model route, inheritance should remain disabled unless the compaction policy explicitly sets an effort for that target. Raising maxTokens to 32768 is a useful workaround, but it should not be treated as the fix because it changes checkpoint size and context headroom.

[gavenma]

Thanks @orziz — here is exactly what the deployment records, with the honest limit stated first. The archived compaction/summary events do not capture a reasoning/answer split, so I cannot produce a clean truncation receipt from history; the reason it is absent is itself the strongest concrete evidence for the observability change.

What is recorded (75 compaction/summary events across the session logs, all at maxTokens=8192):

provider distribution:
  local-qwen-fp8 / qwen3.8-27b-fp8      = 57
  openai         / gpt-5.6-sol          = 17
  deepseek-official / deepseek-v4-pro   = 1

Example event (local Qwen): provider: local-qwen-fp8, model: qwen3.8-27b-fp8, maxTokens: 8192, shadowedTokenCount: 161592, with a complete ~12.9K-char summary.

What is NOT recorded. Every event's usage is folded to the harness shape:

{ "inputTokens": 172458, "outputTokens": 6265 }

The event keys are compactionId, llmStreamCall, maxTokens, model, provider, rawOutput, shadowedRange, shadowedSeqs, shadowedTokenCount, summary, usage. There is:

• no reasoning_tokens / completion_tokens_details (so no reasoning-vs-answer split),
• no effective reasoningEffort field, and
• no finishReason for the compaction call.

Consequently the shared-pool-truncation and thinking-disabled receipts you asked for cannot be produced from the archive: the checkpoint summaries in these records are complete, and the reasoning usage that would discriminate the two paths is simply not captured. This is the exact gap the proposal targets.

To get the receipts, the compaction/summary event needs to record the effective reasoningEffort plus the raw provider usage (with completion_tokens_details.reasoning_tokens) and the compaction call's finish reason. With that in place, a controlled repro — policy maxTokens back to 8192, local Qwen at xhigh, one extended session — would emit a definitive example of each path rather than a code-path inference.

I can run exactly that controlled repro and paste the resulting event once the instrumentation (or a manual logging patch) is available. Happy to share the redacted raw JSON of any single existing event now if useful.

[orziz]

Thanks, this clarifies the evidence boundary. The archived events demonstrate the observability gap, but they do not independently establish either shared-pool truncation or thinking-disabled behavior, so there is no need to send an existing raw event.

A controlled reproduction after adding the instrumentation would be useful. It should not block the minimal same-route inheritance fix, since the dropped field and the route-specific adapter behavior can be tested independently. Recording the effective reasoningEffort, raw provider usage, and finish reason would be a valuable follow-up.