compaction-basic: add a prompt-message budget — long sessions die on gateways that cap message count

[layen-king]

Summary

A long-lived session became permanently unsendable: every model call returned HTTP 400 from the provider, and nothing in DSH noticed. The token meter reported pressure below the compaction threshold, so compaction-basic never ran, and each retry rebuilt the same oversized prompt.

The root cause is that some OpenAI-compatible gateways cap the prompt message count per request independently of the model's context window. compaction-basic only reacts to token pressure, so a tool-heavy session can reach that cap while still "fitting" the declared window.

Observed failure

Session: 180 goal rounds, 2057 projected prompt messages, ~790k tokens, model declaring contextWindow: 1000000. From one step onwards every request failed:

400: {"message":"{\"message\":\"the configured rewrite value still does not satisfy the rule trace_id: <redacted>\",\"type\":\"invalid_request_error\"}\n","type":"invalid_request_error"}

The message is opaque (it comes from the gateway's internal request-rewrite layer), and it never clears on retry, because the history is already over the cap.

Root cause (bisected against the live endpoint)

projected messages	result
1022 (first half of the failing history)	200
2047	200
2048	400, same "rewrite value" message
5001	400, a single request cannot evaluate more than 8192 path candidates

The cap is message count, not bytes or content:

• replaying the captured failing body with every message truncated to 40 chars → still 400
• replaying it with all reasoning_content stripped → still 400
• keeping the same 2057-message structure and only shortening contents → still 400
• cutting the history to its first 1022 messages (byte-identical contents) → 200

DSH side: resolveCompactSpec derives pressure purely from tokens — min(contextWindow * thresholdRatio, contextWindow − reservedCompletionTokens − headroomTokens). With thresholdRatio: 0.8, headroomTokens: 65536, a 32k output reservation and a 1M window that is 800,000 tokens, while 2048 messages of a tool-heavy session sat at ~790k. Compaction therefore never triggered, and the session could never be recovered.

Suggested fix

Add a second, independent pressure trigger — a prompt-message budget — rather than relying on token scaling to approximate one:

1. CompactionPolicyConfig.maxMessages?: number (default e.g. 1600; 0 disables), validated as a non-negative integer, inherited by exact-target modelPolicies like the other policy fields.
2. Carry it through ResolvedPolicyFields → resolveTargetPolicy → resolveCompactSpec.
3. In compactIfNeeded (the pressure branch), treat measurement.nodes.length >= spec.maxMessages as pressure, and re-check the same predicate after the pruner pass and after each compaction attempt.

TokenMeasurement.nodes already gives the projected message count, so no new service or projection is needed. A per-route override (modelPolicies) is worth exposing too, since gateways may cap lower than 2048.

Sketch of the predicate change:

const overBudget = (m: { totalTokens: number; nodes: readonly unknown[] }): boolean => (
  m.totalTokens >= spec.thresholdTokens
  || (spec.maxMessages > 0 && m.nodes.length >= spec.maxMessages)
)

With that patch the same session compacts at 1600 messages and continues (verified locally: the failing request goes 400 → 200 after compaction).

Environment

• dsh 0.2.1-rc.2, web profile
• Linux/WSL2 host and the packaged Windows desktop build (reproduced on both)
• provider: an OpenAI-compatible gateway (api: openai-completions), single model route, reasoning_effort in use

Workaround applied meanwhile

Lowering the declared contextWindow for that provider's models (1M → 500k) so token pressure lands before the message cap — it works, but it is a guess at the gateway's real limit and costs usable context, which is why a first-class message budget seems preferable.