Max-tokens truncation is unrecoverable for large tool calls on small context windows - four proposals (rc.7, code-traced)

[scottconverse]

Running dsh 0.1.0-rc.7 as a daily cockpit against a local model (Qwen3.8-27B via LM Studio, 65,536-token context window) surfaced a failure mode that matters a lot on smaller context windows, and after tracing the shipped package code I'd like to propose four changes. Everything below is from reading the rc.7 lib/*.js sources, so file references are to the published packages.

The failure

When a reply hits the output-token budget mid-emission, the turn ends with reason: {kind:"max-tokens"} and the harness waits for a human "continue". For large single emissions — cordis_define plugin source is the sharpest case, since its schema requires the complete function body in one tool call — this combination makes the truncation unrecoverable:

1. dsh-llm's BlockAssembler.assembled() drops every tool-call block from a max-tokens reply (the comment says exactly why: they cannot be executed safely). So the partial emission is discarded, by design.
2. "continue" is an ordinary new user turn: dsh-agent-loop rebuilds the full message array and issues a brand-new completion. The protocol has ReplayEnvelope/replayState support for true resumption, but the pi-ai adapter never populates it.
3. Nothing clamps the request's max_tokens to remaining context room before dispatch — dsh-llm's resolveCallFor notes "no clamping or aliasing is performed" — so the serving stack cuts the stream wherever the window runs out, and the harness only classifies the result afterward.
4. Under the autonomous goal driver, a max-tokens turn/end disarms the drive loop (dsh-goal-round-driver), so unattended runs stall precisely on the failure they'd most need to survive.

Field data point: one afternoon of Creator-mode plugin authoring on the 65K local window produced seven truncations, each requiring the model to re-emit the entire plugin source into a fuller window than the attempt that failed. On big-window cloud models this is rare; on local 32–65K windows it's routine. (Full measured record: https://github.com/scottconverse/halo-stack/blob/master/docs/experiments/creator-mode-2026-08-18.md)

Four proposals, smallest first

1. Pre-flight budget check. Before dispatch, compute room = contextWindow - promptTokens (the token-meter's contextPressure projection already tracks this) and either clamp max_tokens to it or refuse the request with a clear "not enough room for the configured reply budget — compact or start fresh" error. Refusing before generating beats truncating after: the model's work isn't wasted and the operator gets an actionable signal.

2. Populate replayState in the pi-ai adapter. The resumption protocol already exists in dsh-llm; the adapter's finish path just never fills it. For providers that support completion continuation this would make "continue" resume the actual stream instead of re-generating from scratch.

3. Treat max-tokens like context overflow. dsh-compaction-basic already has an elegant reactive path for CONTEXT_WINDOW_EXCEEDED: emergency compaction + bounded retry. A max-tokens finish on a turn whose prompt was near the window is the same disease with a different symptom, and could route through the same recovery instead of disarming the goal driver.

4. An append/patch mode for cordis_define. The schema (additionalProperties: false, whole code.host/code.client strings) makes whole-source re-emission a hard requirement. A baseParcel + patch or chunked-append variant would let large plugins be authored incrementally — which is also how every other file-writing tool in the harness already works (create/insert/str_replace).

Happy to provide the full trace notes or reproduce any of this against rc.7. The stack this came from (configs, benches, the measured record above) is public at https://github.com/scottconverse/halo-stack.