Context-overflow recovery sends a larger request than the one that overflowed; images sent in encodings the endpoint cannot decode

[nnnet]

Two defects found while running dsh against a local llama.cpp route

Running the harness against a local LM Studio (llama.cpp) endpoint, 58 deduplicated turns in one workspace ended 41% in error. Most classes were my own misconfiguration, but two look like harness defects. Both have a fix prepared on a fork branch — CONTRIBUTING.md says external PRs are not accepted, so these are linked for reference rather than proposed.

1. Context-overflow recovery sends a request larger than the one that overflowed

compactIfNeeded selects the overflow region with retainTokens = 0, so summarizeWithLlm replays essentially the whole surface that just overflowed — behind the same system prompt and tool set — plus a 459-token compaction instruction, while reserving maxTokens (default 8192) for output. The recovery request is strictly larger than the request the provider just refused.

Priced in a regression test against a 4000-token window: the conversation request overflows at 4916 tokens, the recovery request costs 5617.

In the session logs this appeared as 28 of 74 compaction attempts (37.8%) ending with an error — 18 × summarization truncated at the token cap (incomplete checkpoint), 3 × summarization produced no text summary content, 2 × summary is not smaller than the shadowed content, 5 × provider errors leaking into compaction. One turn retried compaction ~28 times over 2.6 hours of wall clock.

A second, smaller defect gates the first. llama.cpp reports overflow as {"code":500,"message":"Context size has been exceeded.","type":"server_error"}. The structured classifier required length/window as the bound with no copula between bound and verb, so this wording fell through to the generic 5xx rule, was labelled SERVER, retried to exhaustion, and never reached overflow recovery at all. Fixing the classification is what makes the first defect reachable.

Branch: https://github.com/nnnet/deepseek-harness/tree/fix/context-size-overflow-classification — description at nnnet#1

2. Images are sent in encodings the routed endpoint cannot decode

ImageRequestPolicy carries only pixel and byte budgets, so attachment-local picks WebP for any transparent source and passes stored WebP through untouched, with no knowledge of what the routed endpoint decodes. LM Studio decodes via llama.cpp's stb_image, which has no WebP support, and reports the decode failure as 400 "'url' field must be a base64 encoded image" — wording that reads as a malformed field rather than an unsupported format, which sent the first investigation down the wrong path.

Observed: one 176×225 WebP attachment caused 7 consecutive failed turns over 67 minutes. Every later turn, including a bare continue and unrelated new user messages, resent the same history carrying that image and failed identically. It ended only when the provider was switched by hand.

Branch: https://github.com/nnnet/deepseek-harness/tree/fix/request-image-media-type-negotiation — description at nnnet#2

Not defects, for completeness

Three classes looked like harness bugs and were not, in case the wording is worth improving anyway:

• Model entries without an explicit contextWindow inherit DEFAULT_CONTEXT_WINDOW = 262144 while the local server's real per-slot n_ctx was 32768. Compaction's 0.8 × contextWindow threshold then never fires and the request dies on the server's hard error. The existing defaultContextWindow connection field fixes it, but the default is a sharp edge for local-server routes.
• OpenRouter gpt-oss-120b and the local Qwen3.8 Jinja template both reject reasoning effort off — 8 turns lost before the config dropped that level. The harness classified these as non-retryable and failed fast, which is correct.
• LM Studio JIT model-load failures, transport drops, and quota exhaustion were all handled correctly.

A related observation rather than a bug: nothing detects that N consecutive turns failed with byte-identical errors. That is what turned defect 2 into 67 wasted minutes, and separately a turn burned 98 steps repeating unknown tool "str_replace_editor" (17×) and missing required property "description" (~97×) without changing course. Loop hygiene felt like the natural home, but it is out of scope of both branches above.

[PixelWave-max]

Independent confirmation of both defects, on different hardware and a different model. Same two failure modes, same wall-clock cost.

Setup: dsh 0.1.1-rc.2 -> local relay -> LM Studio 0.4.21 (llama.cpp 2.29.1), Qwen3.8-27B-Uncensored Q4_K_M with mmproj, Mac Studio M1 Max 32 GB, ctx 32768, speculative draft Qwen3.5-0.8B.

Defect 2, WebP

Confirmed with a controlled A/B against the same server. Same image, two encodings:

PNG    -> 200, the model describes the image
WebP   -> 400 {"error":"'url' field must be a base64 encoded image."}

The base64 is valid in both cases. I captured the outgoing request with a proxy: messages[40].content[1].image_url.url carried a well-formed data:image/webp;base64, payload of 41811 characters. Confirming your point about the wording, I spent the first hour looking for a truncated or malformed field.

Cost here matched yours: read_image failed on every turn, and every later turn resent the same history and failed identically.

A workaround that does not require patching dsh. Anyone routing through a proxy can convert in flight. About thirty lines: walk messages[*].content[*].image_url.url, and for any data:image/webp;base64, decode, convert, re-encode as PNG. On macOS sips -s format png is already installed, cwebp or ImageMagick elsewhere. Verified end to end: the server log now shows "url": "data:image/png;base64,iVBORw0KGgo..." and a 1920x1080 screenshot is read correctly, down to the small text in the status bar.

One number worth knowing: a 31 KB WebP becomes a 667 KB PNG, so the request grows from ~42 K to ~889 K characters. Irrelevant on a local link, less so on a metered one. JPEG would be smaller but lossy, and screenshots are the main use of read_image.

Defect 1, compaction

Also reproduced, with one measurement that may narrow it. My server caps output at 4096 and compaction-basic maxTokens was 3584. The summarizer stopped naturally at 3637 generated tokens. It was 53 tokens short of finishing its checkpoint, so dsh declared it incomplete and retried, ten minutes per attempt, indefinitely.

Raising maxTokens to 6144 fixed it on the first attempt: compaction/summary then compaction/end with no error, Compacted 18 history items (~8350 tokens).

So beyond the recovery request being larger than the one that overflowed, there is no feedback when the summary is cut a few tokens from the end. The retry replays the identical request and produces the identical truncation. A single retry with a raised budget, or simply surfacing the shortfall, would break the loop. Users who cannot fork can at least raise maxTokens above what their model actually emits, and that number is worth measuring rather than guessing.

Happy to supply the full server logs or the relay code if either is useful.

[zzusp]

I reproduced a related but distinct non-converging compaction failure on @deepseek-ai/dsh 0.1.1-rc.2 with the openai-codex adapter and gpt-5.6-sol. The profile used the default compaction-basic policy (thresholdRatio: 0.8, retainRatio: 0.16, compactionRetries: 1).

Observed sequence

One automatic compaction succeeded. After that checkpoint had become the oldest visible surface node, later pressure checks repeatedly selected only that checkpoint:

shadowedTokenCount: 843-844
framed summary:     843-856
error: summary is not smaller than the shadowed content

The failed transaction correctly left the surface unchanged. However, every later agent/pre-step used the same retained-tail budget, selected the same single checkpoint again, and paid for another summarization call. I observed this across many consecutive turns with different compaction IDs but the same effective range and error.

This was not caused by a missing context-window declaration or by auto-compaction being disabled. The pressure listener was running and compaction/start / error compaction/end pairs were persisted each time.

Why it does not converge

selectCompactableRange() derives one head range from the fixed resolved retainTokens. If the retained tail and balanced-boundary adjustment leave only a small existing checkpoint in the compactable head, summarizeCompaction() can reasonably reject the replacement because the required structured checkpoint plus framing is not smaller.

That error exits compactIfNeeded() immediately. compactionRetries only applies after a successful replacement that still leaves total pressure above the threshold; it does not adapt after a non-shrinking summary. The next turn therefore repeats the same selection.

Suggested behavior

For the specific non-shrinking-summary failure:

1. classify it separately from provider, cancellation, surface-change, commit, and persistence failures;
2. retry range selection in the same pressure check with progressively smaller retention budgets, for example resolved retain tokens, half, then zero;
3. skip a retry when the balanced boundaries still produce the same start:end range;
4. stop retrying the same failed range until surface.replaceGeneration or the selected range changes;
5. preserve the existing invariant that a summary must be smaller than the content it replaces.

The important part is expanding the selected source range, not weakening the shrink check. A conceptual regression fixture would be:

existing compact checkpoint (~843 tokens)
+ sufficiently large recent tail retained by the normal policy
→ initial selection contains only the checkpoint
→ summarizer returns a valid but non-smaller framed checkpoint
→ policy must expand to a later balanced boundary or terminate without repeating the same range

I can prepare a minimal test or a source-based replacement package if that would help. I am adding this here because repository Issues are disabled and this discussion already records the same error string, although the selection/retention cause differs from the overflow-request-size cause in the original post.

[zzusp]

Update: I completed a source-based replacement and a real-session validation. The initial “expand the range after a non-shrinking summary” diagnosis was correct but incomplete for this particular session.

Additional root cause

The pathological session contained two tool calls from a historical step that already had step/end, but no corresponding tool results. The surface contained 60 calls and 58 results. Because the official balanced-boundary helper carries that deficit across all later boundaries, all three retention budgets still resolved to the same ~843-token checkpoint. Reducing retainTokens alone therefore could not expand the range.

Implemented solution

I published an MIT, source-based replacement derived from the official dsh-compaction-basic implementation:

• repository: https://github.com/zzusp/dsh-compaction-convergent
• latest validated release: https://github.com/zzusp/dsh-compaction-convergent/releases/tag/v0.1.1-rc.2-convergent.5
• convergence change: 修复 DSH 自动压缩重复小范围导致的不收敛循环 zzusp/dsh-compaction-convergent#1
• pathological-session repair: 支持恢复前分层压缩病理 Session zzusp/dsh-compaction-convergent#3

The replacement:

1. classifies only the “summary is not smaller” condition as a typed retryable error;
2. tries retainTokens, floor(retainTokens / 2), then 0;
3. skips duplicate balanced ranges, so the summarizer is not called twice for the same range;
4. caches a failed range by Session + replaceGeneration + range across turns;
5. for closed historical steps only, adds temporary synthetic error results to the summary input for orphan calls, without modifying the original log;
6. preserves the existing behavior for provider, abort, surface-change, commit, persistence, open-step, manual compaction, and overflow errors.

The fifth rule is deliberately narrow: an orphan call from a still-open step is not synthesized, and no tool/result event is appended to the source session.

Real-session result

Using Node 24 and the Web profile’s real LLM adapter, a gated repair of a copy produced:

before: 131,393 tokens, 880 surface nodes
after:   12,900 tokens,   1 surface node

The report passed semantic validation, replace generation advanced, the output persisted and reloaded at 12,900 tokens, and a subsequent message could be appended. The source session SHA-256 was unchanged. After installing the validated output, the Web UI showed about 8% context usage and the real model completed the next turn.

One deployment detail was also important: installing the package is insufficient. The profile must explicitly disable @deepseek-ai/dsh-compaction-basic and insert the replacement provider; dump-config must show exactly one enabled compaction provider.

This is a user-space workaround and reproduction, not a claim that upstream has fixed the issue. I hope the closed-step orphan-call fixture and the non-shrinking range retry policy are useful as upstream regression cases.