Automatic compaction never triggers in image-heavy sessions — token-meter prices image blocks by JSON reference size, not visual tokens

[matrikslee]

Summary

In image-heavy sessions (agent calls a read-image tool repeatedly), the automatic
compaction never triggers even though real prompt usage far exceeds the configured
threshold, and manual /compact fails to shrink context. The session gets "stuck"
at a high context percentage until a new session is started.

Environment

• dsh 0.2.0-rc.2 (desktop), compaction-basic (thresholdRatio 0.5) + compaction-tool-result-pruner

Evidence (from session log analysis)

1. Real prompt usage (from assistant/message usage records): grew monotonically
   152k → 178k tokens against a 256k contextWindow (60-70% occupancy), with ZERO
   automatic compactions triggering after image reading became frequent.
   Earlier in the same session (text-only), compaction triggered correctly
   (e.g. 61k/72k-token ranges shadowed, usage dropped 123k → 14k).
2. All compactions observed in the image-heavy phase were manual /compact runs,
   and each only shadowed the 2-3k-token command message itself — the compaction
   range selector never selected the actual conversation body.
3. 78 read-image tool calls; per-call usage jumps of +8k to +37k input tokens
   confirmed images are billed at visual-token cost in real requests.

Root cause (source-level)

@deepseek-ai/dsh-token-meter estimateStructuralBlock() prices an image
content block by the JSON length of its reference (BLOCK_OVERHEAD + JSON length / 4),
i.e. a few dozen tokens. The code comment itself notes "request price is
route-owned rather than fixed" — the routed request bills images at visual-token
rates (often 1k-2k+ tokens per image).

Consequences:

• compaction-basic compactIfNeeded() compares meter.measure(session).totalTokens
  against thresholdTokens — with images underpriced, the estimate stays well
  below the threshold (estimated ~40k vs real 158k), so automatic compaction
  never fires.
• Manual /compact uses the same meter for range selection, so it also
  under-selects what to compact.

Suggestion

Give image blocks a route-calibrated price floor (e.g. max(structural estimate,
known visual-token estimate per image dimensions), or feed back the provider's
reported inputTokens into the meter so threshold comparison uses measured usage
instead of the heuristic.

Environment

• dsh 0.2.0-rc.2, Windows desktop client
• compaction-basic: thresholdRatio 0.5

[argszero]

Thanks for the source-level writeup — every link in the chain checks out against master, and the plugin-shaped half of your suggestion now exists.

The gap, confirmed

packages/llm/token-meter/src/estimate.ts prices a retained image by the JSON length of its own reference:

if (block.type === 'image') {
  const { offloaded: _offloaded, ...reference } = block
  return BLOCK_OVERHEAD + Math.ceil(JSON.stringify(reference).length / CHARS_PER_TOKEN)
}

The doc comment on that function already names the reason this is a placeholder rather than a price — "image references, whose request price is route-owned rather than fixed" — and the route-owned answer is resolved one call away, in packages/llm/token-meter/src/index.ts:

return this.ctx.get('llm')?.imageRequestPricing(config.provider, config.model)

The base declares none by design: packages/llm/llm/src/index.ts documents LlmAdapter.imageRequestPricing() as "The default declares none, so consumers fall back to their own neutral estimate", and the runtime delegates per route (this.adapters.get(provider)?.adapter.imageRequestPricing(provider, model)).

Why it bites the routes you were on

llm-deepseek implements the method. The pi-ai adapter — the one behind every OpenAI-compatible and Anthropic-compatible gateway route — does not: imageRequestPricing appears zero times in packages/llm/llm-pi-ai/src/index.ts at dsh-v0.1.7-rc.2, dsh-v0.2.0-rc.2, dsh-v0.2.1-alpha.2 and current master. It overrides providerInfo, providerRetryPolicy, listModels, resolveModel and prepareCall, and inherits the base undefined. Meanwhile its modelInfo() still advertises the model's real inputModalities, image included — so the request genuinely carries the pixels while the meter prices them as a string. That is the whole gap: not a wrong constant, a missing override.

Which of the two consumers is affected

Worth separating, because only one of them has a safety net:

• measure().totalTokens compares against the provider-anchored baseline, so on the steps whose request header matches the anchored one the real usage is what the gate sees.
• measure().nodes[].tokens are priced by priceSurface() and are never anchor-corrected. Compaction's range selection reads exactly those (compaction-basic/src/region.ts), so an image-heavy span priced at a few dozen tokens per image is not the span that gets summarized.

That second bullet is your evidence item 2 — "the range selector never selected the actual conversation body" — stated mechanically.

A plugin now fills it

@argszero/dsh-image-price-fallback@0.1.0 — npm: https://www.npmjs.com/package/@argszero/dsh-image-price-fallback, source: https://github.com/argszero/dsh-image-price-fallback

It wraps the llm service instance's imageRequestPricing — an own-property shadow on the same object every ctx.get('llm') view reads, so it is the exact seam the meter resolves through — calls the route's own implementation first, and answers only when the route answered undefined and resolveModelInfo() reports that the model accepts images. A route that declares its own pricing is never second-guessed; a model that cannot receive an image is never priced.

Mount it from a profile with the bundle patch the package ships:

- insert:
    - id: image-price-fallback
      name: '@argszero/dsh-image-price-fallback'

The estimate is ceil(width * height / 750), clamped to 85..1600, and every constant is a config knob: divisor (750 = pixels per visual token, the rate Anthropic publishes), minTokens (85 = OpenAI's low-detail base cost) and maxTokens (1600). Offloaded occurrences price 0 and take their placeholder text from the harness's own offloadedImageText(). A route whose modality has not resolved yet is left exactly as it is today, because modality resolution is async and imageRequestPricing must answer synchronously inside measure().

Honest boundaries

• This is an estimate, not a measurement. imageRequestPricing is resolved per measurement and must answer synchronously, so no tokenizer for an unknown gateway model can be called. The plugin prices from the durable reference's normalized width/height — the input every published vision accounting is built from. The point is to stop pricing a retained image at the length of its own reference; it is a floor in the right order of magnitude, and no more. Gateways that bill vision differently are what the three knobs are for.
• It does not do your second suggestion. Feeding the provider's reported inputTokens back into the meter is the more exact fix, and it is a core-side change — the plugin cannot make the meter trust a usage record it does not already trust.
• Tested at 43 unit tests plus a 23-arm defect-injection pass, and the published tarball resolves and installs by name (0.1.0, byte-identical to the packed artifact).

[matrikslee]

Verified on aml-llm gateway routes (GLM-5.3-Flash / GLM-5.3-Flash-NVFP4 via an OpenAI-compatible gateway).

The plugin works as described. Deployment: dsh plugin add (desktop profile, bundle mount), restart, and the meter resolves imageRequestPricing through the wrapper — pricing now lands at ceil(width*height/750) clamped 85..1600 on a route that previously answered undefined, and compaction range selection finally sees image-heavy spans at vision-order magnitude instead of JSON-reference length.

One distribution note from the install path (v0.1.0):

npmmirror (the default registry for much of China) serves latest as 0.0.0-stage — a staging placeholder with a corrupted tarball and no dsh manifest. Both the desktop UI installer and the dsh plugin add passthrough resolve latest through the configured registry first, so the UI path fails with "declares no dsh.bundle" (the stage placeholder has no dsh field; installBundle validates the manifest after a successful pnpm add, then rolls back) and the CLI path fails with ERR_PNPM_TARBALL_EXTRACT (invalid TAR checksum). Pinning @0.1.0 --registry=https://registry.npmjs.org works around both. May be worth a mirror-sync nudge so the stale dist-tag stops shadowing the real release, and a README note suggesting the CLI with an explicit registry for fresh releases.

Thanks for the fast turnaround on the source-level confirmation and for shipping the seam fix — the offloaded-occurrences-price-0 behavior (reusing offloadedImageText()) closes the exact hole our interim estimate.ts patch left open.