The request-pressure estimate is text.length / 4 — it ignores script density and JSON escaping, so compaction can trigger after the request is already too large

[Ragnoryok1]

Summary. The token estimate that drives the compaction threshold is a fixed CHARS_PER_TOKEN = 4 over the raw JavaScript string length, plus a small per-block overhead. It counts neither the character density of non-Latin scripts nor the escaping the wire form adds, so for a conversation dominated by Cyrillic the real token count — and the real request body — can be well above what the harness believes it is sending. Combined with a token-only compaction trigger, the threshold can be crossed only after the provider has started refusing the request.

Where

• packages/llm/token-meter/src/estimate.ts:13 — const CHARS_PER_TOKEN = 4
• :16 — BLOCK_OVERHEAD
• :48 — tokens += Math.ceil(block.text.length / CHARS_PER_TOKEN) + BLOCK_OVERHEAD (raw UTF-16 length, no escaping)
• :50-52 — the same constant for tool-call names and arguments

Evidence. A Russian-dominant session reached HTTP 413 while the harness's own accounting still reported headroom, and the compaction that then fired could not run because its summarization request was refused for the same reason (my earlier report on the missing byte budget — the two defects compound). I am deliberately not claiming a measured ratio: the ~2x figure for Cyrillic is an estimate from BPE density, while the code facts above (fixed constant, raw length, no escaping) are verified. A calibration from the maintainers' side, or a measured tokenizer comparison, would settle it.

Suggested directions

1. Make the estimate script-aware: a small character-class to tokens-per-character table, or at least a per-script factor.
2. Account for the escaping the serialized request adds (newlines, quotes, backslashes) for text that will be sent.
3. Expose estimate-vs-actual calibration so a route can be tuned without a code change.
4. Consider gating compaction on a byte estimate as well as a token estimate (the same direction as in my compaction report).

Question. Is the fixed constant a deliberate conservative bound for cheap prompt-cache decisions, and would a script-aware estimate break a documented invariant (cheapness, monotonicity, or stability across turns)?