session_fold: folds don't persist/compound across turn boundaries — context grows despite folding, no live-vs-folded feedback

[WZHKAWF]

Summary

session_fold collapses steps on the wire for the current turn, but the savings do not persist or compound across turn boundaries. Over a long session the model folds repeatedly, yet the per-request input token count climbs monotonically and prefix-cache reuse collapses — so cost grows even as the model is actively trying to shrink context. The model also gets no durable signal about what is already folded vs. what is still live, so it re-folds noise and can't see the live foldable set.

Impact

• Long sessions become progressively more expensive per turn despite aggressive folding.
• Prefix cache thrashes: cache_read collapses toward base-prompt-only while cache_write balloons, i.e. the wire is rebuilt near-scratch each turn.
• The model loses the plot: it cannot tell what remains foldable, so it repeats folds or folds already-folded history, wasting iterations.

Observed behavior (from a real long session, anonymized)

Aggregate shape of one ~25-turn session:

• ~27 session_fold calls were issued across the session.
• Per-request inherited input tokens (the first request of each turn — i.e. the history the turn starts from) climbed monotonically across the session (order-of-magnitude 30k → 110k+), including a large single step-jump (~57k → ~98k) at one turn boundary. Folds did not reduce this baseline.
• Prefix cache: in later turns cache_read collapsed to roughly the base-prompt size while cache_write grew to 80–95k — the conversation prefix was effectively re-sent/re-written rather than reused.
• Fold gists were tiny (low single-digit-k tokens total). The bulk of the growing context was unfolded live history persisting across turns, not the gists.
• No "saved / live" meter ever surfaced to the model in a durable, per-turn way, so the model had no feedback loop on what was foldable or already folded.

Suspected root cause

1. Folds are effectively turn-local (or DB-only) and don't compound. A fold applied in turn N does not keep the folded region collapsed when the wire is reassembled for turn N+1; prior turns' history is re-materialized in full at the next turn boundary. So the "saved" bytes come back.
2. Cross-turn history is not itself subject to folding. There is no mechanism that carries a fold decision forward so that already-collapsed regions stay collapsed as the session grows.
3. The prefix-cache boundary moves, so re-materialized history invalidates the cached prefix → cache_read drops, cache_write spikes.
4. No persistent fold-state feedback to the model. The model can't distinguish already-folded from live, so it can't target the live set and re-folds noise.

Expected behavior

• A fold should stay applied across turn boundaries: once a region is collapsed to its gist, subsequent turns should inherit the collapsed form, so per-request input stays flat (± new work) instead of climbing.
• Folds should compound: folding more should monotonically decrease live wire, and the effect should hold as the session grows.
• Collapsed prefixes should preserve prefix-cache reuse (stable cached prefix → high cache_read, low cache_write).
• The model should receive a durable per-turn signal of saved vs live (and which step-ranges are still foldable) so it can target the live set and avoid re-folding.

Repro (generic)

1. Start a long session; do multi-step work each turn so each turn accrues several iterations of tool output.
2. Every turn, session_fold the prior turn's completed steps into a one-line gist.
3. Track, per turn, the input tokens of the first request of the turn and the cache_read / cache_write split.

Expected: first-request input stays roughly flat across turns (gists only).
Actual: first-request input climbs monotonically; cache_read collapses toward base-prompt size and cache_write balloons — i.e. folds don't persist across turn boundaries.

Suggested fixes

• Persist fold state as part of the durable transcript so the assembled wire for every subsequent turn honors prior folds (collapse-then-inherit), rather than re-expanding history at each turn boundary.
• Make folds idempotent and compounding across turns; a broader/older fold should keep superseding finer ones on the wire permanently, not just within the turn it was issued.
• Keep the collapsed prefix stable for prefix caching so cache_read is preserved after a fold.
• Surface a persistent saved / live meter + a compact list of still-live foldable step-ranges to the model every turn, so it targets the live set and stops re-folding already-folded history.

No conversation contents included; figures are anonymized aggregate token/cache metrics only.