Compaction aims precisely: stale thinking, superseded reads, staged caps (#56)

[webmatze]

Closes #56. Two commits: the raw transcript log first, so it is already collecting evidence while the numbers below are in use, then the stages themselves.

Together with #55 this finishes #53.

The transcript record (15f66a8)

Compaction rewrites the message array in place and session.json holds only that working history, so shortening a session was the same as destroying the record of it. #55 made that loss small; the stages in this PR make it large.

Every message is appended to transcript.jsonl beside session.json before compaction can reach it — one JSON object per line, append-only, never read back on the normal path. It is also the only basis on which anyone can later check whether 80/50, the caps and the window were good numbers; without it, the next calibration is guesswork again.

The flush points are where messages can be lost: at the top of compact_history, since compaction is the only thing that removes them, and in an ensure on Agent#send, because the last turn's messages arrive after the final compaction check. A session recorded before the log existed is seeded once on its first resume and told that its next turn will compact hard. A log that cannot be written warns once and is then given up on.

Also there: build_agent takes the session rather than its pieces. Passing messages and context_ratio separately is how one call site came to carry a transcript without what had been learned about measuring it — a bug found reviewing #55.

The stages (43ec951)

Compaction reached its target with one blunt instrument: truncate oldest-first to 2 KB, then summarize. Now four stages run, free ones first, each stopping as soon as the target is met — so a compaction often reaches it without mangling any tool output at all.

Stage 0 — stale thinking. Bulky, worthless once its turn is over, free in fidelity terms. The current turn is preserved byte for byte, because Anthropic validates the signature on a thinking block (anthropic.cr:243-247). A message that is only thinking keeps it: an assistant message with no blocks serializes to content: null, which providers reject, and no amount of reclaim is worth that.

Stage 1 — superseded reads. Read the same file five times and only the newest is kept; the earlier ones become a note saying why. Only read_file, grep, glob take part. bash is deliberately excluded — running make test twice is not a duplicate, it is a before and an after, and collapsing the earlier one deletes exactly the comparison the model is making. Stubs keep their tool_call_id and is_error.

Stage 2 — staged caps. 300 stale results at the old 2 KB floor is ~150k tokens, more than the whole budget; at 512 bytes it is ~37k, and a result already carrying a marker collapses to one line instead of keeping its head forever. It recognises its own past work from the marker in the text — no side table of state, so the policy survives a session round-trip through disk. Candidates are taken largest first rather than oldest first, so reclaiming a given number of tokens mangles as few results as possible.

The recency window. Stages 1 and 2 leave the last three real turns alone, so compaction cannot truncate the file being edited out from under the model. It is given up only as a last resort: when there is no boundary to summarize at, one oversized cat inside the only turn there is gets cut anyway, because protecting the work in hand is worth less than a request the provider will accept at all.

Real turns. LLM::Message gains synthetic, set on the continuations the agent injects itself (agent.cr:220, agent.cr:284). Without it, three continuations inside one turn consume the whole window and it protects nothing. safe_cut_index uses it too — cutting just before a continuation strands it referring to a response that is no longer there. Matching on the text of the known constants would have worked until someone reworded one, and the failure would have been silent.

Incremental accounting. The estimator re-walked the whole history once per truncation candidate, quadratic in the number of tool results. Each message is now measured once and the total adjusted by the delta. Bytes rather than tokens, so the running total agrees exactly with what estimate_tokens says over the finished array.

Testing

854 examples, green. assert_tool_pairing on every new path, as before.

The examples are built so each one is about the stage it names: a small tail of quiet turns keeps the recency window from exceeding the target on its own, which would otherwise have made every case a story about the summarizer. Worth naming:

• Stage 0 reaches the target on thinking alone, with every tool result still at its original size.
• The current turn's thinking survives byte for byte, signature included.
• Duplicate read_file calls collapse; duplicate bash calls do not.
• Largest-first: one 200 KB result among a tail of small ones is the only thing mangled.
• A result carrying a marker collapses to a stub on the next pass.
• Three synthetic continuations do not push three real turns out of the window.
• safe_cut_index never lands on a synthetic boundary.
• 400 tool results compact without pathological work.
• The transcript log keeps every result at 20 KB while the agent's own copy comes back shortened.

Not in this change

Anchoring checkpoints to a stable message id instead of an absolute index — #57. Reading the raw transcript back (a smith transcript command, or restoring from it). Replacing the bytes // 4 heuristic with a real tokenizer.

🤖 Generated with Claude Code