feat(agent): summarize the conversation when the context window fills - #87
Merged
Conversation
A session that outgrew its model context window had one escape: the model calling `new_context`, which discarded the conversation without summarizing it. Nothing acted on the token budget on its own, and a provider that refused an oversized history failed the turn outright. Compaction replaces a spent window with a summary of the work so far, keeping the past user messages within a token budget so the original asks survive. It runs before a turn that starts on a full window, after any tool round that crosses the budget, on request through `/compact`, and once as recovery when a provider refuses the history itself. The replacement history holds user messages only, so it cannot carry a `function_call_output` whose `function_call` was dropped, and it reuses the context epoch the reset tool already established.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A session that outgrew its model context window had exactly one escape: the model calling
new_context, which discarded the conversation without summarizing it. Nothing acted on the token budget on its own, and a provider that refused an oversized history failed the turn outright.This ports the local-compaction design from the Codex CLI (
codex-rs/core/src/compact.rs). Codex's remote compaction variants depend on OpenAI-only endpoints, so they are out of scope.What
ConversationCompactorreplaces a spent window with a summary of the work so far, keeping past user messages within a token budget so the original asks survive. It runs:/compact(session.compactRPC),The replacement history holds user messages only, so it cannot carry a
function_call_outputwhosefunction_callwas dropped — a shape both provider APIs reject. It reuses the context-epoch machinerynew_contextalready established, so the timeline still shows everything; only the model forgets.Every policy value lives in one
CompactionPolicyblock: the 0.9 trigger ratio, the 20k retention budget, and both prompts.Notable decisions
CompactionTargetinstead of aModelRequestprototype. The between-turns path would otherwise have to rebuild an agent's system prompt. The compactor now owns its own instructions, so the automatic and requested paths use exactly the same prompt.Verification
dart run melos verify— all 12 gates passdart run melos verify:debug— all 6 Linux E2E shards pass (real Debug Flutter runner + embedded daemon)context.compactionfeature contract.