Guides Context Compression Explained

How Minnow's context compression works

A practical guide for implementing the same design in another AI application.

Based on Minnow's source code as inspected on October 3, 2026. Paths at the end point to the implementation in this repository; the explanation is self-contained so you can share this file on its own.

The main idea

Minnow saves the full conversation, but sends the model a smaller projection of that conversation when context gets crowded. A projection is a temporary view built from the original messages and a saved compression checkpoint.

The default policy is called compact. It combines three techniques:

1. Replace large, older tool outputs with short references.
2. Fold older conversation sections into a structured summary.
3. Let the model retrieve missing details through recall_history.

The summary is generated by ordinary code that extracts information from messages and tool results. It does not make an additional LLM call. This makes compression predictable and avoids paying for a separate summarization request.

The original transcript remains available. Compression loses detail in the model's immediate view, rather than deleting that detail from storage.

1. Work out the available space

Before a model round, the runner estimates the size of the messages and reserves space for the tool schemas. By default, it uses the model's full context window. An optional workingContextTokens setting can impose a smaller ceiling.

Conceptually:

message ceiling = minimum of the available limits:
  model context window minus tool-schema tokens
  optional working-context cap minus tool-schema tokens
  optional calibrated or overflow-retry message ceiling

System messages count toward the message ceiling. Tool schemas are reserved separately so they are not accidentally treated as free space. If the model window is unknown and no other ceiling is available, automatic budget trimming is skipped.

Minnow uses estimates rather than a tokenizer for every model:

Content	Basic estimate
Prose	Characters divided by 3.6
Tool payloads and tool-call arguments	Characters divided by 3
Tool schemas	Characters divided by 4

Images receive their own estimates. Reasoning fields are counted according to whether they will be replayed in the current tool loop. These estimates are useful for deciding when to compact, but do not guarantee an exact provider token count.

One implementation detail matters if you copy this: Minnow does not subtract the configured maximum reply length from its ordinary message ceiling. For its local llama.cpp and MLX providers, it separately adjusts requested generation tokens against remaining space. Another project's provider may require a different input/output reservation strategy.

2. Trigger at 80%, aim for 50%

The default settings are:

Setting	Default	Purpose
highWater	0.8	Start automatic compression above 80% of the message ceiling
lowWater	0.5	Aim to reduce the request to 50% of the ceiling
minRecentTurns	2	Initially protect the two most recent turns from whole-turn folding
keepToolRounds	4	Protect tool results in the last four segmented rounds from elision
Summary budget	12% of the model window, bounded to 400–6,000 tokens	Bound the generated summary by default

An explicit summary budget can override that default; it is normalized to at least 200 tokens. The formatter ultimately enforces a character budget derived from the requested token budget.

The gap between the trigger and target is hysteresis. Compressing from 81% to 79% would force another compression almost immediately. Aiming for 50% creates room for more work.

For example, a 32,000-token window with 2,000 tokens of tool schemas gives a 30,000-token message ceiling. Compression starts above an estimated 24,000 message tokens and aims for about 15,000. That target includes the system prompt, summary, and remaining messages.

The target is best effort. Protected content can prevent the request from reaching 50%.

3. Compress in stages

Stage A: Shorten older tool results

First, Minnow replaces eligible tool results outside the last four segmented rounds with small stubs. String outputs shorter than 1,200 characters remain unchanged because a stub would save little.

A stub contains the original row number, tool name when available, output size, a short first-line preview, and instructions for recalling the original. For read_file, it can also include a small historical source excerpt.

An illustrative stub looks like:

[tool result elided from context — #412, read_file, 18.4k chars]
...short preview...
Call recall_history with rows "412" for the full text.

The tool message and its tool_call_id still exist in the outgoing request; its large body is what changes. If this saves enough space, Minnow does not need to fold more conversation into the summary.

Stage B: Fold complete older turns

A turn starts with a real user message and includes the following assistant and tool activity until the next user message.

Minnow considers older turns first, initially leaving the configured recent turns outside whole-turn folding. It extracts their useful information into summary state and omits their original messages from the outgoing view.

Stage C: Fold older rounds within recent turns

One user request can create a very long tool loop. Keeping recent turns alone therefore cannot bound context.

A tool round groups an assistant tool-call message with its consecutive tool results and any screenshot follow-ups. The segmenter also treats standalone messages as rounds. Minnow cuts at these boundaries rather than splitting a tool exchange.

If whole-turn folding is insufficient, automatic compression can fold earlier rounds within the recent turns. It preserves the current turn's final round and keeps the user request for a continued turn in the projection. Thus, “keep two recent turns” is an initial protection, not a promise that every byte of those turns remains forever.

The outgoing messages are also sanitized to remove invalid tool pairing.

Stage D: Enforce the hard ceiling

If the projection still exceeds the actual message ceiling, Minnow first removes the summary from the outgoing view, restoring any user message it had been merged into. If necessary, it then applies longest-row truncation.

That emergency truncation is not persisted into the original history or saved summary. The runner checks whether the result fits; content that cannot fit must not be sent blindly.

4. Build a structured, incremental summary

Minnow extracts a bounded state object rather than repeatedly asking a model to rewrite an earlier summary.

It records:

• The initial session goal and detected later scope changes.
• Standing user preferences and explicit compression notes.
• Selected assistant findings, labeled as claims to revalidate.
• File operations and short historical source observations.
• Commits, sub-agent outcomes, and brief earlier-turn descriptions.
• Tool failures and later resolutions.
• The latest todo list and current activity status.

This extraction is tailored to a coding assistant. For example, it recognizes file-editing tools, parses apply_patch, reads commit output, detects command failures, and recognizes phrases such as “actually” or “change of plan.” Selected assistant lines starting with labels such as Finding: or Blocked: become working findings.

Different categories use different merge rules: the initial goal is sticky, file operations accumulate, and current status takes the newest value. Collections have caps so summary state cannot grow forever. Examples include 80 file entries, 40 turn descriptions, and 12 problems. Read-only file entries are evicted before changed files when the file cap is exceeded.

Only newly folded rows are ingested into the previous state. This avoids repeatedly extracting the same history and counting its edits again. It does not mean the entire compression pass is constant-time: estimating, segmentation, and projection still examine messages.

The formatter emits sections under a Prior context header. When the summary is too large, it removes whole lines in a defined order, first trying to retain one line per section, then allowing empty sections. A final character cut handles anything still over budget.

This is heuristic extraction, not complete semantic understanding. It can miss a decision expressed in unusual language, shorten a crucial detail, or retain a stale claim. References to original rows are essential for recovering evidence.

5. Save a checkpoint, keep the transcript

A live checkpoint has this general shape:

type CompactionCheckpoint = {
  version: 1;
  foldThroughRow: number | null;
  elideThroughRow: number | null;
  summary: string;
  state: CompactionState;
  trigger: 'auto' | 'manual' | 'overflow';
  tokensBefore: number;
  tokensAfter: number;
};

foldThroughRow identifies the history boundary represented by the summary. elideThroughRow identifies the boundary through which eligible retained tool bodies become stubs.

For chats, Minnow persists this as a context notice row alongside the original history. The stored boundary names are foldThroughIndex and elideThroughIndex; the normalizer maps them to the live field names. Row identifiers are transcript indices in this implementation, so those indices must stay stable.

On the next send, projection reconstructs approximately:

Leading system messages
Prior-context summary
Retained user / assistant / tool messages

When the first retained message is a real user message, the summary is prepended to that message with an [End of prior context] separator. Otherwise, it is inserted as a synthetic user message. This supports model templates that require alternating roles and keeps the summary out of the system role.

The same source rows and checkpoint produce the same projection bytes. Between checkpoints, avoiding unnecessary summary rewrites keeps the prefix stable, which can help provider prompt caching. Actual cache reuse depends on the provider and the rest of the request.

On resume, an automatic checkpoint can be reopened if the full original history fits below the current high-water threshold, such as after switching to a larger context window. Manual and provider-overflow checkpoints remain in effect.

6. Retrieve omitted details

The model can call recall_history against the original transcript:

{ "query": "authentication failing test", "page": 1 }

Or retrieve a cited row range:

{ "rows": "412-416", "include_tool_results": true, "page": 1 }

Local search uses BM25 keyword ranking and groups hits by turn. It can combine an external ranking, such as SQLite full-text search, through reciprocal rank fusion. This path does not require embeddings or an LLM.

Retrieval is bounded: search returns five hits per page, row reading returns twenty rows per page, and message bodies have character limits. Tool results default to short previews; requesting full tool results increases the allowance to 12,000 characters per result. Even “full” recall is therefore bounded and can truncate oversized content. It does not guarantee delivery of every byte in one response.

For a friend's project, this means the transcript is both the source of truth and a searchable backing store. A short summary tells the model where it left off; recall supplies details when needed.

7. Manual compression and provider overflow

Users can run /compact, with /compress and /summarize as aliases. Optional text is kept as a summary note:

/compact Keep the authentication decisions and unfinished tests.

That text is saved as a note; it does not instruct an LLM to generate a custom summary. Manual compaction folds available older whole turns while preserving the configured recent turns. It can also elide eligible older tool results, but does not use automatic round folding within those retained turns.

Provider overflow is handled separately. The runner recognizes context-length errors and, where possible, reads the provider's reported request and window token counts. It records estimation bias, narrows the known window when appropriate, and forces compression against a smaller retry ceiling using a 0.9 safety margin.

When those counts are unavailable, the retry ceiling falls back to 81% of the sent estimate (0.9 × 0.9). Retries are bounded and only proceed when the revised messages fit the ceiling and are smaller than the failed request.

How to adapt this to another project

Implement the design as five separate pieces:

1. Transcript storage: append original messages with stable IDs, including tool calls and results.
2. Budget calculation: estimate the actual outgoing request, reserve tool schemas, and resolve the model's usable limits.
3. Compactor: choose valid boundaries, ingest newly folded rows into structured state, and format a bounded summary.
4. Projection: produce API messages from originals plus the checkpoint without mutating storage.
5. Recall tool: search and read original rows with bounded responses and pagination.

This pseudocode shows the integration pattern, rather than a drop-in Minnow API:

async function prepareModelRequest(session, model, tools) {
  const originals = await transcript.load(session.id);
  let checkpoint = await checkpoints.latest(session.id);
  const limit = resolveMessageCeiling(model, tools, session.policy);

  // Optional: reopen automatic checkpoints when all originals fit comfortably.
  checkpoint = maybeReopenAutomaticCheckpoint(originals, checkpoint, limit);

  let messages = project(originals, checkpoint);

  if (limit !== null && estimate(messages) > limit * 0.8) {
    const result = compact({
      originals,
      messages,
      previous: checkpoint,
      limit,
      target: limit * 0.5,
      trigger: 'auto',
    });

    if (result.checkpointChanged) {
      await checkpoints.append(session.id, result.checkpoint);
    }
    messages = result.messages;
  }

  messages = sanitizeToolPairing(messages);
  if (limit !== null && estimate(messages) > limit) {
    throw new Error('Context cannot fit safely');
  }
  return messages;
}

Run preparation before each model round, including rounds following tool results. Compressing only when a user sends a new message leaves long tool loops unbounded. Persist original tool outputs before replacing their outgoing bodies with stubs.

Customize the extraction rules to your domain. A support assistant might track customer constraints, unresolved questions, and case actions instead of commits and file changes. Keep historical evidence and assistant claims clearly labeled, and validate important state against current data after changes.

Before relying on the implementation, test these behaviors:

• Compression never overwrites original transcript bodies.
• Reusing a checkpoint reproduces the same outgoing projection.
• Repeated compression ingests only newly folded rows.
• Cuts preserve tool call/result pairing and the active user request.
• Large tool loops can shrink even within a single user turn.
• Saved checkpoints survive reload, and recall retrieves their cited rows.
• Unknown model windows, tiny windows, oversized system prompts, and overflow retries terminate predictably.
• Manual notes, changed context caps, and switching model windows behave as intended.

Where the implementation lives

Paths are relative to the Minnow repository root.

File	Responsibility
server/runner/turn-runner.js	Per-round enforcement, checkpoint projection, resume, and overflow retry
server/runner/context-budget.js	Ceilings, policy selection, estimates, tool pairing, and truncation
server/runner/token-estimate-core.js	Text, schema, and image estimation helpers
server/runner/compaction/index.js	Defaults, checkpoint normalization, and staged compression
server/runner/compaction/segment.js	Turn and round boundaries
server/runner/compaction/elide.js	Tool-result reference stubs
server/runner/compaction/extract.js	Domain-specific information extraction
server/runner/compaction/merge.js	Structured state, caps, and merge rules
server/runner/compaction/format.js	Summary sections and budget enforcement
server/runner/compaction/project.js	Deterministic outgoing view
server/runner/compaction/recall.js	Search, row retrieval, pagination, and ranking fusion
src/chat/context/compact-command.ts	Manual chat compression commands
src/chat/context/context-notice.ts	Persisted chat checkpoint notices
test/runner/compaction.test.mjs	Core compaction tests
test/runner/compaction-turn.test.mjs	Turn-runner integration tests

These modules have supporting dependencies; copying just the compaction folder will not produce a standalone library. The reusable design is retain originals, build a bounded working view, and make omitted evidence retrievable.