.NET Compaction - Introducing compaction strategies and pipeline

[crickman]

Motivation and Context

Context management (compaction) is one of the key features in the "dev-harness" effort. This change introduces structured handling of long-running AI chat conversations by compacting historical context while preserving key decisions and intent. By reducing token growth and context drift, it improves response quality, performance, and cost predictability over extended sessions. The implementation is designed to be extensible and transparent, making context lifecycle management a first‑class concern for agent development.

[Spec]

Description

The goal of this approach is inject compaction as part of the IChatClient architecture. This approach maintains an index that allows for presenting a compacted version of the conversation without modifying the source "chat history".

Features

• Built-in compaction strategies
• Supports pipeline strategy
• Supports direct invocation
• Supports IChatReducer
• State retained with session serialization
• Incremental message processing
• Atomic tool-call grouping
• Includes tool loops
• Retains original chat history

Details

Compaction occurs via a CompactionStrategy. A set of strategies are incuded as part of this initial release, including a pipeline strategy that is able to sequentially apply one or more strategies:

Strategy	Aggressiveness	Preserves context	Requires LLM	Best for
ToolResultCompactionStrategy	Low	High — only collapses tool results	No	Reclaiming space from verbose tool output
SummarizationCompactionStrategy	Medium	Medium — replaces history with a summary	Yes	Long conversations where context matters
SlidingWindowCompactionStrategy	High	Low — drops entire turns	No	Hard turn-count limits
TruncationCompactionStrategy	High	Low — drops oldest groups	No	Emergency token-budget backstops
PipelineCompactionStrategy	Configurable	Depends on child strategies	Depends	Layered compaction with multiple fallbacks
ChatReducerCompactionStrategy	Configurable	Depends on the IChatReducer	Depends	Compact using an existing IChatReducer

Code

// Setup a chat client for summarization
IChatClient summarizingChatClient = ...;

// Configure the compaction pipeline with one of each strategy, ordered least to most aggressive.
PipelineCompactionStrategy compactionPipeline =
    new(// 1. Gentle: collapse old tool-call groups into short summaries like "[Tool calls: LookupPrice]"
        new ToolResultCompactionStrategy(CompactionTriggers.TokensExceed(0x200)),

        // 2. Moderate: use an LLM to summarize older conversation spans into a concise message
        new SummarizationCompactionStrategy(summarizingChatClient, CompactionTriggers.TokensExceed(0x8000)),

        // 3. Aggressive: keep only the last N user turns and their responses
        new SlidingWindowCompactionStrategy(CompactionTriggers.TurnsExceed(4)),

        // 4. Emergency: drop oldest groups until under the token budget
        new TruncationCompactionStrategy(CompactionTriggers.GroupsExceed(0x1000)));

AIAgent agent =
    agentChatClient.AsAIAgent(
        new ChatClientAgentOptions
        {
            Name = "ShoppingAssistant",
            ChatOptions = new()
            {
                Instructions = "...",
                Tools = [...],
            },
            CompactionStrategy = compactionPipeline,
        });

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines
• All unit tests pass, and I have added new tests where possible
• Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.