[Feature] Implement Dynamic Chunking & Recursive State Management in the Summarizer Agent

[vakrahul]

Context & Problem

Currently, the summarizer agent (src/agents/summarizer.py) is responsible for memory compression and text summarization. However, when dealing with massive context payloads or extensive historical logs, we risk hitting hard token limits or suffering from LLM "lost in the middle" attention degradation.

Proposed Solution

We need to supercharge the summarization pipeline by introducing a dynamic, token-aware chunking mechanism. By hooking into the model registry (src/models/registry.py), the agent can assess the maximum context window of the active model (whether it's OpenAI, Claude, Deepseek, etc.) and intelligently route the data through a recursive, graph-based summarization workflow.

Key Implementation Steps

• Token Estimation: Integrate a lightweight token counting utility before the payload is dispatched to the model.
• Dynamic Routing: Update the agent logic to split large payloads into semantic chunks if they exceed a safe threshold (e.g., 80%) of the active model's context limit.
• Recursive Aggregation: Build a stateful loop utilizing a graph-based state machine pattern—to summarize individual chunks and map-reduce them into a final, highly accurate master summary.
• Fallback Logic: Ensure seamless degradation and retry mechanisms if an API rate limit is hit during a multi-step summarization loop.

Why This Matters

This will make the overarching agent workflows significantly more resilient. It ensures we extract high-fidelity context for the user without dropping critical data points during memory retrieval.

Impacted Files

• src/agents/summarizer.py
• src/models/registry.py
• src/models/base.py

Open Questions for the Maintainers

Should we expose the chunk overlap size and recursive depth limits as configurable parameters in the global environment settings, or keep them strictly defined within the agent's base class?

[vakrahul]

@hunterbastian @sirenexcelsior @Genmin could assign me

[vakrahul]

@ishaanxgupta and @ved015 could you assign me this

[ved015]

Suree assigned

[ved015]

I think we can keep them in the base class i dont think we need configurable parameters