feat: add token budget circuit breaker and session log to agent loop#245
Merged
Conversation
…nd token budget circuit breaker **Added:** - Session logging framework for agent loops, writing append-only JSONL logs per task via new `SessionLog` and `SessionLogConfig` types - Support for replaying message history from session logs for crash recovery and debugging - Proactive conversation context compaction with configurable thresholds and cadence to manage token usage, including semantic recap of trimmed messages - Per-loop token budget circuit breaker (`BudgetConfig`) with env-var overrides, terminating loops that exceed configured input, output, or total token limits - New `LoopEndReason::BudgetExceeded` variant for reporting budget-related terminations - Env-var driven configuration for agent loop, context, budget, and session log parameters **Changed:** - Anthropic provider now attaches prompt cache breakpoints to system and tool definitions when enabled, improving caching effectiveness - Agent loop runner tracks session log events for every major step (messages, usage, compaction, terminal outcome) for observability - Conversation compaction logic now distinguishes between proactive (threshold-based) and reactive (hard ceiling) triggers, with improved summary markers - All configuration objects extended to support env-var overrides for fine-grained tuning without code changes - Tests updated and expanded to cover new compaction, budget, and session log behaviors **Removed:** - Legacy conversation trimming logic replaced with new proactive compaction flow
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #245 +/- ##
==========================================
+ Coverage 74.94% 75.10% +0.16%
==========================================
Files 382 383 +1
Lines 80418 81461 +1043
==========================================
+ Hits 60268 61183 +915
- Misses 20150 20278 +128 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
**Changed:**
- Reformatted multiline function calls and match arms for improved readability in `context.rs`, `runner.rs`, and `session_log.rs`
- Standardized closure formatting for iterator methods in `session_log.rs`
- Corrected typo in log message ("unparseable" to "unparsable") in `session_log.rs`
…ion log **Added:** - Added tests for environment variable parsing functions to ensure correct defaulting and fallback logic in `config.rs` - Introduced tests for `describe_reason` covering all `LoopEndReason` variants in `runner.rs` - Added tests for session log compaction, message replay, unparsable line handling, and path sanitization in `session_log.rs`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Key Changes:
Added:
BudgetConfig) that ends loops withBudgetExceededreason when input, output, or total token budgets are reached; configurable via env varsSessionLogandSessionLogConfig) that records all conversation events, tool results, usage stats, and outcomes for each agent loop in a crash-tolerant mannerChanged:
Removed:
maybe_compactand semantic recap mechanism