Deal with large tool results, enforce size limit, trigger compaction#94
Merged
Deal with large tool results, enforce size limit, trigger compaction#94
Conversation
Two-layer defense against tool results that blow up the LLM context: Layer 1: read_files pre-read size check - Files over 200KB are rejected with an error showing exact size and line count - Detects single-line/minified files and mentions it in the error message - New 'ignore_size_limit' parameter lets the LLM explicitly opt in to large reads - Line-range reads always bypass the limit since the user chose a specific range Layer 2: Prompt-too-long error recovery in agent loop - Catches 'prompt too long' errors from all LLM providers (Anthropic, OpenAI, etc.) - Replaces large tool results (>50KB) from the current turn with PromptTooLongError placeholders containing actionable suggestions - Scoped to current turn only — older tool results are not touched - Fallback when no single result is large enough: drops the last assistant/tool-result exchange and forces context compaction - Works generically for all tools (read_files, execute_command, web_fetch, etc.) New type: PromptTooLongError — a ToolResult placeholder with tool-specific guidance (line ranges, search_files, head/tail, CSS selectors). Includes 5 new tests for the read_files size check covering: rejection, ignore_size_limit bypass, line-range bypass, line count reporting, and minified file detection.
UI updates: - replace_large_tool_results() now returns Vec<(tool_id, error_message)> instead of bool, enabling the caller to send UpdateToolStatus events - Caller sends ToolStatus::Error with 'Prompt Too Long' message for each replaced tool, updating both GPUI and terminal UI tool blocks Integration tests with mocked LLM provider: - test_prompt_too_long_replaces_large_tool_results: verifies the full flow of tool execution -> prompt-too-long error -> result replacement -> successful retry, including UI event assertions - test_prompt_too_long_fallback_drops_exchange_and_compacts: verifies the fallback path when no tool result exceeds the 50KB replacement threshold — exchange is dropped and compaction is forced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.