Fix LRU eviction silent failure allowing unbounded memory growth (#449)#567
Merged
Fix LRU eviction silent failure allowing unbounded memory growth (#449)#567
Conversation
github-actions
bot
added
agents
kovtcharov
approved these changes
Mar 19, 2026
itomek
force-pushed
the
fix/lru-eviction-449
branch
from
49f417f to
6cb61aa
Compare
itomek
force-pushed
the
fix/lru-eviction-449
branch
from
6cb61aa to
34fbfa1
Compare
itomek
added a commit
that referenced
this pull request
Mar 19, 2026
Forward --max-indexed-files to UI server via GAIA_MAX_INDEXED_FILES env var. Add DB-level capacity check with LRU eviction in upload_by_path() since per-upload RAGSDK instances can't track cross-upload state. Add O_BINARY flag on Windows for safe_open_document() and RAGSDK._safe_open() to prevent binary/text mode issues with fd-based file reads.
- Add pre-flight capacity check (_has_indexing_capacity) that rejects indexing when memory limits are exceeded and eviction is disabled, returning success=False with memory_limit_reached=True and a descriptive error message - Change _check_memory_limits() to return bool (True if limits met) - Add structured logging to _evict_lru_document and _check_memory_limits for eviction attempts, failures, and limit violations - Fix pre-existing bug: cache-load path never set file_access_times or file_index_times, making cached files invisible to LRU eviction - Add max_indexed_files and max_total_chunks to index_document stats dict - Add > 1 guard to files eviction loop (parity with chunks loop) - Add 6 unit tests covering all acceptance criteria Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add --max-indexed-files and --max-total-chunks flags to `gaia chat` so memory limits can be set at runtime for UI testing and validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use monotonic counter for LRU ordering instead of time.time() (Windows resolution issue) - Remove redundant default fallbacks in cli.py kwargs.get() - Document --max-indexed-files and --max-total-chunks in CLI reference - Rename misleading test and add proper eviction-failure test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_has_indexing_capacity()(new read-only method) checks limits before indexing starts. If at limit with eviction disabled, returnssuccess=Falsewithmemory_limit_reached=Trueand a descriptive error — no silent growth._evict_lru_document()and_check_memory_limits()now emit structuredself.logwarnings/errors on all failure paths (disabled, no files, eviction failed, limit exceeded)._check_memory_limits()returnsbool: callers can now detect post-index limit violations; setsmemory_limit_warning=Truein stats.file_access_times/file_index_times, making them invisible to LRU eviction.max_indexed_filesandmax_total_chunksalways present inindex_document()stats dict.--max-indexed-filesand--max-total-chunkswired throughChatAgentConfig→RAGConfigfor runtime configuration and UI testing.TestMemoryLimitscovering all acceptance criteria.Test plan
uv run python -m pytest tests/test_rag.py -xvs— 28/28 passpython util/lint.py --all --fix— passes--no-lru-evictionflag, not yet implemented):gaia chat --ui --max-indexed-files 2 --no-lru-eviction→ upload 3 docs → 3rd rejected with "Memory limit reached" errorgaia chat --ui --max-indexed-files 2→ upload 3 docs → 3rd succeeds, 1st evictedOpen items
--no-lru-evictionCLI flag not yet implemented — needed to fully validate the rejection path through the UI (see issue comments for full test instructions)Closes #449
🤖 Generated with Claude Code