Skip to content

State size threshold for Redis with MinIO overflow#62

Merged
usnavy13 merged 2 commits into
devfrom
session-fix
Mar 3, 2026
Merged

State size threshold for Redis with MinIO overflow#62
usnavy13 merged 2 commits into
devfrom
session-fix

Conversation

@usnavy13
Copy link
Copy Markdown
Owner

@usnavy13 usnavy13 commented Mar 3, 2026

Summary

  • Route large Python session states (> configurable threshold) to MinIO instead of Redis, storing only a hash+metadata pointer in Redis
  • Add STATE_MAX_REDIS_SIZE_MB config (default 100MB, range 1-500) to control the threshold
  • Increase Redis socket timeout from 5s to 30s to handle large state transfers
  • Fix file upload sanitization: sanitize filename before storing in MinIO so all systems use the same name
  • State archival restore now respects the size threshold — won't push oversized states back into Redis

Test plan

  • 145 unit tests passing
  • 48 functional tests passing against live endpoint (all 12 languages, sessions, state persistence, file upload/download/execution)
  • Verified DataFrames persist across executions
  • Verified file upload → execute → generate → download round-trip works

🤖 Generated with Claude Code

Joe Licata and others added 2 commits March 2, 2026 21:02
- Added a new configuration setting `state_max_redis_size_mb` to define the maximum state size for Redis storage.
- Updated the state saving logic in the orchestrator to store large states in MinIO and only save pointers in Redis.
- Enhanced the state archival service to conditionally restore states to Redis based on the size threshold.
- Introduced a new method `save_state_pointer` in the state service to handle saving state metadata when the state exceeds the Redis limit.
- Improved logging to provide insights on state storage decisions and outcomes.
@usnavy13 usnavy13 merged commit 5634235 into dev Mar 3, 2026
2 checks passed
djuillard pushed a commit to On-Behalf-AI/LibreCodeInterpreter that referenced this pull request Apr 21, 2026
State size threshold for Redis with MinIO overflow
@usnavy13 usnavy13 deleted the session-fix branch May 7, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant