Release: Production-ready improvements - Performance, metadata, and developer experience enhancements#23
Merged
savourylie merged 30 commits intomainfrom Oct 10, 2025
Merged
Release: Production-ready improvements - Performance, metadata, and developer experience enhancements#23savourylie merged 30 commits intomainfrom
savourylie merged 30 commits intomainfrom
Conversation
- Implement concurrent LLM calls with controlled delays to avoid server overload - Add new command-line arguments for concurrency and delay settings - Refactor run_benchmark_evaluation to use concurrent tasks - Update related scripts to support new concurrency options
…tions - Remove concurrent_delay argument from run_benchmark.py, scripts/load_and_run.py, and scripts/run_evaluation.py - Update _evaluate_single_question and run_benchmark_evaluation functions to remove staggered execution logic - This change simplifies the concurrency control mechanism, using asyncio's built-in concurrency primitives without adding artificial delays
…iciency - Implement concurrent evaluation of questions using asyncio Semaphore - Refactor question evaluation logic into a separate function - Update default LLM provider to OpenAI in run_benchmark.py - Change default LLM provider to Gemini in utils.py - Optimize data loading and evaluation flow
…king improvements - Added concurrent evaluation support from feat/concurrent branch - Extended retrieval metrics to support MSC dataset alongside LME - Enhanced string normalization with regex-based punctuation removal for better matching - Improved argument parsing with provider-specific model defaults - Updated gitignore to include .vscode/ directory - Maintained backward compatibility while adding new features
…luation - Fixed 'list' object has no attribute 'lower' error by using correct function call - Restored calculate_enhanced_retrieval_metrics() instead of simple calculate_retrieval_metrics() - Includes both enhanced and legacy metrics for comparison - Maintains concurrent evaluation while preserving enhanced retrieval tracking features
…e mode - Add --retrieval-verbose flag to capture and display full retrieved memory content - Enhance incorrect questions CSV export with recall flags for better analysis - Support CSV format parsing in question IDs file for compatibility - Improve memory content display with score information and truncation
Feat/110 retrieval tracking
…attern Add singleflight pattern to prevent concurrent agent creation attempts for the same agent name. Implement POST-first flow with server-side idempotent create and fallback GET handling. Add retry logic with exponential backoff for network and server errors. Include thread-safe locking for sync client and proper error handling for conflict scenarios. Also includes: - Add idempotency_key parameter to agent creation API - Fix potential NoneType errors in benchmark retrieval - Update default LLM provider for benchmarks - Add comprehensive tests for agent idempotency
- Add .cursorindexingignore to .gitignore - Exclude .specstory directory from version control
Bug/118 none agentname
…king Add version compatibility utility and update health endpoint to include version information for SDK compatibility checking as requested in issue #116.
feat: add version field to health endpoint for SDK compatibility chec…
refactor: replace openrouter with openai compatible client in MSC acc…
…-digit patch versions
…t models, and implement deprecation warnings for legacy parameters
…bility with OpenAI and Gemini APIs
…-resp-fields Feat/146 prompts handle new query resp fields
…i-calls feat: add debug logging support controlled by MEMFUSE_DEBUG environme…
feat: update MemFuse base URL to use port 8765 across examples and tests
feat: add optional UI dependencies and update examples to handle miss…
feat: streamline server health checks during session initialization
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Overview
This release brings significant improvements to MemFuse Python SDK focusing on performance optimizations, new API features, and enhanced developer experience. Includes 2,308 additions across 51 files with major refactorings and new capabilities.
✨ Key Features
Performance & Reliability
API Enhancements
Developer Experience
Infrastructure
🧪 Testing
📦 Migration Notes
🔗 Related PRs