This repository was archived by the owner on Sep 28, 2025. It is now read-only.
Repository navigation
v1.2.0: Add Extractous Framework and Improve CI Timeout Handling
🚀 New Features
Extractous Framework Integration
- Added Extractous - Fast Rust-based extraction library with Python bindings
- 18x faster than traditional Python solutions on some workloads
- Native performance without garbage collection overhead
- Built-in OCR support via Tesseract integration
- Minimal memory footprint (11x less than Python alternatives)
Framework Updates
- Removed LlamaParse (cloud-based service, not suitable for local benchmarking)
- Updated framework list to include only locally-running extractors
🔧 CI/CD Improvements
Timeout Handling
- Added 2-hour timeout for slow frameworks (Docling, MarkItDown, Extractous)
- Jobs now fail gracefully instead of being cancelled on timeout
- Create failure markers for timed-out jobs to include in aggregation
- Enhanced aggregation logic to handle both success and failure cases
Pipeline Robustness
- Aggregation now runs even when some frameworks timeout
- Failure reporting includes timeout information
- Graceful handling of partial benchmark results
- Added new Extractous benchmark job with timeout protection
📊 Performance & Quality
Memory Profiling
- Confirmed memory profiling working correctly (251.5MB peak, 43.2% avg CPU)
- Extractous configured with 1MB text extraction limit
- Language detection for OCR configuration
Code Quality
- Applied linting and formatting
- Removed unused LlamaParse dependencies
- Updated framework enums and documentation
- Pre-commit hooks passing
🎯 Framework Comparison
The benchmark suite now includes:
- Kreuzberg (multiple OCR backends)
- Docling (IBM Research ML models)
- MarkItDown (Microsoft lightweight converter)
- Unstructured (enterprise solution)
- Extractous (fast Rust-based NEW!)
🚦 Testing
This release triggers the comprehensive benchmark pipeline with:
- Enhanced timeout handling for slow frameworks
- Improved failure reporting and aggregation
- Testing of the new Extractous framework
- Validation of memory profiling accuracy
The pipeline will now handle timeouts gracefully and produce comprehensive results even when some frameworks fail or timeout.