Skip to content
This repository was archived by the owner on Sep 28, 2025. It is now read-only.

v1.2.0: Add Extractous Framework and Improve CI Timeout Handling

Choose a tag to compare

@Goldziher Goldziher released this 11 Jul 07:46
· 140 commits to main since this release
da565c1

🚀 New Features

Extractous Framework Integration

  • Added Extractous - Fast Rust-based extraction library with Python bindings
  • 18x faster than traditional Python solutions on some workloads
  • Native performance without garbage collection overhead
  • Built-in OCR support via Tesseract integration
  • Minimal memory footprint (11x less than Python alternatives)

Framework Updates

  • Removed LlamaParse (cloud-based service, not suitable for local benchmarking)
  • Updated framework list to include only locally-running extractors

🔧 CI/CD Improvements

Timeout Handling

  • Added 2-hour timeout for slow frameworks (Docling, MarkItDown, Extractous)
  • Jobs now fail gracefully instead of being cancelled on timeout
  • Create failure markers for timed-out jobs to include in aggregation
  • Enhanced aggregation logic to handle both success and failure cases

Pipeline Robustness

  • Aggregation now runs even when some frameworks timeout
  • Failure reporting includes timeout information
  • Graceful handling of partial benchmark results
  • Added new Extractous benchmark job with timeout protection

📊 Performance & Quality

Memory Profiling

  • Confirmed memory profiling working correctly (251.5MB peak, 43.2% avg CPU)
  • Extractous configured with 1MB text extraction limit
  • Language detection for OCR configuration

Code Quality

  • Applied linting and formatting
  • Removed unused LlamaParse dependencies
  • Updated framework enums and documentation
  • Pre-commit hooks passing

🎯 Framework Comparison

The benchmark suite now includes:

  • Kreuzberg (multiple OCR backends)
  • Docling (IBM Research ML models)
  • MarkItDown (Microsoft lightweight converter)
  • Unstructured (enterprise solution)
  • Extractous (fast Rust-based NEW!)

🚦 Testing

This release triggers the comprehensive benchmark pipeline with:

  • Enhanced timeout handling for slow frameworks
  • Improved failure reporting and aggregation
  • Testing of the new Extractous framework
  • Validation of memory profiling accuracy

The pipeline will now handle timeouts gracefully and produce comprehensive results even when some frameworks fail or timeout.