Skip to content

v0.4.0 - Document Extractor Registry & Cost Estimation

Latest
Latest

Choose a tag to compare

View all tags
@molotovsingh molotovsingh released this 18 Oct 06:34
· 47 commits to main since this release
v0.4.0
8b312f4

Major Features

Document Extractor Registry

Unified catalog for all document extractors with centralized prompt injection system.

Key additions:

  • document_extractor_catalog.py (352 lines) - Centralized metadata registry with enabled flags, prompt IDs, and cost tracking
  • doc_extractor_prompts.py (94 lines) - Named prompt registry with version control
  • Dynamic UI/CLI generation from catalog.list_extractors(enabled=True)
  • Factory validation and auto-prompt injection
  • 348 comprehensive regression tests

Impact: Lego-style extractor additions - new extractors require only catalog entry update, no factory/UI/CLI changes.

Two-Layer Cost Estimation System

Pre-extraction cost preview with fast page count heuristics (no API calls required).

Key additions:

  • cost_estimator.py (540 lines) - Model catalog integration with dual-layer cost breakdown
  • document_page_estimator.py (277 lines) - Lightweight page counting for PDF/DOCX/images
  • UI components for cost visualization with confidence indicators
  • 995 test cases + 919 lines of documentation

Layers:

  1. Document extraction costs (Layer 1) - OCR/vision processing per page
  2. Event extraction costs (Layer 2) - LLM token usage per document

Improvements

Bug Fixes

  • OpenRouter JSON Parsing - Improved error recovery for malformed responses with graceful degradation
  • Added regex-based fallback for markdown-wrapped JSON
  • Enhanced logging with response previews (96 line changes)

Infrastructure

  • Archived Gemini Vision document extractor to src/core/archive/
  • Removed deprecated GeminiDocConfig from config.py
  • Updated order template with enhanced structure (acceptance criteria, scope, constraints)
  • Enhanced CLAUDE.md with 88 new lines of registry architecture docs

Completed Development Orders

✅ doc-extractor-registry-001 - Centralized registry implementation
✅ cost-estimator-001 - Initial cost estimation framework
✅ cost-estimator-002 - Two-layer architecture implementation
✅ cost-estimator-003 - UI integration and testing
✅ remove-gemini-vision-001 - Legacy extractor cleanup

Statistics

  • Files changed: 23
  • Lines added: 5,207
  • Lines removed: 125
  • Net change: +5,082 lines
  • Commits: 5
  • Test coverage: 1,343 new test cases

Migration Notes

✅ No breaking changes - All changes are backward compatible.

  • Document extractor registry is optional - existing direct extractor instantiation continues to work
  • Cost estimation is opt-in via UI components
  • No changes required to existing extraction workflows

What's Next

Recommended versions:

  • Patches/bug fixes: v0.4.1 (PATCH)
  • New features: v0.5.0 (MINOR)

🤖 Generated with Claude Code

Assets 2
Loading