DocIntel

Vision-first document AI. Drop a PDF or image, get structured JSON in under 2 seconds. Local or cloud.

🔗 Live dashboard: https://docintel.ysiddo-ai-projects.app/ · drag-drop a PDF/image. On-demand backend (first request ~30–60 s to wake). Self-hosting: see SELF_HOSTING.md. Route B local vision spins up a GPU on demand (~4–5 min cold).

What It Does

• 3 extraction routes: Claude Sonnet 4.6 Vision (premium), Ollama local vision (private/$0-per-page — .env default: llama3.2-vision; production-validated: qwen2.5-VL:7b on NVIDIA T4 GPU, see eval/BENCHMARK.md), Tesseract+LLM (fallback)
• Multi-currency & multi-locale: amounts in US/EU/spaced/Swiss formats and 45+ currencies (USD, EUR, GBP, JPY, INR, CNY, XOF/FCFA, …) are normalized to ISO 4217 + float; dates to ISO 8601 — a deterministic layer (services/normalize.py) on top of the LLM. OCR runs eng+fra+deu+nld+spa+ita.
• Inputs: PDF (native or scanned), PNG, JPEG — auto-detected. PDFs are rendered per page; images flow straight through.
• Multi-page & large documents: every page is processed and fields aggregated across pages (a total on a later page, multi-page contracts). 100+ page PDFs are handled via map-reduce — pages are split into chunks, extracted concurrently, and merged (MAX_PDF_PAGES default 200). The OCR route concatenates/chunks full-document text the same way.
• Handwriting & mixed languages: the vision routes read handwritten entries and EN/FR/DE/NL/ES/IT documents; numbers are normalized (EU 1.234,56 → 1234.56; West-African 1 003 000 FCFA → 1003000) and currencies to ISO-4217, including the West-African CFA franc (FCFA/CFA → XOF, Central-African → XAF).
• Doc-type-aware schemas: invoice, contract, receipt, financial_report, auction_listing, form
• Confidence scores on every extraction; retry-on-bad-JSON for reliability
• /classify-image endpoint: vision-first object classification for auction/inventory aggregation
• Batch at scale: bounded-concurrency async jobs that process hundreds/thousands of files with per-file error isolation
• /process: one-shot upload → auto-classify → multi-page extract → JSON
• Full web dashboard at /

Quick Start

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env  # fill in API keys
uvicorn api:app --port 8001

Open http://localhost:8001/

Endpoints

Method	Path	Purpose
GET	/health	Liveness check
POST	/process	One-shot: upload → auto-classify → multi-page extract
POST	/classify	Fast doc-type classification
POST	/classify-image	Vision-first object classification
POST	/extract	Full extraction (file + route + doc_type), multi-page
POST	/extract-llm	LLM extract from raw text
POST	/extract-tables	PDF tables via pdfplumber
POST	/batch/upload	Start background batch job
GET	/batch/{job_id}	Job status
GET	/batch/{job_id}/results	Job results

Architecture

              ┌─────────────┐        PDFs → render EVERY page (pdf_to_pngs)
PDF / IMG ───►│   api.py    │───►    or extract full-document text (extract_text_from_pdf)
              │  FastAPI    │              │
              └──────┬──────┘              ▼
                     │            ┌──────────────────┐
        route ┌──────┼──────┐     │ multi-page images │
              ▼      ▼      ▼     └──────────────────┘
        vision_   vision_   ocr_extractor ─► llm_extractor
        (premium) (local)   (Tesseract)      (text → JSON)
         Claude    Ollama      multilingual   Haiku cleanup
         Vision    Vision      OCR            + confidence
              └──────┴──────────────┴───────────────┘
                     ▼
        structured JSON  { ..fields.., _confidence, _pages }

Validation

Validated on real, multilingual third-party invoices (EN/FR/DE/NL, invoice2data test set) — see eval/EVAL_REAL.md. Route A (Claude Vision) and Route C (Tesseract + LLM) both score 100% on the fields each document carries; /classify-image returns invoice 0.98–0.99. Reproduce with bash eval/fetch_real_invoices.sh then python eval/run_real_eval.py --route vision_premium.

Scope & Notes

• Multi-page / large docs: up to MAX_PDF_PAGES (default 200) per document; documents larger than VISION_PAGES_PER_CALL (default 8) pages are chunked and merged via map-reduce. Vision pages are downscaled past VISION_MAX_EDGE px to bound token cost.
• Handwriting: handled by the vision routes (Route A is strongest). The pure-OCR route (Route C) is weaker on handwriting — use a vision route for handwritten docs.
• Currencies: ISO-4217 generic; EU decimal/comma and West-African FCFA (space-grouped, no decimal subunit → XOF/XAF) formats normalized. Ambiguous thousands/decimal separators on low-quality scans can still mislead the pure-OCR route.
• Route C non-English: install the matching Tesseract packs (tesseract-ocr-fra/deu/nld/...); falls back to English automatically if a pack is missing.

Benchmark

A 550-document, multi-type, multilingual benchmark (receipts, invoices, forms; including multi-page and handwriting) is reproducible via python eval/build_corpus.py and scored with python eval/run_benchmark.py. See eval/BENCHMARK.md for full results.

Route	Model	Test set	Accuracy
A — vision_premium	Claude Sonnet 4.6 Vision	multilingual invoices (multi-page)	100%
A — vision_premium	Claude Sonnet 4.6 Vision	phone-photo receipts (CORD, 40 docs)	92.5%
A — vision_premium	Claude Sonnet 4.6 Vision	SROIE world-standard invoices	95%
B — vision_local	Ollama qwen2.5-VL 7B (NVIDIA T4)	CORD phone-photo receipts (100 docs)	77%
B — vision_local	Ollama qwen2.5-VL 7B (NVIDIA T4)	French + FCFA (XOF) sample	100%
C — ocr_fallback	Tesseract + Claude Haiku	clean invoices	100%
C — ocr_fallback	Tesseract + Claude Haiku	CORD phone-photo receipts	28.5%

Robustness: All 550 documents processed — 550/550 success rate with per-file error isolation.

Tests

62 test functions across smoke, API, extraction, batch, and benchmark scripts:

pytest tests/ -q

License

AGPL-3.0

⚖️ License & Enterprise Use (Dual-License)

This project is open-source under the AGPL-3.0 License. It is completely free for researchers, students, and open-source hobbyists.

Commercial Use: The AGPLv3 license requires that any proprietary network service (SaaS, internal corporate tools) that uses or modifies this code must also open-source its entire backend.

If you wish to use this framework in a closed-source commercial environment, or require Enterprise features (SSO, Active Directory, Custom VPC Deployment, Strict RBAC), you must obtain a Commercial License. Please reach out to discuss commercial licensing and integration consulting.

📡 Anonymous Telemetry

This project collects anonymous, GDPR-compliant startup pings to help the author understand usage volume and prioritize development.

• What is collected: A startup event timestamp and anonymized deployment origin. No API keys, no user prompts, and no sensitive application data is ever collected.
• How to disable: We respect your privacy and development environment. To opt-out, simply set TELEMETRY_OPT_OUT=true in your .env file.

Licensing

This project is licensed under the AGPL-3.0 License.

Commercial Use: If you wish to use this software commercially without releasing your own source code, please see COMMERCIAL.md to obtain a commercial license.

Telemetry: See TELEMETRY.md for our privacy-first data practices.