Skip to content

mtaha-ai/agentic-document-intelligence

Repository files navigation

Agentic Document Intelligence

Multi-agent RAG that reads charts, tables, and figures, not just text.

Open In Colab

A multi-agent system that answers questions and extracts structured data from documents where the interesting information is not always in the text layer. Reports, invoices and spec sheets tend to hide their best numbers in charts and tables. Text-only RAG pipelines silently miss those. This project treats figures as first-class citizens: a vision agent opens the actual pixels, reads the chart, and feeds structured findings back into the same grounding and critique loop as every text source.

Built on LangGraph as a real state machine (conditional edges, a genuine correction cycle, checkpointing, human-in-the-loop interrupts), not a linear chain with extra steps.

Architecture

How it works

A supervisor routes each request. The retriever pulls text and figure chunks from Qdrant. If the question depends on visual content, the vision agent reads the referenced figures with a vision-language model and returns structured FigureReading objects. The reasoner drafts an answer with Pydantic-validated output, and a critic grades it against the retrieved sources. An ungrounded draft does not reach the user: the critic sends it back, either to retrieval (sources were missing something, the critique is folded into the next query) or to the reasoner (sources were fine, the draft misused them). The loop is capped, a stubborn failure gets surfaced with low confidence instead of spinning.

Extractions pause at a LangGraph interrupt before finalizing. The draft and the critic's verdict are shown to a human, who approves or edits. Because the graph is checkpointed to SQLite, the pause survives process restarts, which is what makes the pattern usable outside a notebook.

The part I care about most, coming from a computer vision background: the sample quarterly report deliberately states the Q4 revenue figure only inside a bar chart, never in the text. A text-only pipeline answers that question wrong or not at all. Here, retrieval surfaces the figure by its caption, the vision agent reads the bars, and the critic checks the final answer against what the figure actually shows.

Demo

Fastest way to see it run: open the Colab notebook above, it goes from a fresh VM to the full eval on a free T4 in about 20 minutes.

Output from a real run (Colab T4, all-local models). The question can only be answered by reading the report text; the system cites the exact source chunk:

$ agentdoc ask "According to the revenue chart, which quarter had the largest growth?"
{
  "answer": "Q3",
  "citations": [
    {
      "chunk_id": "quarterly_report-p1-text",
      "quote": "Q3 was the strongest quarter in company history at EUR 4.8M,
                driven by the rollout of the document automation product line."
    }
  ],
  "confidence": "high"
}
grounded=False  correction_loops=2

That grounded=False is the critic being deliberately conservative: "largest growth" requires arithmetic over the chart values, so it refuses to certify and attaches its critique instead of hiding it. A make demo target exists to record a GIF of the interactive session (needs vhs and a running Ollama).

Results

Model: llama3.1:8b + qwen2.5vl:7b via ollama, 8 tasks, deterministic string/field matching (see src/agentdoc/eval/). Run on a free Colab T4.

Configuration Accuracy Grounded Retrieval hit Avg. correction loops
Full graph (with critic) 88% 0% 100% 2.0
Critic disabled 88% n/a 100% n/a

Scoring is deterministic (exact string and field matching, see src/agentdoc/eval/metrics.py), so the numbers are stricter than LLM-judged ones and fully reproducible. The dataset includes a trap question whose answer is not in the documents at all; credit is only given for saying so. The Colab notebook in notebooks/ reproduces the whole table on a free T4 GPU. Rerun with make eval, make eval-no-critic, make update-readme.

Two honest observations from this run. First, the 8B critic never certified a draft as grounded: it treats any claim requiring arithmetic over chart values as unsupported, so every run hit the loop cap and surfaced with the critique attached. Correct behavior, conservative grader. On this small dataset that strictness did not change accuracy, which is exactly the kind of thing an ablation column is for. Second, the one miss (reading the Q4 bar value out of the chart into a numeric answer) is a vision-to-reasoner handoff failure, not retrieval: the figure was retrieved every time. Both are on the improvement list.

Quickstart (free, local)

Requirements: Python 3.11+, Docker, Ollama.

git clone https://github.com/mtaha-ai/agentic-document-intelligence.git
cd agentic-document-intelligence
pip install -e ".[dev]"

ollama pull llama3.1:8b
ollama pull qwen2.5vl:7b

docker compose up -d        # Qdrant
python scripts/make_samples.py
agentdoc ingest examples/sample_docs

agentdoc ask "What revenue value does the bar chart show for Q4 2025?"
agentdoc extract "extract this invoice" --schema invoice

The Python package and CLI are named agentdoc (package names cannot contain hyphens). The extract command runs until the human review interrupt, prints the draft plus the critic's verdict, and waits for your approval before finalizing.

No Docker? Set vector_store.backend: chroma in configs/default.yaml.

Switching providers

The whole backend swaps with one config field. Everything else, including the vision agent and structured output, stays identical:

llm:
  provider: openai          # ollama | openai | anthropic | azure | bedrock | vertex
  model: gpt-4o-mini
  vision_model: gpt-4o-mini
Provider Text example Vision example Install extra
Ollama (default) llama3.1:8b qwen2.5vl:7b none
OpenAI gpt-4o-mini gpt-4o-mini [openai]
Anthropic claude-sonnet-5 claude-sonnet-5 [anthropic]
Azure OpenAI your deployment your deployment [openai]
AWS Bedrock eu.anthropic.claude-... same [aws]
Vertex AI gemini-2.0-flash same [vertex]

Development and evaluation ran entirely on the free local path. If you use a paid provider, set a spending cap in the provider's console first; the full eval is a few hundred requests, which is well under one euro on gpt-4o-mini but caps are free insurance.

Observability

Set LANGSMITH_TRACING=true and an API key in .env (free tier) and every run is traced end to end: routing decision, retrieved chunks, each vision call with its image, critic verdicts and loop iterations. Debugging a multi-agent graph without traces is guesswork, with them it is reading.

Design decisions

Figures are indexed by caption, read on demand. Embedding image content directly (CLIP-style) sounds nicer but makes retrieval quality harder to reason about, and captions plus page context already get the right figure into the top results on realistic documents. The expensive vision call happens only for figures that retrieval actually surfaced.

The critic distinguishes two failure modes. "Sources are missing information" and "the draft misstates fine sources" need different fixes (new retrieval vs. a rewrite). Collapsing them into one retry edge wastes loops on the wrong fix, so the conditional edge is three-way.

Deterministic eval scoring. LLM-as-judge would be friendlier to the numbers and much easier to game by accident. Exact matching is harsh but nobody has to trust a judge prompt.

One process, real interrupts. Human review uses LangGraph's checkpointed interrupt(), not an input() call bolted into a node. The same graph works unattended (eval sets require_human_review: false) and interactive, and a paused extraction can be resumed by a different process later.

What I would improve next

Table extraction currently trusts the text layer; for scanned documents it should fall back to the vision agent on a page render. Retrieval is dense-only, and BM25 hybrid would help on exact identifiers like invoice numbers. The figure caption heuristic is a regex, a layout model would do better on messy documents. And the eval set is small; growing it is mechanical work that pays off linearly.

Project layout

src/agentdoc/
  graph.py            LangGraph state machine (routing, critic loop, interrupt)
  state.py            shared graph state
  agents/             supervisor, retriever, vision, table, reasoner, critic
  providers/          one factory, six backends
  retrieval/          BGE embeddings, Qdrant/Chroma stores
  ingestion/          PDF parsing with figure extraction, chunking
  schemas/            Pydantic contracts (also the structured-output specs)
  eval/               dataset, metrics, runner
configs/              YAML config, one file per environment
examples/             sample docs (synthetic) and a walkthrough
notebooks/            Colab notebook reproducing the eval for free
scripts/              sample generator, README updater, demo recorder
tests/                unit tests for the deterministic parts

License

MIT

About

Multi-agent document intelligence system built on LangGraph. A vision agent reads charts, tables, and figures that text-only RAG misses, a critic agent verifies every answer against its sources, and structured extractions pause for human review. Runs fully local on Ollama, swaps to any cloud LLM via one config line.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors