InspectIQ

Document-intelligence and anomaly-detection pipeline over real PDFs, built for a property-management setting. It turns unstructured documents — inspection reports and supplier invoices — into grounded, source-cited answers and into auditable lists of cost irregularities, with a human-approval gate in front of every consequential output.

Stack: Python · pdfplumber · Qdrant · multilingual e5 embeddings · Claude

Two capabilities from one ingestion pipeline:

1. Document intelligence (Q&A). Ask questions in natural language and get answers grounded strictly in the source PDFs, cited down to the page (enligt besiktningsprotokoll.pdf, sid 2). If the answer isn't in the documents, the system says so instead of inventing one.
2. Invoice anomaly audit. Parse a supplier invoice into structured line items and totals, then flag cost irregularities — duplicate charges, VAT miscalculations, totals that don't add up, line-item math errors, missing mandatory fields.

The domain is deliberately Swedish property management (besiktningsprotokoll + faktura), but nothing in the pipeline is domain-specific.

Why it's built this way

• Page-level citations. Every chunk keeps its source file and page number, so answers are checkable, not just plausible.
• Multilingual embeddings. The corpus is Swedish; an English-only embedding model retrieves poorly on it, so InspectIQ uses intfloat/multilingual-e5-small with the correct query: / passage: prefixes. Runs locally, no API key.
• Deterministic anomaly core. The financial checks are arithmetic, not a prompt — duplicates, VAT, totals and line math are caught exactly and reproducibly. An optional Claude pass adds semantic judgement (vague or unrelated charges) on top.
• A validation gate is the trust boundary. AI proposes, a human approves:
  • Q&A: if the best retrieval score is below threshold, the question is routed to a human rather than answered on weak evidence.
  • Audit: every finding is returned with needs_review: true. Nothing is auto-approved.
• Measured, not assumed. A key-free retrieval-eval harness reports Hit Rate, MRR and Recall, and checks the gate doesn't reject valid questions.

Architecture

PDF (inspection report / invoice)
        │
        ▼
  pdfplumber  ──►  page-aware text + line-item tables
        │
        ├──────────────► Q&A path
        │      chunk → embed (e5) → Qdrant → retrieve
        │      → validation gate → grounded answer w/ page citations (Claude)
        │
        └──────────────► Audit path
               parse line items + totals
               → deterministic checks (dup / VAT / total / math / fields)
               → optional semantic check (Claude)
               → findings, each needs_review: true

Qdrant local/server toggle: with no config, InspectIQ uses an embedded on-disk Qdrant (no Docker). Set QDRANT_URL=http://localhost:6333 to point at the Qdrant server in docker-compose.yml — same client API either way.

Quickstart

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

python gen_samples.py            # generate the sample PDFs
python cli.py ingest --recreate  # parse + embed into Qdrant (embedded mode)

python cli.py ask "Vilka akuta anmärkningar finns i besiktningen?"
python cli.py audit-invoice      # audit samples/faktura.pdf
python eval/eval_retrieval.py    # retrieval metrics

Grounded answer synthesis and the semantic invoice check use Claude — set ANTHROPIC_API_KEY (see .env.example). Everything else (retrieval, the gate, the arithmetic audit, the eval harness) runs without a key.

Example: invoice audit

samples/faktura.pdf ships with three planted irregularities; the auditor catches all three, deterministically:

DUBBLETT  (hög)   Möjlig dubblettdebitering: raden förekommer 2 gånger.
MOMS_FEL  (hög)   Moms felaktig: 25% av 17050.00 = 4262.50, men angivet 4600.00.
TOTAL_FEL (hög)   Totalsumma: netto + moms = 21650.00, men angivet 22000.00.
→ 3 avvikelser, needs_review=True

Eval

python eval/eval_retrieval.py over 10 labelled questions (key-free):

Metric	Value
Hit Rate@5	1.000
MRR	0.850
Recall@5	1.000
Gate false-rejects	0 / 10

The corpus is small and clean, so retrieval metrics saturate — the harness exists to make quality measurable and to catch regressions, the same discipline I'd apply to a larger corpus. The off-topic control question (Vem vann fotbolls-VM 1994?) scores 0.74 and is correctly refused by the gate.

Deploying behind a safe internal LLM gateway

In an organisation you wouldn't expose Claude directly. InspectIQ is designed to sit behind a gateway that centralises this: a single audited entry point that enforces what the model may see (tenant-scoped retrieval filters — note the doc_type filter already in store.search), redacts PII before it reaches the model, logs every prompt/response, and keeps the validation gate as the last word. The retrieval + gate layers here are exactly the pieces such a gateway wraps.

Notes

• Sample documents are synthetic (generated by gen_samples.py); no real data.
• claude-opus-4-8 is the default model; override with INSPECTIQ_MODEL.
• Config is centralised in inspectiq/config.py, all env-overridable.