docrag — layered local RAG (he/en)

A faithful, lightweight implementation of a layered knowledge-retrieval architecture (classify → extract → SQL/vector behind a router; see §7–§13 of the design notes). 100% local and offline via Ollama — no servers, no cloud keys, no Postgres/Qdrant. Just Python stdlib + numpy + SQLite.

Setup

./setup.sh          # python deps + pulls bge-m3 + the extraction LLM into Ollama

The idea (why it's layered, not "just RAG")

Most questions about financial/business documents are not semantic — they're about metadata: "total paid to supplier X in Q3", "invoices over ₪10,000". Vector search is wrong and harmful for those (§4, §7). So docrag runs two layers behind one router:

document ─► [classify] ─► [extract fields] ─┬─► SQL layer   (one table per type)  ── exact sums/filters
                                            └─► vector layer (bge-m3 + hybrid)     ── meaning / open questions
                                                                                        │
                                              question ─► [router] ─────────────────────┘  picks the path

• Classify → Extract (extract.py): an LLM tags each document's type and pulls a fixed schema of fields out of its text. Metadata is created from content, not read off the file (§8).
• SQL layer (db.py, one field-table per type from schemas.py): numbers, names, dates → exact filters and SUMs, 100% accurate, no hallucination (§7, §11.4).
• Vector layer (search.py): bge-m3 embeddings + hybrid dense⊕BM25 retrieval (RRF), then a token-budget packer so a small context window never overflows (§4, §13, §11.13).
• Router (router.py): a fast intent check sends each question to SQL / hybrid / overview (§7).

Setup (manual, what setup.sh automates)

ollama pull bge-m3            # multilingual embeddings (he/en), 1024-dim  — REQUIRED
ollama pull qwen2.5-coder:7b  # extraction/SQL/phrasing LLM (or set DOCRAG_LLM)
pip install numpy

Run from the directory containing the docrag/ package (i.e. the repo root).

Use

python3 -m docrag ingest /path/to/docs              # a file or a whole folder
python3 -m docrag ingest invoice.pdf --type invoice # skip the classifier
python3 -m docrag stats
python3 -m docrag ask "כמה שילמנו לאלקטרה ב-2026?"   # → SQL path (exact)
python3 -m docrag ask "אילו חשבוניות קשורות למיזוג אוויר?"  # → hybrid path
python3 -m docrag ask                               # interactive REPL
python3 -m docrag sql "SELECT supplier_name, SUM(total) FROM invoice GROUP BY 1"

Inputs & OCR (the §7 step-0 risk)

.txt / .md are read directly; text-based .pdf via pdftotext if installed. Scanned / image documents need OCR first — Hebrew OCR is failure-point #1 (§7) and no engine is hardcoded. Either pre-convert to .txt, or pass an OCR command:

python3 -m docrag ingest ./scans --ocr-cmd "tesseract {file} stdout -l heb+eng"

Files with no extractable text are skipped and reported — never silently treated as empty.

Adding a document type (§10)

Add an entry to SCHEMAS in schemas.py (fields + a one-line description for the classifier). That's it — the table, extraction, routing and search all pick it up. No new pipeline (§10).

Config (env vars, see config.py)

DOCRAG_DB, DOCRAG_LLM, DOCRAG_EMBED (don't change after ingesting — re-index needed, §5), DOCRAG_CTX_BUDGET (token budget for the answer context, §13), DOCRAG_FINAL_K.

Honest limits (vs the full reference stack)

• Brute-force cosine over chunk vectors (no ANN). Instant to tens of thousands of chunks; move to Qdrant/pgvector beyond that (§7 production note).
• Sparse half is BM25, not bge-m3's native sparse vectors (those need FlagEmbedding, not the Ollama API). Equivalent for exact-term/number matching.
• No reranker model yet (§4 mentions BGE-reranker-v2) — RRF fusion covers most of the gain; a rerank pass is the natural next add (open point §12.9).
• No human-in-the-loop / abstention UI (§4 control layer) — answers are grounded and cited, but high-stakes numeric sign-off (§7 step 5) is left to the caller.