A runtime grounding gate for LLM outputs - catch hallucinated claims before they ship, with exact citations.
Spell-check, but for facts. Give anchor an AI-generated answer and the source documents it should have drawn from: it splits the answer into atomic claims, verifies each against the sources, attaches an inline citation to every grounded claim (pointing to the exact doc_id:start-end), and flags the ones that aren't supported. 0.93 F1 on the FEVER public benchmark, fully local, no API.
Live demo: https://anchor-production-44a7.up.railway.app/
Two-pane evidence viewer: the verified answer on the left (blue = cited, red = needs review); click a citation number to jump to its source chunk on the right, which flashes.
pip install -e . # anchor library + NLI corefrom anchor import Source, annotate, render_markdown
sources = [Source(id="policy", text="...the document the AI should have used...")]
answer = "...the AI-generated answer..."
result = annotate(answer, sources) # verifies each claim + cites
print(render_markdown(result))
# result.text -> answer with inline [n] citations and warning flags
# result.citations -> [{n, doc_id, start, end, snippet}, ...]
# result.flags -> [claims not found in any source]See examples/usage.py for a runnable version.
anchor is tested on two benchmarks:
| benchmark | what it tests | accuracy | F1 |
|---|---|---|---|
| FEVER (public dataset) | real fact-checking: does a claim follow from its evidence? | 0.93 | 0.93 |
| Stress set (31 edge cases) | paraphrases, wrong numbers, entity swaps, negations, compound sentences | 0.94 | 0.95 |
FEVER is the standard academic benchmark for fact-checking. The stress set tests edge cases that matter in production - hallucinations, number swaps, "do NOT" vs "must", and sentences with mixed right/wrong facts.
Reproduce: python benchmarks/eval.py (stress set) - python benchmarks/bench_fever.py (FEVER)
python examples/usage.py # how to use anchor in your code
python examples/cite.py # verify an AI answer against a source doc
python benchmarks/eval.py # the stress-test eval (precision / recall / latency)
python benchmarks/bench_fever.py # the FEVER public benchmark
python benchmarks/bench.py # benchmark on benchmarks/bench.json - or YOUR own dataset
python benchmarks/distractors.py # retrieval robustness vs decoy docs (lexical vs semantic)pip install -e .[app] # adds fastapi + uvicorn
cd playground && uvicorn app:app --reload # http://localhost:8000Already live: https://anchor-production-44a7.up.railway.app/
Deploy your own to any container PaaS - see Dockerfile / render.yaml. The NLI model needs ~2 GB RAM; for a low-traffic demo, enable scale-to-zero so you don't pay while idle.
Telling an LLM "only answer from the sources" helps - but it doesn't guarantee or prove anything. The model still invents, and you can't show an auditor "this was grounded." anchor is the independent check plus the citation/audit trail.
- vs RAGAS - RAGAS grades your RAG pipeline offline over a test set.
anchorchecks a live output at runtime, per claim, with citations. - vs NeMo Guardrails - NeMo is a broad behavioral-rails framework (Colang, an LLM call per rail).
anchoris a focused, cheap-first grounding layer: local models, an audit trail, and the LLM judge only on the residue.
anchor verifies each claim against your source documents through a layered pipeline.
NLI (Natural Language Inference) judges the logical relationship between a source and a claim - entailment, contradiction, or neutral. The model uses cross-attention to compare them word-by-word, which is why it catches contradictions that similarity models miss.
source sentence (retrieved) -> PREMISE
AI's claim -> HYPOTHESIS
|
NLI model (DeBERTa)
|
+-------------+-------------+
entailment contradiction neutral
| | |
grounded flagged -> confirmers
(cite source) (lex + negation gate)
The three-way signal drives anchor's routing:
- Entailment -> grounded. Cite the source. Done.
- Contradiction -> flagged. The source disproves the claim. Reliable catch.
- Neutral -> the model is unsure. The confirmers decide: lexical check (is it a paraphrase?) + negation gate (is there a "not" flip?). If neither fires -> keep flagged (conservative).
| layer | what it does | cost |
|---|---|---|
| semantic retrieval | finds the right source by meaning (embeddings) | local, ~15 ms |
| NLI (DeBERTa) | entailment / contradiction / neutral judgment | local, ~40 ms |
| lex confirmer | clears paraphrase false-alarms (high overlap = paraphrase) | pure Python, free |
| negation gate | keeps "do not need vs must" flagged | pure Python, free |
| decomposition | splits compound sentences ("A and B") into atomic claims | pure Python, free |
| LLM judge (optional) | clears the last false alarms (precision confirmer) | your key, ~c per call |
Semantic retrieval is the default (handles vague phrasings like "10 to 3"); lexical retrieval is available as a faster fallback. When a sentence has multiple facts joined by "and", anchor splits them and checks each separately - so "tokens expire after 24h and the pro plan costs $99" becomes two independent checks, each with its own citation or flag. Every result is logged to a permanent audit trail.
anchor/ # the library package (importable: from anchor import ...)
examples/ # usage + citation demos
benchmarks/ # eval, bench, FEVER runner, sample cases
playground/ # FastAPI app + UI (separate [app] deps, thin layer over the library)
docs/ # UI screenshots
pyproject.toml # installable; deps + [app] extra
Dockerfile / render.yaml # one-command container deploy
anchor is the verification layer, not a retriever - point it at your existing corpus (RAG retriever, Confluence, SharePoint, vector DB) and it verifies + cites.
- Citation layer - inline citations with exact
doc:start-end - Local NLI core (DeBERTa) + lexical & negation confirmers
- Public benchmark on FEVER (0.93 F1)
- Provider-agnostic + local judge (Ollama) for air-gapped deployments
- Human-in-the-loop gate (block / strip / flag-for-review)
MIT