Skip to content

Releases: tonydzi/verbatim-citation-gate

Release list

v0.2.0 — on PyPI, with CI, and honest false-positive numbers

v0.2.0 — on PyPI, with CI, and honest false-positive numbers Latest
Latest

Choose a tag to compare

View all tags
@tonydzi tonydzi released this 25 Aug 12:49
v0.2.0
8205953

the release where this became a real package:

  • packaged, with CI — first pyproject.toml, first pipeline
  • unicode-script normalization for citations, with a PEP 561 type marker
  • false positives 42.5% → 17.2% — repaired transit damage in normalization, measured by the new bench (detection rate + FP rate)
  • perf: normalized citation text is cached
  • deal-hygiene pack: SECURITY (threat model), GOVERNANCE (succession, inbound=outbound MIT)
  • docs: demo GIF + 30-second quickstart from a real run; 40 dead links fixed after the account rename

credit where due: Guan Tong fixed three of the shipped issues (we found them, he fixed them — the README says exactly that).

cut by the weekly release pass, first run.

correction, 2026-08-29. the first bullet originally read "on PyPI, with CI — first packaging, first pipeline". that was wrong: this package is not on PyPI. the trusted-publishing workflow landed on 26 Aug, a day after this release was cut, so nothing was ever uploaded — and checking now, PyPI returns 404 for the name. the README always said "not on PyPI yet"; only this note claimed otherwise. install from the repository until that changes:

pip install "git+https://github.com/tonydzi/verbatim-citation-gate"

corrected by the weekly release pass, run #2. we log these rather than editing quietly.

Assets 2
Loading

v0.1.0 — the gate goes first

v0.1.0 — the gate goes first

Choose a tag to compare

View all tags
@tonydzi tonydzi released this 04 Aug 19:28
v0.1.0
096cedd

Catch fabricated RAG citations before they reach the user. First tagged release — the code has been in the open since 24 July; from now on it has a version you can pin.

What it does

Two stages, and the cheap one goes first:

  1. The verbatim gate (gate.py) — pure re + substring matching over a normalized form. Zero dependencies, zero model calls. It answers the prior question a judge model never asks: does this quote physically exist in the document it is attributed to? Fabrications, frankenquotes (every word real, the sentence never written) and misattributed real quotes are rejected here, and misattributed stays a separate verdict from not_found because they need different fixes upstream.
  2. The skeptical judge (judge.py) — for quotes that do exist, a burden-of-proof prompt: the verdict starts at unsupported, outside knowledge is inadmissible, and support must match population, direction, magnitude and certainty or it caps at partial. Framework-agnostic — it plugs into whatever LLM your pipeline already speaks.

14 tests, each one a named failure mode rather than a coverage number — including test_audit_fabrication_never_calls_judge, which is the whole economic argument for the ordering.

In this release

  • verbatim_citation_gate.gate — audit_citation, normalization, the three rejection verdicts.
  • verbatim_citation_gate.judge — the burden-of-proof prompt and verdict parsing.
  • Docs site: https://palo-alto-ai-research-lab.github.io/verbatim-citation-gate/
  • New here: CI now runs the suite on every push and weekly, on Linux/macOS/Windows × Python 3.9/3.11/3.13. Until today the tests existed and nothing ran them.

Known limits — read these before you rely on it

They are open issues, not footnotes:

  • #1 — non-Latin scripts. normalize() keeps only [a-z0-9%.], so in Cyrillic/CJK the comparison degenerates to digits and a fabricated quote can come back found. In those scripts the tool does not yet do its job. This is the top of the queue.
  • #4 — a one-word quote returns found; there is no minimum-evidence rule yet.
  • #2 — the corpus is re-normalized on every call (2.4s per miss on 300 docs).
  • #3 — no py.typed, so downstream type checkers see nothing.
  • #8 — not on PyPI yet; install from git.

Install

pip install "git+https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate@v0.1.0"

What's next

The roadmap board is public and every open row is a claimable issue: https://github.com/users/Palo-Alto-AI-Research-Lab/projects/1 — issues labelled accepted are scoped, free to take, nobody is on them. Comment "claiming this" and it is yours for 7 days. You keep the copyright to your code; no CLA, ever.

From here on, every noticeable change ships as a new minor release — so this feed, not the commit graph, is the honest record of how mature this is.

Full Changelog: https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate/commits/v0.1.0

Loading