Verity v1.0

Verity v1.0

First stable release of Verity. It reads invoices and receipts, checks each one against a written set of accounts-payable controls and tells you which ones are fine and which ones need a person. It does not pay anything, and it does not approve anything. It reads, it checks and it holds up its hand.

Every finding carries the exact wording of the policy it fired under, quoted in full rather than paraphrased. Nothing is held because a model "seemed unsure." Every person-decision is recorded against a name, overrides are flagged explicitly and the settings in force at audit time are stored permanently with each record.

The fourteen controls

ID	Control	What it looks for	Outcome
P-001	Supporting documentation required	No amount could be read at all, or a sizeable amount with neither a supplier name nor a date anywhere on the document	Blocked
P-002	Duplicate payment prevention	Same supplier, same amount, same date as a document already on file	Blocked
P-003	Supplier identification	No legible supplier or merchant name on the document	Review
P-004	Transaction date required	No date for when the goods or services were supplied	Review
P-005	Late submission	Document is older than the submission window (90 days by default)	Review
P-006	Future-dated documents	Document is dated after today, usually a transposed or fabricated date	Blocked
P-007	Arithmetic integrity	The line items don't reconcile to the stated total or subtotal, beyond a small rounding tolerance	Blocked
P-008	Authorised signature	No visible signature or stamp on a document above the signature threshold	Review
P-009	Delegated approval limit	Amount exceeds what can be approved without escalation. Verity routes this upward; it does not approve it	Blocked
P-010	Round-amount anomaly	Amount is suspiciously exact (a round hundred or more). Not proof of anything on its own, just worth a second look	Review
P-011	Threshold avoidance by splitting	Two or more invoices from the same supplier, close together in time, each sitting just under the approval limit	Blocked
P-012	Approved supplier list	Supplier is not on your approved list. Only active once your team has actually configured a list	Blocked
P-013	Low-confidence extraction	Verity could not read the document with enough confidence to trust its own answer	Review
P-014	Intake rejection	The file itself couldn't be opened: corrupt, blank, password-protected, or an unsupported format	Blocked

The rule: anything serious fires, the document is blocked and the payment is held until a named person decides. Something less serious with nothing serious means review. Nothing flagged means it passes automatically.

The defaults it ships with

All of these are settings your finance team owns; no code changes needed to adjust them.

Setting	Default	What it governs
Receipt required above	75	Below this, a missing receipt isn't flagged. Above it, a document with no supplier and no date trips P-001
Signature required above	2,500	Above this amount, a document needs a visible signature or stamp (P-008)
Approval limit	10,000	Above this, an expense needs sign-off at the next level (P-009) and it's the reference point for the splitting check (P-011)
Arithmetic tolerance	0.05	How far the line items can miss the stated total before it counts as a mismatch (P-007)
Maximum explainable mark-up	35%	How far above the line items a total can sit before the gap stops looking like tax or a service charge (P-007)
Submission window	90 days	Documents older than this are flagged as late (P-005)
Splitting window	14 days	How close together two near-limit invoices from the same supplier need to be before they're reviewed together (P-011)
Splitting proximity	90% of the approval limit	How close to the limit an invoice has to sit to count as "just under" for the splitting check
Minimum reading confidence	60%	Below this, the document goes to a person regardless of what else it shows (P-013)
Approved supplier list	Off (empty by default)	Switches on only once your team supplies a list of approved suppliers (P-012)

Working at scale and getting answers out

• Folder audit: point Verity at a folder of invoices and receipts for a period; it works through every document and one broken file does not stop the batch. Outputs a breakdown of outcomes by control plus a CSV exception report ready for Excel.
• Review screen: a simple queue with three buttons (approve, reject, ask for more information). A decision with no name typed against it is refused outright.
• Exception report: a CSV of everything currently held, for people who will never open Verity itself.
• Audit pack: one document's full record assembled in the order an external auditor would read it: verdict, every firing control with its quoted policy text, the fields read, the settings in force and the complete reviewer history.

Calling Verity from another agent

Two integration paths and neither moves any policy logic off the server. Both go through POST /audit (content_base64 for the file itself, fields for a reading the caller already did). A pass means no control objected; it is never an instruction to pay anything.

• MCP: one stdio server (uv run verity-mcp) covers Claude, Gemini CLI, Cursor and Codex, exposing a single tool, audit_document.
• OpenAPI: uv run scripts/export_openapi.py --out openapi.json --server https://verity.example.com produces a schema for ChatGPT Custom GPT Actions. Hosting and authentication are yours.

What the testing shows

• 19 hand-written policy cases: perfect match on all of them. This checks the logic does what was designed, no more.
• 500 invoices with planted problems, labelled by someone other than the control author: Verity caught all 30 of the 30 planted problems its controls cover. One clean document was flagged by mistake (an exactly round amount, which is P-010 doing its job). None of the other 460 clean invoices tripped a fraud control.
• 10 of the 40 planted problems Verity cannot catch at all: three quoting nonexistent purchase orders and seven submitted at odd hours. There is no purchase-order data and no submission-time signal, so no control covers either. If PO matching is central to your spend control, Verity does not do it in this release.
• Reading is the weak step and the numbers say so plainly. On 25 rendered pages the controls scored near-perfectly when handed correct numbers, but reading those pages live, Verity flagged all 25 including all 22 legitimate ones and the confidence score proved unreliable (average reported confidence 95% against 8% of pages read correctly). Do not treat a high-confidence read as a checked one and treat the 60% P-013 threshold as an untuned default.
• Extraction against a public benchmark of 25 scanned receipts: the total was right on 100% of claimed reads and found on 96% of those present. Line items: 67% correct of claimed, 71% found. Supplier name and transaction date come from a weaker step and have no measured accuracy figure; expect to correct them.
• Policy search: the right policy comes back first about 4 times in 10 (hit@1 0.433) and in the top three about 7 times in 10 (hit@3 0.733, MRR 0.561). Search never decides anything; it only affects how easily a reviewer can look up why something was flagged. No RAGAS figures exist; those metrics have never been run.
• Optional citation check (off by default): asks Claude whether the quoted policy passage genuinely supports a finding. It cannot change a verdict or release a payment. Tested against fixed replies only; no accuracy figure at volume.

Nobody has run this against a real company's ledger yet. Everything above comes from test documents and planted examples. Before relying on Verity for an actual period close, run it alongside your existing process for at least one full period and compare the two.

Quickstart

uv sync
cp .env.example .env
docker compose up -d db
uv run alembic upgrade head
uv run uvicorn verity.api.main:app --reload

Audit a folder:

uv run scripts/audit_folder.py ./inbox --out exceptions.csv

Add --save to persist runs; preview mode writes only the CSV and the duplicate-payment (P-002) and splitting (P-011) controls cannot fire without saved history.

Key endpoints: POST /documents, POST /audit, GET /runs, GET /runs/{run_id}, GET /runs/{run_id}/pack, POST /runs/{run_id}/review, GET /reports/exceptions.csv, GET /policies.

Known limitations in v1.0

• Scanned documents only. Born-digital PDFs with no picture of a page inside them are reported as unreadable rather than guessed at.
• Reading accuracy is the bottleneck; line-item extraction in particular is rough (roughly a third of reported lines wrong or spurious).
• The confidence score is not currently a reliable signal of correctness.
• The round-amount check (P-010) is not currency-aware; treat it as advisory if you transact in more than one currency.
• The duplicate and splitting checks look back six months and need saved history to fire.
• No purchase-order matching and no behavioural signals such as submission time.
• The MCP and OpenAPI integrations are verified against the standards but have never been run inside Claude, Gemini CLI, Cursor, Codex, or ChatGPT.
• Supplier name and date accuracy are unmeasured; they come from a weaker reading step.

Verification

Deterministic evaluations, no key and no GPU, run in CI on every push and pull request:

uv run -m verity.eval.policy_eval        # 14 controls, 19 hand-written cases
uv run -m verity.eval.fraud_eval --corpus sample/verity_sample_invoices_m_500.csv
uv run -m verity.eval.trajectory_eval    # did the agent take the right route
uv run -m verity.eval.rag_eval --corpus sample/verity_sample_invoices_m_500.csv
uv run -m verity.eval.extraction_eval --limit 25

Tests and lint: uv run pytest, uv run ruff check src tests scripts.

Full detail on all of the above is in the README.