Reference

The dry stuff, all in one place: what the database looks like, what the API exposes, and which environment variables do what. Read How-It-Works first if you want the story; this page is for looking things up.

The tables

Two groups of tables live in the database — the domain data and the system caches.

Domain (written by the pipeline, decided by humans):

• documents — the source PDFs that were ingested.
• candidates — extracted claims awaiting review, each with a mandatory page number and verbatim excerpt.
• commitments — candidates that a human accepted, organised in a parent/child tree (objectives → measures → targets).
• evidence — official pages/documents the finder surfaced, with the matched passage, extracted entities, and a status hint quoted from the text.
• links — the proposed commitment ↔ evidence relationships. Each has a score, a suggested relationship, human-readable reasons, and a review_status that starts at unreviewed no matter what.
• budgets — money figures quoted verbatim from evidence, typed as estimated cost / approved allocation / reported expenditure. Never merged, never invented.
• jobs — the durable background-job queue.
• official_sources — the registry of official pages the evidence search always fetches. It's data, not code — and it also feeds the fetch allowlist.

Caches (the cost-savers, safe to wipe):

• pages — fetched page text by URL; a page is fetched once, ever.
• chunks — chunk embeddings per (page, model). pgvector does the top-k search inside Postgres via match_chunks_hybrid — dense cosine fused with a stemmed tsvector rank (the tsvector column's regconfig is fixed at migration time; PAPER_TRAIL_LANGUAGE must match). If migration 006 isn't applied it falls back to dense-only match_chunks.

When Supabase isn't configured, the same shape exists locally: a SQLite file under data/ plus JSON files for the caches. Tests run entirely on that path.

The API

Everything is REST; interactive docs live at /docs on a running instance. Reads and decisions are synchronous — the two long operations (ingesting a PDF, finding evidence) return 202 with a job object, and you poll GET /api/jobs/{id} until it's done or failed.

Endpoint	What it does
GET /healthz	{ok: true} — is it up
GET /api/meta	which backend, model, and search provider are active, plus review_required
GET /api/documents · GET /api/documents/available	ingested docs · files waiting in the store
POST /api/documents/ingest	job: read a PDF into candidates — the result warns when nothing matched the layout
GET /api/candidates	the review queue (?status= to filter)
POST /api/candidates/{id}/accept · …/reject · POST /api/candidates/bulk	review decisions
GET /api/commitments	confirmed commitments
POST /api/commitments/{id}/find-evidence	job: run the evidence finder
GET /api/links	proposed links (?status=, ?commitment_id=); ?lite=1 drops snippets and budgets so the workspace list stays small
GET /api/links/{id}	one link with the full evidence record — what the detail panel calls when you expand a match
POST /api/links/{id}/decide · POST /api/links/bulk	accept/reject, optionally overriding the relationship
GET /api/trail	the finished record — commitments, accepted evidence, gaps
GET /api/jobs/{id}	job status, progress line, result or error
POST /api/translate	HU→EN machine translation (needs HF_TOKEN, else 503)

Writes need a key, reads don't. When PAPER_TRAIL_REVIEW_KEY is set, every POST that changes the record (review decisions, ingest, find-evidence — but not translate) expects X-Review-Key: <key> and answers 403 without it. The frontend keeps it in the browser's localStorage after you enter it once — the "Review key" dialog in the top bar, shown when GET /api/meta reports review_required. Unset means open access, which is fine for local dev. It's a shared key, not accounts — good enough to stop strangers from editing the record on a public deployment, not a multi-user permission system. For a private UI, NEXT_PUBLIC_REVIEW_KEY bakes it into the bundle instead.

CORS is PAPER_TRAIL_API_ORIGINS, comma-separated.

Configuration

Everything lives in .env (see .env.example). The short list:

Variable	Default	What it controls
SUPABASE_URL / SUPABASE_KEY	unset	set both → Postgres + pgvector + Storage; unset → local SQLite + files
PAPER_TRAIL_EMBED_MODEL	MiniLM (local ONNX)	hashing for offline stubs, jina for hosted, openai:<model> / bge-m3 for an OpenAI-compatible server, or any fastembed model name
PAPER_TRAIL_EMBED_BASE_URL / _API_KEY	unset	endpoint (and optional bearer token) for the openai: embedder, e.g. http://my-box:8081/v1
PAPER_TRAIL_REVIEW_KEY	unset	shared key gating all mutations via X-Review-Key; unset = open
NEXT_PUBLIC_API_URL	http://localhost:8000	web side — where the API lives
NEXT_PUBLIC_REVIEW_KEY	unset	web side — bakes the review key into the bundle; private UIs only
PAPER_TRAIL_RERANKER	auto	auto uses hosted Jina when JINA_API_KEY is set; none disables; a model name runs it locally
PAPER_TRAIL_RERANK_K	20	candidates sent to the reranker
PAPER_TRAIL_SEARCH	ddgs	ddgs = live DuckDuckGo, fixture = offline replay for tests
PAPER_TRAIL_API_ORIGINS	http://localhost:3000	CORS allowlist
PAPER_TRAIL_MODEL_CACHE	data/models	where downloaded models live
PAPER_TRAIL_DB	data/processed/paper_trail.db	SQLite path (local/offline mode)
PAPER_TRAIL_LANGUAGE	hu	language profile for the deterministic parsers — must match the migration-006 tsvector regconfig
PAPER_TRAIL_ALLOWED_DOMAINS	empty	extra evidence domains on top of the registry hosts
PAPER_TRAIL_DEPLOYMENT	empty	badge shown in the UI (e.g. the municipality tracked)
PAPER_TRAIL_BASELINE_YEAR	derived	evidence published earlier is forced to background
PAPER_TRAIL_LLM_BASE_URL / _API_KEY / _MODEL	unset	LLM extractor via any OpenAI-compatible endpoint, instead of the rule parser
PAPER_TRAIL_LLM_WINDOW	4	pages per LLM call; use 1 for small local models
PAPER_TRAIL_STORAGE_BUCKET	source-documents	Supabase Storage bucket for the PDFs
JINA_API_KEY	unset	enables the hosted reranker under auto mode; PAPER_TRAIL_EMBED_MODEL=jina also routes embeddings through it
HF_TOKEN	unset	enables the translation endpoint
PAPER_TRAIL_MT_MODEL	opus-mt-<lang>-en	override the translation model
PAPER_TRAIL_MAX_DOC_CHARS	4,000,000	emergency document bound — truncates and warns