Repository navigation
Contributing
Want to change Paper Trail? Great. This page covers the few rules every change has to respect, the everyday development loop, how a change makes its way into main, and the house style for code and docs. You don't need to be an expert to contribute code, docs or labels, but you do need to care about the first section.
Paper Trail exists to be honest about what a city didn't do, so a change that weakens any of the following won't get merged.
The most important rule is that a human always has the last word. The pipeline suggests things, but nothing enters the record until a person accepts it. That means no auto-acceptance, ever, and the worker's database login has to stay unable to write decisions. A person can only judge what they can check, though, so every claim has to be traceable too: a page and a verbatim quote for a promise, a URL and a passage for evidence. If it can't be quoted, it doesn't count.
The same carefulness applies to money and status. Budget states are never merged or summed, currencies are never mixed and numbers are never made up. Status comes from the text, so "completed" means a source said so, not that a deadline passed. Entities, money, status and audit cues are read from the matched passage, never the whole document. And anything a machine wrote, like notes and translations, is labelled as machine-written and never decides anything.
If your idea sits awkwardly with one of these, open an issue before writing code. Accountability-Rules explains why each rule exists.
You set things up once (Getting-Started has the longer version):
make setup
docker run -d --name paper-trail-test -p 127.0.0.1:55433:5432 \
-e POSTGRES_USER=paper_trail -e POSTGRES_PASSWORD=test-only \
-e POSTGRES_DB=paper_trail pgvector/pgvector:pg16
export TEST_DATABASE_URL=postgresql://paper_trail:test-only@localhost:55433/paper_trail
uvx pre-commit install # run the hooks on every commitAfter that, the loop is mostly about running the smallest useful check first. Start with the narrowest test file that covers your change, for example uv run pytest tests/ml/test_extraction.py -q, and run make test for the whole suite once that passes. Before you call anything done, run make check, which runs the tests and then the pre-commit hooks (ruff lint with fixes, plus a check that uv.lock matches pyproject.toml). make fmt lets the hooks fix whatever they can, and make dev gets you the app at http://localhost:8000 so you can try things in the browser.
The tests only need one thing from you, which is TEST_DATABASE_URL, an owner login on a throwaway pgvector server. The suite migrates a template database once per run and gives every test its own fresh copy, so tests never see each other's data. They don't need API keys or the network either, because models are stubbed and search replays fixtures. If TEST_DATABASE_URL isn't set, the run stops right away and tells you to set it, and a test that only works with the real strategy PDF skips itself when the file isn't there.
Warning
Never point TEST_DATABASE_URL at a database you care about. The suite creates and drops databases on that server.
CI (.github/workflows/ci.yml) runs the same two things on every pull request and every push to main: the pre-commit hooks, and uv run pytest -q against a pgvector service container. So if make check passes locally, CI should pass too.
For anything bigger than a typo, open an issue first, since agreeing on a direction is much cheaper before there's code. Then do one task per branch and one pull request per branch, branching from main and never committing to it directly. If one change builds on another, stack the second branch on the first and mention that in the PR.
Keep the diff focused on the task, with no unrelated refactors or drive-by renames. A bug fix should come with a test that failed before the fix, and a feature should extend the existing test files where it fits (tests/ mirrors src/). Database changes are always a new numbered file in db/sql/ plus a new Alembic revision in db/alembic/versions/, and released files never get edited (Database).
If you add a dependency, run uv lock and commit the lock file. A well-maintained library that's already in pyproject.toml, or a small established one, beats hand-rolled code, but don't pull in a dependency for something a few lines can do. Along the same lines, look for an existing helper in services/, repositories/, ml/ or ui/ before writing a parallel one. A few things are done "the hard way" on purpose (money is parsed with regular expressions in ml/entities.py because it has to be exact), so match those instead of "fixing" them.
When behaviour changes, update this wiki as part of the same piece of work (see below). Fill in the PR template with what you changed and why, the honesty checklist and how you tested it. A maintainer merges once there's an approving review and CI is green.
Match the file you're in. Ruff enforces the basics (line length 100, sorted imports, pyupgrade). Code, identifiers and comments are in English, and Hungarian belongs in language profiles, test fixtures and domain strings.
Comments are short notes on why something is the way it is, or on a gotcha, written the way a person would jot them down: lowercase, no trailing period, casual is fine, and skip the obvious. No "This function returns..." ceremony. Something like this:
# float, the sdk asks for base64 by default and not every server does it
resp = client.embeddings.create(model=model, input=batch, encoding_format="float")and not like this:
# This line calls the embeddings endpoint and returns the embeddings.The repository doesn't keep any docs beyond a short README and the GitHub community files, so everything else lives in this wiki. When behaviour changes, update the page that describes it. Deployment data (sources, domains, streets) doesn't go in the repository either. It belongs in the database or an untracked .env.
To edit the wiki locally, clone it like any other repo:
git clone https://github.com/unnobatroo/paper-trail.wiki.gitThe pages are grouped the Diátaxis way, into one tutorial, a few how-to guides, explanation pages and reference pages. Write them in plain, casual English, the way you'd explain the project to a friend who's sharp but new to it. Each paragraph should follow from the one before it rather than being a pile of bullet points. Define a term the first time you use it, or link to Concepts, and use a real example whenever you can. Tables are fine for things people look up, like settings, endpoints and stages. For pictures, use real screenshots of the workspace (they live in images/ in the wiki repo) and Mermaid diagrams for flows and structures, which GitHub renders. Please don't draw boxes with ASCII characters.
The easiest way to help is to review and label suggestions in a running workspace, because every label makes the model better (Models, Reviewing). If you'd rather write code, organisation, place and money extraction in ml/entities.py and ml/ner.py can always be improved, as long as the change comes with tests. If you know another country's public documents, you could add a language profile for it: copy tests/fixtures/profiles/en.json, translate the cue lists and patterns, and load it with maintenance profile import. And if anything on this wiki is wrong or confusing, fixing it is a perfectly good first contribution.
By contributing, you agree that your work is licensed under GPL-3.0. When you're ready to dig into the code, Project-Layout shows where everything lives, and Reference lists every setting and command.
Repository · GPL-3.0 · Nothing reaches the record without a human click · Something wrong here? Fix this wiki
Tutorial
How-to guides
Explanation
Reference