The local document stripper. Drop in a contract, a client file, a case document — get back a blanked copy with every name, company, brand, address, phone, email and ID number replaced by placeholders, plus a separate key file mapping each placeholder to what it replaced. The blanked copy is safe to paste into a cloud AI or send onward; the key never leaves your side.
Nothing leaves your machine. The model is a stock, hash-verifiable 3 GB Gemma running locally; this codebase makes no network call beyond localhost — no telemetry, no analytics, nothing. One grep proves it.
What this repository is, exactly. The engine, the deterministic rails, the command-line tool, and the entire benchmark — corpus, hand-built ground truth, scoring scripts, methodology, and every named miss. It runs today from a clone. The desktop application is not released yet: no installers, no binaries, no signed builds. That order is deliberate — the part that can be independently checked ships before the part that can only be trusted. You bring the model; it is Google's, ~3 GB, and not ours to redistribute.
simpler.red — simplifying red tape.
The headline is out-of-sample, and it leads because that is the only honest place to
start (bench/oos-walkforward/): a walk-forward protocol — freeze the engine, acquire
whole public documents it has never seen, build an exhaustive adversarially-audited ground
truth BEFORE the engine runs once, publish the number as-is; fixes from one round are only
ever proven by the next round's fresh documents. Five rounds in: 84.62 → 95.38 → 88.46 →
93.76 → 94.97% — the v1.0 headline (round 5, pre-registered publish-as-drawn) is
94.97% of entities and 98.2% of printed occurrences on 18 hostile unseen documents,
and the release engine scores 95.82% across the full 118-document battery
(BENCHMARK.md has the one-engine table with every tier's saturation labeled).
The measured envelope, stated plainly: those numbers hold for the genres the battery
contains — SEC filings and exhibits of every kind, press releases, earnings transcripts,
US and Singapore court judgments, US government documents, UK RNS, and OCR-degraded scans.
On genres deliberately outside that history the measured band is lower: round 3 went wide
on purpose and read 88.46% overall with frontier genres near 78%
(bench/oos-walkforward/round-3/RESULT.md). A document type this engine has never been
measured on should be expected to land in that band, not at the headline — that is what
the review table is for, and new-genre misses are exactly what the walk-forward protocol
eats (KNOWN-ISSUES.md names every open one).
Baselines on the same metric: Microsoft Presidio as shipped, 43.10% — measured on 90 of
these documents (walk-forward rounds 1–4), never on round 5 or the full battery; our
engine on the TAB legal benchmark's must-mask tier (their ground truth, not ours),
96.12%. Annotation quality is itself measured: blind second-pass agreement 0.925 — the
medical gold standard's human-human bar is 0.927. Every miss is named and family-sorted in
each round's RESULT.md; every surprising number carries its discovery path in
paper/FINDINGS.md. In-sample and out-of-sample are never blended. That is the difference
between a benchmark you grade yourself on and one that can hurt you.
Verify the claims yourself in seconds, no model needed: npm run audit recomputes
every offline-verifiable published number from the committed artifacts — the per-doc rows,
the aggregates, and a full from-scratch rescore of rounds 4–5 against the committed masked
outputs — and exits red on any drift. To score YOUR tool on the same documents and metric:
LEADERBOARD.md.
Two engines run in series. Engine A (the wide sweep) reads the original: a chunked extraction pass plus deterministic name rails, with a small local model doing only judgment — every span it nominates must occur verbatim in the text or it is refused. Engine B (the specialist) reads only the already-redacted output and hunts the classes that history says get left behind: a zero-model regex battery (phones in any format, ID families, URLs, tickers, address fragments) plus a narrowly-scoped second model sweep. One entity table, kept in code — find a name once and every copy in the document is masked.
The engine also carries an internal regression floor — 8 real SEC filings
(hand-labeled, sources in bench/corpus/PROVENANCE.md) plus 44 synthetic adversarial
documents (generators included, ground truth by construction), in-sample by definition and
never the generalization claim:
| configuration | recall on identity mentions (in-sample ratchet) |
|---|---|
| Engine A alone, as first shipped | 84.8–93.7% |
| + the structural rails | 98.0–100% |
| A + B, the shipped default | 100.00% — 765/765 core + 226/226 shorthand |
npm run bench:all re-runs that floor and exits red on any core leak: a measured miss
never returns. It is the ratchet the walk-forward rounds climb on, not evidence of
generalization — the rounds above are.
Large documents are measured, not assumed — and measuring found a bug, which is the
point: two real SEC annual reports run as a growing ladder to 79,000 words
(bench/scale/) — throughput ~0.8 min per 1,000 words on the CPU floor, the whole 79k
hand-labeled exhaustively (262 identity entities, 2,102 occurrences). First measurement:
95.80%, plus a real defect (junk fragments passing verbatim verification shredded the
output at XBRL scale). The fix — a footprint floor + boundary-guarded masking — re-measured
to 98.47% with a clean output, before/after published unedited in SCALE.md. A full 10-K
≈ 31 minutes; drop the archive in overnight.
The leak history that produced those rails is preserved unedited in
bench/hardening-2026-07-19/REPORT.md — including what the early versions missed and the
one-line fixes each miss became. npm run bench:all re-runs everything and exits red on any
core leak; that floor is the law for every engine change.
What 100% means here, precisely: perfect recall on every document and planted trap in this repo's benchmark, enforced as a permanent regression floor. It is NOT a proof about arbitrary documents — no system can honestly make that claim. That is why the final step is always human: you review the entity table before the stripped file goes anywhere, and an incomplete run writes nothing at all.
And what stripping does not claim: the blanked copy deliberately preserves dates,
amounts, places, and business facts — that is the point (a contract with its dates masked
is unusable, and the profile is what you want the cloud AI to reason about). It follows
that a sufficiently distinctive profile can sometimes be re-identified from preserved
context alone, by exactly the kind of model the output is destined for. The claim is that
identities are stripped, never handed over — not that inference is impossible. The scope
boundary is measured, not hand-waved: our scores on TAB's quasi-identifier tiers
(bench/baselines/tab/TAB.md) are published for precisely this reason. Where inference
risk matters, review the table with that lens — and consider whether context lines
(a distinctive market position, a unique address pattern) belong in the strip too.
- Get the model (once):
gemma-4-E2B_q4_0-it.gguf— stock Google QAT weights, from Google's official repo on Hugging Face (requires an HF account and accepting Google's Gemma terms — seeTHIRD_PARTY_NOTICES.md). Then verify it —npm run model:verify -- <path-to-gguf>checks the pinned sha256 (3646b4c1…455e6fd, 3,349,514,112 bytes). Wherever the file came from, if the hash matches, you have the stock weights. - Start the local server — llama.cpp's
llama-server, from llama.cpp releases. One line, so it pastes the same way into bash, zsh, PowerShell and cmd:(llama-server --model gemma-4-E2B_q4_0-it.gguf --port 49399 --host 127.0.0.1 --ctx-size 8192 --parallel 1 --jinja --chat-template-kwargs "{\"enable_thinking\":false}"--jinja+enable_thinking:falseare load-bearing — grammar enforcement depends on them. The numbers in this repo were measured on the CPU floor, adding--device none --fit off --n-gpu-layers 0; a GPU will be faster but is a different measured configuration.) - Strip a document:
Out come
node bin/red.mjs contract.txt # full strip (two engines, the measured default) node bin/red.mjs contract.txt --mode quick # rails-only tier, ~3x faster, single-layer guaranteecontract.stripped.txtandcontract.entities.json. Review the table; never ship the two files together.
There is nothing to install. Requires Node ≥ 20 and nothing else — the engine, the rails,
the scorer and every test import node built-ins only, so on a fresh clone both of these run
immediately, with no npm install, no model and no network:
npm run verify # the gate: 70 tests, every measured lesson pinned as a failing-loudly test
npm run audit # recomputes EVERY published number from committed artifacts; red on drift
If you only run one thing, run npm run audit — it re-derives the per-document rows, the
aggregates, a from-scratch rescore of rounds 4 and 5 against the committed masked outputs, and
the utility verdict count, then exits non-zero if any of it disagrees with what this repo
claims. That is the whole trust argument in one command, and it does not need our cooperation.
npm run bench:all runs the live in-sample ratchet (52 documents) against your local model. Fresh clones:
node bench/oos-walkforward/fetch-sg.mjs first rebuilds, sha-verified, the four Singapore
judgments this repo deliberately does not redistribute (see each round's PROVENANCE.md).
| what | where |
|---|---|
| The engine (Engine A: extraction + name rails + classify) | lib/anonymize.mjs |
| The structural battery + junk flagging | lib/rails.mjs |
| Engine B (strict-serial specialist) + the shipped pipeline | lib/engineB.mjs |
| Engine B3 (surface-shape suspicion scorer) | lib/suspicion.mjs |
| The CLI | bin/red.mjs |
| Benchmarks: corpus, stress suites, generators, evidence | bench/ |
| Every open miss, named, with its fix-set queue | KNOWN-ISSUES.md |
| Why the engine is built this way — every law with the measurement that taught it | paper/FINDINGS.md |
| The full battery record and the protocol behind it | BENCHMARK.md · bench/oos-walkforward/PROTOCOL.md |
| Any tool, this battery, one metric — submit a row | LEADERBOARD.md |
| The offline recomputation of every published number | bench/audit-claims.mjs |
| The gate: every measured lesson pinned as a failing-loudly test | test/ |
| What the model weights and corpus are licensed under | THIRD_PARTY_NOTICES.md |
This repository stays the general-purpose tool, and stays feature-frozen. What is in development is a set of separate products — variants tuned for legal, finance and medical documents — each with its own corpus, its own rails and its own published benchmark. That separation is the design, not a technicality: a small local model cannot hold every vertical's vocabulary at once without paying for it in false positives, so precision comes from which rails are loaded rather than from one engine trying to know everything.
Two things follow, and they are the honest version:
- No dates. Nothing here is a commitment, and a vertical ships when it has a measured benchmark to publish, on the same walk-forward protocol as this one.
- No measurement is claimed for any of them yet. In particular this battery contains no clinical documents at all — medicine has i2b2, and a medical variant will need its own ground truth, its own annotator agreement and its own leak ledger before it says anything.
For this engine, the open work is in KNOWN-ISSUES.md — the over-nomination bracket is the
largest item, and fix set 5 is queued for a sixth round of fresh documents.
support@simpler.asia reaches the maintainer. For anything that is a leak in the privacy
sense — a way unredacted content or the key can escape — use the private channel in
SECURITY.md first, not email and not a public issue.
Code: Apache-2.0 (© 2026 Simpler AI). Model weights: Google's Gemma Terms of Use —
not ours to relicense; see THIRD_PARTY_NOTICES.md. Born 2026-07-19 as a fork of the
simpler-harness privacy membrane's anonymizer lane; the measurement history that shaped every
rule in this engine is part of the repo on purpose.
This tool is feature-complete by design. It does one thing, measured. What's welcome:
bug reports with a reproducing document or a failing bench case — those are gold, and
misses found this way become permanent regression-floor entries. What will be closed with
thanks: feature requests, integrations, and industry-specific modes inside this repository —
the vertical variants above are separate products precisely so that this one stays general and
small. No support is offered or implied; the Apache-2.0 license means forks are not just
permitted but encouraged. Security issues: see SECURITY.md — privately, please.
The ~3,600 hand-verified ground-truth annotations are released CC BY 4.0 — use them to
evaluate any tool, ours or a competitor's, with attribution. This repository was published as
a single fresh commit rather than an imported development history; the numbers, not the commit
log, are the record, and npm run audit recomputes them.