Skip to content

SimplerAI/simpler-redact

Repository files navigation

Simpler Redact

The local document stripper. Drop in a contract, a client file, a case document — get back a blanked copy with every name, company, brand, address, phone, email and ID number replaced by placeholders, plus a separate key file mapping each placeholder to what it replaced. The blanked copy is safe to paste into a cloud AI or send onward; the key never leaves your side.

Nothing leaves your machine. The model is a stock, hash-verifiable 3 GB Gemma running locally; this codebase makes no network call beyond localhost — no telemetry, no analytics, nothing. One grep proves it.

What this repository is, exactly. The engine, the deterministic rails, the command-line tool, and the entire benchmark — corpus, hand-built ground truth, scoring scripts, methodology, and every named miss. It runs today from a clone. The desktop application is not released yet: no installers, no binaries, no signed builds. That order is deliberate — the part that can be independently checked ships before the part that can only be trusted. You bring the model; it is Google's, ~3 GB, and not ours to redistribute.

simpler.red — simplifying red tape.

The numbers (reproducible — the whole benchmark ships in this repo)

The headline is out-of-sample, and it leads because that is the only honest place to start (bench/oos-walkforward/): a walk-forward protocol — freeze the engine, acquire whole public documents it has never seen, build an exhaustive adversarially-audited ground truth BEFORE the engine runs once, publish the number as-is; fixes from one round are only ever proven by the next round's fresh documents. Five rounds in: 84.62 → 95.38 → 88.46 → 93.76 → 94.97% — the v1.0 headline (round 5, pre-registered publish-as-drawn) is 94.97% of entities and 98.2% of printed occurrences on 18 hostile unseen documents, and the release engine scores 95.82% across the full 118-document battery (BENCHMARK.md has the one-engine table with every tier's saturation labeled).

The measured envelope, stated plainly: those numbers hold for the genres the battery contains — SEC filings and exhibits of every kind, press releases, earnings transcripts, US and Singapore court judgments, US government documents, UK RNS, and OCR-degraded scans. On genres deliberately outside that history the measured band is lower: round 3 went wide on purpose and read 88.46% overall with frontier genres near 78% (bench/oos-walkforward/round-3/RESULT.md). A document type this engine has never been measured on should be expected to land in that band, not at the headline — that is what the review table is for, and new-genre misses are exactly what the walk-forward protocol eats (KNOWN-ISSUES.md names every open one).

Baselines on the same metric: Microsoft Presidio as shipped, 43.10% — measured on 90 of these documents (walk-forward rounds 1–4), never on round 5 or the full battery; our engine on the TAB legal benchmark's must-mask tier (their ground truth, not ours), 96.12%. Annotation quality is itself measured: blind second-pass agreement 0.925 — the medical gold standard's human-human bar is 0.927. Every miss is named and family-sorted in each round's RESULT.md; every surprising number carries its discovery path in paper/FINDINGS.md. In-sample and out-of-sample are never blended. That is the difference between a benchmark you grade yourself on and one that can hurt you.

Verify the claims yourself in seconds, no model needed: npm run audit recomputes every offline-verifiable published number from the committed artifacts — the per-doc rows, the aggregates, and a full from-scratch rescore of rounds 4–5 against the committed masked outputs — and exits red on any drift. To score YOUR tool on the same documents and metric: LEADERBOARD.md.

Two engines run in series. Engine A (the wide sweep) reads the original: a chunked extraction pass plus deterministic name rails, with a small local model doing only judgment — every span it nominates must occur verbatim in the text or it is refused. Engine B (the specialist) reads only the already-redacted output and hunts the classes that history says get left behind: a zero-model regex battery (phones in any format, ID families, URLs, tickers, address fragments) plus a narrowly-scoped second model sweep. One entity table, kept in code — find a name once and every copy in the document is masked.

The engine also carries an internal regression floor — 8 real SEC filings (hand-labeled, sources in bench/corpus/PROVENANCE.md) plus 44 synthetic adversarial documents (generators included, ground truth by construction), in-sample by definition and never the generalization claim:

configuration recall on identity mentions (in-sample ratchet)
Engine A alone, as first shipped 84.8–93.7%
+ the structural rails 98.0–100%
A + B, the shipped default 100.00% — 765/765 core + 226/226 shorthand

npm run bench:all re-runs that floor and exits red on any core leak: a measured miss never returns. It is the ratchet the walk-forward rounds climb on, not evidence of generalization — the rounds above are.

Large documents are measured, not assumed — and measuring found a bug, which is the point: two real SEC annual reports run as a growing ladder to 79,000 words (bench/scale/) — throughput ~0.8 min per 1,000 words on the CPU floor, the whole 79k hand-labeled exhaustively (262 identity entities, 2,102 occurrences). First measurement: 95.80%, plus a real defect (junk fragments passing verbatim verification shredded the output at XBRL scale). The fix — a footprint floor + boundary-guarded masking — re-measured to 98.47% with a clean output, before/after published unedited in SCALE.md. A full 10-K ≈ 31 minutes; drop the archive in overnight.

The leak history that produced those rails is preserved unedited in bench/hardening-2026-07-19/REPORT.md — including what the early versions missed and the one-line fixes each miss became. npm run bench:all re-runs everything and exits red on any core leak; that floor is the law for every engine change.

What 100% means here, precisely: perfect recall on every document and planted trap in this repo's benchmark, enforced as a permanent regression floor. It is NOT a proof about arbitrary documents — no system can honestly make that claim. That is why the final step is always human: you review the entity table before the stripped file goes anywhere, and an incomplete run writes nothing at all.

And what stripping does not claim: the blanked copy deliberately preserves dates, amounts, places, and business facts — that is the point (a contract with its dates masked is unusable, and the profile is what you want the cloud AI to reason about). It follows that a sufficiently distinctive profile can sometimes be re-identified from preserved context alone, by exactly the kind of model the output is destined for. The claim is that identities are stripped, never handed over — not that inference is impossible. The scope boundary is measured, not hand-waved: our scores on TAB's quasi-identifier tiers (bench/baselines/tab/TAB.md) are published for precisely this reason. Where inference risk matters, review the table with that lens — and consider whether context lines (a distinctive market position, a unique address pattern) belong in the strip too.

Quickstart

  1. Get the model (once): gemma-4-E2B_q4_0-it.gguf — stock Google QAT weights, from Google's official repo on Hugging Face (requires an HF account and accepting Google's Gemma terms — see THIRD_PARTY_NOTICES.md). Then verify it — npm run model:verify -- <path-to-gguf> checks the pinned sha256 (3646b4c1…455e6fd, 3,349,514,112 bytes). Wherever the file came from, if the hash matches, you have the stock weights.
  2. Start the local server — llama.cpp's llama-server, from llama.cpp releases. One line, so it pastes the same way into bash, zsh, PowerShell and cmd:
    llama-server --model gemma-4-E2B_q4_0-it.gguf --port 49399 --host 127.0.0.1 --ctx-size 8192 --parallel 1 --jinja --chat-template-kwargs "{\"enable_thinking\":false}"
    
    (--jinja + enable_thinking:false are load-bearing — grammar enforcement depends on them. The numbers in this repo were measured on the CPU floor, adding --device none --fit off --n-gpu-layers 0; a GPU will be faster but is a different measured configuration.)
  3. Strip a document:
    node bin/red.mjs contract.txt              # full strip (two engines, the measured default)
    node bin/red.mjs contract.txt --mode quick # rails-only tier, ~3x faster, single-layer guarantee
    
    Out come contract.stripped.txt and contract.entities.json. Review the table; never ship the two files together.

There is nothing to install. Requires Node ≥ 20 and nothing else — the engine, the rails, the scorer and every test import node built-ins only, so on a fresh clone both of these run immediately, with no npm install, no model and no network:

npm run verify   # the gate: 70 tests, every measured lesson pinned as a failing-loudly test
npm run audit    # recomputes EVERY published number from committed artifacts; red on drift

If you only run one thing, run npm run audit — it re-derives the per-document rows, the aggregates, a from-scratch rescore of rounds 4 and 5 against the committed masked outputs, and the utility verdict count, then exits non-zero if any of it disagrees with what this repo claims. That is the whole trust argument in one command, and it does not need our cooperation.

npm run bench:all runs the live in-sample ratchet (52 documents) against your local model. Fresh clones: node bench/oos-walkforward/fetch-sg.mjs first rebuilds, sha-verified, the four Singapore judgments this repo deliberately does not redistribute (see each round's PROVENANCE.md).

Repo map

what where
The engine (Engine A: extraction + name rails + classify) lib/anonymize.mjs
The structural battery + junk flagging lib/rails.mjs
Engine B (strict-serial specialist) + the shipped pipeline lib/engineB.mjs
Engine B3 (surface-shape suspicion scorer) lib/suspicion.mjs
The CLI bin/red.mjs
Benchmarks: corpus, stress suites, generators, evidence bench/
Every open miss, named, with its fix-set queue KNOWN-ISSUES.md
Why the engine is built this way — every law with the measurement that taught it paper/FINDINGS.md
The full battery record and the protocol behind it BENCHMARK.md · bench/oos-walkforward/PROTOCOL.md
Any tool, this battery, one metric — submit a row LEADERBOARD.md
The offline recomputation of every published number bench/audit-claims.mjs
The gate: every measured lesson pinned as a failing-loudly test test/
What the model weights and corpus are licensed under THIRD_PARTY_NOTICES.md

What's next

This repository stays the general-purpose tool, and stays feature-frozen. What is in development is a set of separate products — variants tuned for legal, finance and medical documents — each with its own corpus, its own rails and its own published benchmark. That separation is the design, not a technicality: a small local model cannot hold every vertical's vocabulary at once without paying for it in false positives, so precision comes from which rails are loaded rather than from one engine trying to know everything.

Two things follow, and they are the honest version:

  • No dates. Nothing here is a commitment, and a vertical ships when it has a measured benchmark to publish, on the same walk-forward protocol as this one.
  • No measurement is claimed for any of them yet. In particular this battery contains no clinical documents at all — medicine has i2b2, and a medical variant will need its own ground truth, its own annotator agreement and its own leak ledger before it says anything.

For this engine, the open work is in KNOWN-ISSUES.md — the over-nomination bracket is the largest item, and fix set 5 is queued for a sixth round of fresh documents.

Contact

support@simpler.asia reaches the maintainer. For anything that is a leak in the privacy sense — a way unredacted content or the key can escape — use the private channel in SECURITY.md first, not email and not a public issue.

License, maintenance, contributions

Code: Apache-2.0 (© 2026 Simpler AI). Model weights: Google's Gemma Terms of Use — not ours to relicense; see THIRD_PARTY_NOTICES.md. Born 2026-07-19 as a fork of the simpler-harness privacy membrane's anonymizer lane; the measurement history that shaped every rule in this engine is part of the repo on purpose.

This tool is feature-complete by design. It does one thing, measured. What's welcome: bug reports with a reproducing document or a failing bench case — those are gold, and misses found this way become permanent regression-floor entries. What will be closed with thanks: feature requests, integrations, and industry-specific modes inside this repository — the vertical variants above are separate products precisely so that this one stays general and small. No support is offered or implied; the Apache-2.0 license means forks are not just permitted but encouraged. Security issues: see SECURITY.md — privately, please.

The ~3,600 hand-verified ground-truth annotations are released CC BY 4.0 — use them to evaluate any tool, ours or a competitor's, with attribution. This repository was published as a single fresh commit rather than an imported development history; the numbers, not the commit log, are the record, and npm run audit recomputes them.

About

Local document redaction, measured. A stock quantized Gemma E2B plus deterministic rails remove identifying spans on your own machine; nothing leaves it. Ships with the whole benchmark — 118 real documents, every miss named, and one offline command that recomputes every published number.

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages