redact

A local scrubber for text you're about to send to an LLM.

Before you paste a log file, a résumé, or a support thread into an LLM, this tool strips out the sensitive parts first — names, emails, phone numbers, SSNs, card numbers, addresses, API keys, database passwords — and hands back a clean copy. It runs entirely on your machine with no LLM involved: a stack of regexes and checksums catches the structured stuff, a small NER model handles the fuzzier things like people and companies, and a merge layer reconciles them when they disagree. You can either blank each value out as <PERSON> or swap in stable placeholders like PERSON_001 so the text still reads coherently, with the original values kept in a separate file that never leaves your machine. Most of the work went into not over-redacting — teaching it that "Django" in a skills list is a framework, not a person.

For how the detection actually works, see PIPELINE.md.

Install

python -m venv redact_venv
source redact_venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_sm

The GLiNER model (gliner_multi_pii-v1, ~1.1 GB) downloads from Hugging Face on first run and is cached.

Run

source env.sh          # thread-safety env vars — see the comments in the file

python -m redact notes.txt                 # -> notes.redacted.txt
python -m redact notes.txt -o clean.txt     # choose the output path
python -m redact notes.txt --pseudonymize   # -> notes.redacted.txt + notes.redacted.txt.map.json

--pseudonymize also writes notes.redacted.txt.map.json (label → original value). That file is sensitive — keep it local, never send it anywhere.

As a library:

from redact import redact_text

sanitized, counts, mapping = redact_text(text, language="en", pseudonymize=False)
# counts  -> {"EMAIL_ADDRESS": 3, "PERSON": 5, ...}
# mapping -> {} unless pseudonymize=True

Test

python run_tests.py

12 fixtures in tests/ (logs, résumés, chat, source code, medical, financial, and adversarial "same shape, different meaning" cases). Each checks both directions: sensitive values must be gone, ordinary values must survive unchanged.