Added
-
CONTRIBUTING.md: contributor workflow doc per the qte77 doc-structure canon — documentation hierarchy, commands, conventional commits, branches, changelog, and releasing (#75). -
CLAUDE.md: one-line@AGENTS.mdimport so the Claude Code loader and agents read the same rules (#75). -
.github/workflows/{bump-my-version,tag-release,publish-release}.yaml: estate-standard semi-automatic release pipeline (dispatch bump → PR → tag-on-merge → optional publish), mirrored frompaperversewith SHA-pinned actions (#75). -
pyproject.toml:[tool.bumpversion]config +bump-my-versiondev dependency, bumpingpyproject.toml,src/pseudonymize_text/__init__.py, and the README version badge from a single source of truth (#75). -
Makefile:changelog_new/changelog_preview/changelog_releasetargets wrapping scriv (#75). -
--allow-broad-patternsCLI flag (detect/apply) — wires up the broad-pattern override thatdetectors.terms.load_termsalready enforced but that was never exposed on the command line (docs/TERMS_CSV.mddocumented it). -
Opt-in
phidetector group (--detectors phi) with checksum-validated NPI (Luhn over the 80840 prefix), DEA (registrant checksum), and VIN (ISO 3779 mod-11) detectors indetectors/phi.py;npi/dea/vinadded to the default--types. Pure-Python, no new dependency. Part of #42 — clinical NER / MRN / date-coarsening remain deferred. Newdocs/PHI.md. -
detectandapplynow print a one-line run summary to stderr on success — total spans, affected file count, and per-type counts (e.g.pseudonymize detect: 5 spans across 2 files — email:3 name:2). Counts only; never plaintext, context, or tokens. Part of D1. -
docs/decisions/README.md— an index of accepted ADRs (001–003) with status and date. Part of B2. -
Opt-in contextual MRN detection via
--phi-context(with--detectors phi): a 6–10 digit run within 60 chars of an MRN cue (MRN,Medical Record Number, …) is tokenized as<MRN:…>. No checksum, so it is higher false-positive — review the report. Part of #42. -
Opt-in EU national-ID detectors via
--detectors eu(taggedeuper ADR_003): German Steuer-ID (de_steuer), French NIR (fr_nir), UK NHS number (gb_nhs), Spanish DNI/NIE (es_dni), Italian Codice Fiscale (it_cf) — all checksum-validated viapython-stdnum. Closes #86. -
examples/— a runnable end-to-end demo (make demo) over a small mixed corpus spanning.txt/.md/.csv/.json/.log/.eml: public-domain excerpts (RFC 2822, a Lincoln letter) plus synthetic records that exercise every detector (structured + opt-in PHI + EU IDs, all checksum-valid) and mail handling (RFC 2047 header tokenization,DKIM-Signaturestrip, attachment drop). The demo uses an ephemeral key (live output gitignored underexamples/_out/);examples/sample-output/holds an illustrative committed snapshot. No key/mapping/plaintext committed.examples/README.mdcross-references the larger doc-pipeline-engine corpus.
Changed
-
README.md: restructured to the canonical estate order (Hero → Badges → What → How → Why → Refs → License); H1 renamed topseudonymize-text, install fixed touv add pseudonymize-text, Version badge linked toCHANGELOG.md, CodeQL badge added; static badges keep the teal58f4c2token by design (#75). -
AGENTS.md:## Commandsnow points toCONTRIBUTING.md § Commandsinstead of duplicating the recipe list (#75). -
apply --planre-detects.eml/.mboxparts (the plan keys spans by file and cannot be replayed across a message's MIME parts); detection is deterministic, so the result still matches the audited plan. Keep--termson theapplycommand for mail corpora — documented indocs/USAGE.md. -
docs/landscape/de-identification.md: refreshed the comparison against verified upstream facts — added rows for philter-lite (maintained philter fork), Faker, and scrubadub; flagged philter-ucsf (last release 2020) and scrubadub (2023) as stale; corrected the scrubadub entry from "streaming" to library-API typed-placeholder redaction. -
tokenize.canonicalizeand the report-recorddetectorpattern now cover the PHI types; docs narrowed to reflect partial PHI support (README.md,docs/{COMPLIANCE,ARCHITECTURE,USAGE,USER_STORIES,HASHING,roadmap}.md,docs/landscape/de-identification.md). -
docs/USAGE.md: documented the previously-missingPSEUDONYMIZE_MAX_FILE_BYTESenvironment variable. -
Link checking now validates intra-document anchors (
include_fragmentsinlychee.toml), so stale#fragmentlinks inREADME.md/docs/are caught in CI. Part of B1. -
markdownlint now also covers the root docs
CONTRIBUTING.md,AGENTS.md, andCHANGELOG.md(in bothmake lintand the CI workflow).CLAUDE.mdis excluded by omission — it is a one-line@AGENTS.mdimport that would otherwise trip MD041. Part of B3. -
Docs now tag detector coverage by jurisdiction (international / US / EU) and link the ADR_003 source-of-truth table from README, COMPLIANCE, GLOSSARY, USER_STORIES, and ARCHITECTURE (previously only USAGE linked it). Reconciled stale "MRN out of scope" wording across these docs (MRN is opt-in via
--phi-context). Part of E4.
Fixed
-
detectnow scans.eml/.mboxmail (routed throughformats/likeapply), so the audit plan is no longer silently empty for mail — closing a leak where the documenteddetect→apply --planworkflow shipped mail with PII intact. -
.eml/.mboxaddress headers (From/To/Cc/Bcc/Reply-To) are pseudonymised viagetaddresses/formataddr, so display-name tokens are no longer mangled to<NAME>:hex>by the RFC 5322 address parser and the output re-parses cleanly. -
docs/USAGE.md: removed the non-existent--ner-confidenceflag from the discovery-pass example (it was never implemented). -
detectnow always writes the report header, even when zero spans are found, so a laterapply --planreads a valid empty plan (exit0) instead of failing exit4on a missing file.applyemits the header for parity. Part of A1. -
apply --planon a mail corpus with no--termsnow warns on stderr: mail parts are re-detected with an empty term list, so literal entities (names/orgs) would silently pass through unredacted. Pass--termsfor mail corpora. Part of A2. -
README install instructions:
uv add pseudonymize-textfailed because the package is not yet on PyPI. The quickstart now installs the CLI from git (uvx --from git+…/uv tool install git+…, PyPI publish planned) and is copy-pasteable (createsruns/in, a sampleterms.csv, and the key). USAGE points to the README for install. -
Mail (
.eml/.mbox) output is now deterministic. Pseudonymized headers are replaced in place instead of being deleted and re-added, which had re-ordered them in per-process (hash-randomized)frozensetorder — so the same input + key now produces byte-identical output across runs, restoring the determinism guarantee for mail.