Releases: qte77/pseudonymize-text
Release list
v0.3.0
Added
-
CONTRIBUTING.md: contributor workflow doc per the qte77 doc-structure canon — documentation hierarchy, commands, conventional commits, branches, changelog, and releasing (#75). -
CLAUDE.md: one-line@AGENTS.mdimport so the Claude Code loader and agents read the same rules (#75). -
.github/workflows/{bump-my-version,tag-release,publish-release}.yaml: estate-standard semi-automatic release pipeline (dispatch bump → PR → tag-on-merge → optional publish), mirrored frompaperversewith SHA-pinned actions (#75). -
pyproject.toml:[tool.bumpversion]config +bump-my-versiondev dependency, bumpingpyproject.toml,src/pseudonymize_text/__init__.py, and the README version badge from a single source of truth (#75). -
Makefile:changelog_new/changelog_preview/changelog_releasetargets wrapping scriv (#75). -
--allow-broad-patternsCLI flag (detect/apply) — wires up the broad-pattern override thatdetectors.terms.load_termsalready enforced but that was never exposed on the command line (docs/TERMS_CSV.mddocumented it). -
Opt-in
phidetector group (--detectors phi) with checksum-validated NPI (Luhn over the 80840 prefix), DEA (registrant checksum), and VIN (ISO 3779 mod-11) detectors indetectors/phi.py;npi/dea/vinadded to the default--types. Pure-Python, no new dependency. Part of #42 — clinical NER / MRN / date-coarsening remain deferred. Newdocs/PHI.md. -
detectandapplynow print a one-line run summary to stderr on success — total spans, affected file count, and per-type counts (e.g.pseudonymize detect: 5 spans across 2 files — email:3 name:2). Counts only; never plaintext, context, or tokens. Part of D1. -
docs/decisions/README.md— an index of accepted ADRs (001–003) with status and date. Part of B2. -
Opt-in contextual MRN detection via
--phi-context(with--detectors phi): a 6–10 digit run within 60 chars of an MRN cue (MRN,Medical Record Number, …) is tokenized as<MRN:…>. No checksum, so it is higher false-positive — review the report. Part of #42. -
Opt-in EU national-ID detectors via
--detectors eu(taggedeuper ADR_003): German Steuer-ID (de_steuer), French NIR (fr_nir), UK NHS number (gb_nhs), Spanish DNI/NIE (es_dni), Italian Codice Fiscale (it_cf) — all checksum-validated viapython-stdnum. Closes #86. -
examples/— a runnable end-to-end demo (make demo) over a small mixed corpus spanning.txt/.md/.csv/.json/.log/.eml: public-domain excerpts (RFC 2822, a Lincoln letter) plus synthetic records that exercise every detector (structured + opt-in PHI + EU IDs, all checksum-valid) and mail handling (RFC 2047 header tokenization,DKIM-Signaturestrip, attachment drop). The demo uses an ephemeral key (live output gitignored underexamples/_out/);examples/sample-output/holds an illustrative committed snapshot. No key/mapping/plaintext committed.examples/README.mdcross-references the larger doc-pipeline-engine corpus.
Changed
-
README.md: restructured to the canonical estate order (Hero → Badges → What → How → Why → Refs → License); H1 renamed topseudonymize-text, install fixed touv add pseudonymize-text, Version badge linked toCHANGELOG.md, CodeQL badge added; static badges keep the teal58f4c2token by design (#75). -
AGENTS.md:## Commandsnow points toCONTRIBUTING.md § Commandsinstead of duplicating the recipe list (#75). -
apply --planre-detects.eml/.mboxparts (the plan keys spans by file and cannot be replayed across a message's MIME parts); detection is deterministic, so the result still matches the audited plan. Keep--termson theapplycommand for mail corpora — documented indocs/USAGE.md. -
docs/landscape/de-identification.md: refreshed the comparison against verified upstream facts — added rows for philter-lite (maintained philter fork), Faker, and scrubadub; flagged philter-ucsf (last release 2020) and scrubadub (2023) as stale; corrected the scrubadub entry from "streaming" to library-API typed-placeholder redaction. -
tokenize.canonicalizeand the report-recorddetectorpattern now cover the PHI types; docs narrowed to reflect partial PHI support (README.md,docs/{COMPLIANCE,ARCHITECTURE,USAGE,USER_STORIES,HASHING,roadmap}.md,docs/landscape/de-identification.md). -
docs/USAGE.md: documented the previously-missingPSEUDONYMIZE_MAX_FILE_BYTESenvironment variable. -
Link checking now validates intra-document anchors (
include_fragmentsinlychee.toml), so stale#fragmentlinks inREADME.md/docs/are caught in CI. Part of B1. -
markdownlint now also covers the root docs
CONTRIBUTING.md,AGENTS.md, andCHANGELOG.md(in bothmake lintand the CI workflow).CLAUDE.mdis excluded by omission — it is a one-line@AGENTS.mdimport that would otherwise trip MD041. Part of B3. -
Docs now tag detector coverage by jurisdiction (international / US / EU) and link the ADR_003 source-of-truth table from README, COMPLIANCE, GLOSSARY, USER_STORIES, and ARCHITECTURE (previously only USAGE linked it). Reconciled stale "MRN out of scope" wording across these docs (MRN is opt-in via
--phi-context). Part of E4.
Fixed
-
detectnow scans.eml/.mboxmail (routed throughformats/likeapply), so the audit plan is no longer silently empty for mail — closing a leak where the documenteddetect→apply --planworkflow shipped mail with PII intact. -
.eml/.mboxaddress headers (From/To/Cc/Bcc/Reply-To) are pseudonymised viagetaddresses/formataddr, so display-name tokens are no longer mangled to<NAME>:hex>by the RFC 5322 address parser and the output re-parses cleanly. -
docs/USAGE.md: removed the non-existent--ner-confidenceflag from the discovery-pass example (it was never implemented). -
detectnow always writes the report header, even when zero spans are found, so a laterapply --planreads a valid empty plan (exit0) instead of failing exit4on a missing file.applyemits the header for parity. Part of A1. -
apply --planon a mail corpus with no--termsnow warns on stderr: mail parts are re-detected with an empty term list, so literal entities (names/orgs) would silently pass through unredacted. Pass--termsfor mail corpora. Part of A2. -
README install instructions:
uv add pseudonymize-textfailed because the package is not yet on PyPI. The quickstart now installs the CLI from git (uvx --from git+…/uv tool install git+…, PyPI publish planned) and is copy-pasteable (createsruns/in, a sampleterms.csv, and the key). USAGE points to the README for install. -
Mail (
.eml/.mbox) output is now deterministic. Pseudonymized headers are replaced in place instead of being deleted and re-added, which had re-ordered them in per-process (hash-randomized)frozensetorder — so the same input + key now produces byte-identical output across runs, restoring the determinism guarantee for mail.
v0.2.0 — runs/ sandbox + default paths
Runtime sandbox + default-path cut
All runtime artefacts (key, report, mapping, term lists, plans, ignore lists) now land under ./runs/ by default. .gitignore ships the sandbox plus belt-and-braces per-artefact globs so default-named outputs stay out of the repo even when an operator runs outside runs/.
Pre-1.0 behaviour change: operators relying on the previous cwd-root defaults must pass explicit --report / --mapping flags or move artefacts under runs/.
CLI defaults
--reportdefault:./pseudonymize-report.jsonl→./runs/pseudonymize-report.jsonl--mappingdefault:./pseudonymize-mapping.json→./runs/pseudonymize-mapping.json- Parent directory of
--reportand--mappingis auto-created on first write, so the new default works on a fresh checkout without an explicitmkdir.
Gitignore
runs/ and local/ sandbox directories; per-artefact globs (*-mapping.json, *-report.jsonl, plan*.jsonl, discovery*.jsonl, ignore.txt, false-positives*.txt); term-list globs (terms.csv, terms.json); !tests/fixtures/** re-include so shipped fixtures stay tracked.
Docs aligned
README Quickstart, every docs/USAGE.md example block, docs/SECURITY.md paths + gitignore guidance, docs/ARCHITECTURE.md data-flow diagram, docs/USER_STORIES.md jq example, docs/roadmap.md (planned convenience items rolled to 0.3.0), docs/landscape/de-identification.md status reference.
Also included (from the pending [Unreleased] cut)
docs/assets/images/architecture-bird.svg— bird's-eye SVG of the public output lane vs the secret artifacts lane, embedded collapsed indocs/ARCHITECTURE.md.- 9 phantom
--flagrows removed fromdocs/USAGE.md; exit code7description widened to cover all three causes. docs/decisions/MADR layout (renamed fromdocs/ADR/);docs/COMPLIANCE.mdPHI disclaimer narrowed;NOTICEexpanded;docs/GLOSSARY.mdalphabetised.
Full diff: v0.1.0...v0.2.0
v0.0.1 — primitives
Initial release. Step 1 primitives: tokenize, mapping, report. Roadmap and contract in docs/.