Skip to content

v0.1.0 — the first release

Latest

Choose a tag to compare

@soulipaco soulipaco released this 22 Aug 18:25
· 14 commits to main since this release

The first release, cut on 2026-08-22 over thirteen working sessions and complete against the nine items of the charter's Definition of done.

What it is: a multilingual, provider-agnostic engine that detects and reduces PII in named free-text columns while preserving the structure around them — locally and on Azure Databricks — with every published quality number locked as a regression gate.

What it is not: an estate scanner. It reduces PII in columns an operator names; it has no column or table discovery, and it makes no claim to regulatory compliance or guaranteed anonymization.

At a glance, as tagged

default-tier tests 1240 — model-free, Spark-free, on every push
integration tests 97 — real models
regression gates 56, across three corpora and both provider chains
decision records 33
CI green on ubuntu-latest and windows-latest
entity taxonomy PERSON, EMAIL, PHONE detected; ADDRESS declared and not detected (ADR-0002)
languages English, German, Greek

Headline quality on the committed synthetic corpus, hybrid chain (deterministic_presidio): strict F1 0.910, leakage 0.067, document clean rate 0.871, over-redaction 0.000. The model-free chain is strict F1 0.723 with the same 0.000 over-redaction — it finds no names at all, and publishing that row is what makes the comparison mean anything.

What it gets wrong, published rather than hidden

  • Greek PERSON recall is 0.500. The good Greek spaCy models are CC BY-NC-SA and cannot enter an MIT project (ADR-0007), so Greek routes through a weaker multilingual model. The gap is diagnosed to three mechanisms (ADR-0019); two are addressed (ADR-0020, ADR-0021) and the third is span absorption, which neither remedy reaches.
  • ADDRESS is in the taxonomy and nothing detects it (ADR-0002). The regex route is refused by the repository's own rules, twice, on probe evidence.
  • The distributed mapInPandas path is shipped and has never run — the workspace's serverless sandbox returns ISOLATION_STARTUP_FAILURE, which is Databricks infrastructure. A databricks-marked test flips from skip to assertion the day it is fixed, with no code change here.

docs/22_EVIDENCE.md is the executed record: what was run, where, and what was not.

Note on this tag

v0.1.0 is the release as tagged. Work has continued on main since — a control panel, caller-settable accuracy knobs and a directory-picker template (ADR-0034/0035/0036) — and it sits under [Unreleased] in CHANGELOG.md, not in this release. No published number moved in that work.

Install

pip install -e ".[dev]"     # core + dev: no NLP model, no provider extra
pytest -q
pii-reduction benchmark     # the measured baseline over the committed corpus

The hybrid numbers need the provider extras and the documented spaCy models — see docs/15_PROVIDERS.md.

Full changelog: CHANGELOG.md