Skip to content

OpenMed v1.6.0

Choose a tag to compare

View all tags
@maziyarpanahi maziyarpanahi released this 23 Jun 09:36
v1.6.0
1863350
This commit was signed with the committer’s verified signature.
maziyarpanahi Maziyar Panahi
GPG key ID: 58C6DE3FED1C5846
Verified
Learn about vigilant mode.

OpenMed 1.6.0 is the privacy assurance and release-governance release.

This release turns de-identification from a single API call into a full, auditable privacy system: policy-aware pipelines, canonical span records, deterministic safety backstops, calibrated thresholds, signed audit reports, re-identification risk scoring, and release gates that fail closed when evidence is missing.

It also adds the infrastructure needed to ship model artifacts responsibly: manifest-driven model cards, Hugging Face publishing controls, dependency and secret-scanning gates, public benchmark reporting, and a richer evaluation suite.

Highlights

  • Added a policy-aware de-identification runtime built around canonical OpenMedSpan records, a ten-stage privacy pipeline, detector arbitration, calibrated thresholds, deterministic safety sweeps, and six bundled policy profiles.
  • Added signed, reproducible audit reports with span provenance, residual-risk metadata, reproducibility hashes, and optional HMAC signatures.
  • Added re-identification risk reporting and adversarial re-identification benchmark mode.
  • Added a leakage-first evaluation harness with synthetic golden fixtures, public/reference dataset adapters, DUA-gated stubs, SHIELD support, weak labeling, cold-start latency, and bootstrap confidence intervals.
  • Added v1.6.0 release-gate infrastructure: last-green baselines, calibration artifacts, G1a-G8 signed reports, quantization recall-delta checks, status/leaderboard generation, and fail-closed CI.
  • Added clinical and interoperability utilities: ConText temporality and uncertainty axes, OHDSI Athena/Usagi ingestion, a Presidio bridge, and deterministic FHIR R4 transaction/batch Bundle assembly.
  • Added a cardiology zero-shot label-map domain plus cardiology keyword routing metadata for future model registration.
  • Added a canonical models.jsonl manifest, manifest refresh workflow, manifest-driven Hugging Face model cards, and HF publishing for converted MLX/CoreML artifacts.
  • Added a packaged openmed CLI surface with benchmark and calibration commands, plus a de-identification cookbook notebook and clinical NER families example.
  • Added governance, compliance, security, device-tier, FAQ, API reference, release-channel, status, leaderboard, and notebook documentation.

The Big Picture

OpenMed v1.6.0 is about moving from "the model found PII" to "the system can prove what happened."

The new privacy stack records normalized spans, detector provenance, policy actions, thresholds, safety-sweep findings, risk metadata, and stable hashes. That makes de-identification outputs easier to audit, reproduce, benchmark, and review before they reach clinical, operational, or demo workflows.

The same theme shows up in release engineering. Model and package releases now have stronger evidence requirements: calibrated thresholds, last-green baselines, quantization deltas, release gate reports, dependency/license checks, and secret scanning.

Privacy Pipeline

deidentify() now routes through a staged privacy pipeline.

The pipeline includes:

  • Unicode normalization and offset mapping
  • language/script routing
  • section and document-type metadata
  • deterministic PII detectors
  • fast PII and clinical PHI model hooks
  • detector arbitration and cascade routing
  • policy action assignment
  • deterministic safety sweep
  • auditable result emission

Bundled policy profiles:

  • hipaa_safe_harbor
  • hipaa_expert_review_assist
  • gdpr_pseudonymization
  • research_limited_dataset
  • strict_no_leak
  • clinical_minimal_redaction

Example:

from openmed import deidentify

result = deidentify(
    "Patient Jane Rivera, MRN 123456, was seen on 2026-06-01.",
    method="replace",
    policy="hipaa_safe_harbor",
    keep_mapping=True,
    audit=False,
)

print(result.deidentified_text)

Use audit=True when you need the signed audit-report path instead of the regular DeidentificationResult.

Audit And Risk

OpenMed now includes signed audit reports and re-identification risk scoring.

Audit reports can include:

  • canonical span records
  • detector and policy provenance
  • threshold metadata
  • residual-risk summaries
  • reproducibility hashes
  • optional HMAC signatures

Risk reporting adds leakage and re-identification views for text and tabular records, including singleton and quasi-identifier analysis. The new adversarial re-identification benchmark mode lets release candidates be tested against round-trip and surrogate-consistency failures.

Evaluation And Release Gates

The new evaluation framework is leakage-first. It adds BenchmarkReport, golden fixtures, SHIELD comparison support, weak labeling, public/gated dataset adapters, cold-start latency, bootstrap confidence intervals, and report rendering.

Release gates now cover G1a-G8 readiness checks with signed reports and fail-closed CI. The release-gate workflow expects evidence for:

  • release metadata
  • calibration thresholds
  • masking and replacement profiles
  • G8 span fixtures
  • quantization recall deltas for quantized formats
  • last-green baseline comparisons

This is not only for dashboards. It is meant to prevent a model artifact from being promoted without the evidence needed to defend the release.

Clinical And Interoperability

v1.6.0 adds several clinical interoperability building blocks:

  • cardiology zero-shot labels: CardiacFinding, ECGFinding, EjectionFraction, CardiacProcedure, CardiacDevice, and Anatomy
  • cardiology keyword routing metadata for ECG findings, ejection fraction, arrhythmias, stents, pacemakers, murmurs, echocardiograms, and related cardiology text
  • ConText temporality axis: recent, historical, hypothetical
  • ConText uncertainty axis
  • OHDSI Athena vocabulary loader
  • Usagi mapping ingestion
  • Presidio adapter
  • FHIR R4 Bundle assembler

The FHIR assembler produces deterministic transaction/batch Bundles with stable urn:uuid fullUrl values, internal reference rewriting, and request metadata for server ingestion.

No cardiology model is registered in v1.6.0. Cardiology routing is forward metadata and zero-shot label-map support; public model suggestions still fall back to existing general medical recommendations until a cardiology model ships.

Model Manifest And Publishing

The canonical model manifest is now a first-class artifact.

This release adds:

  • models.jsonl
  • manifest refresh workflow
  • manifest-driven model registry loading
  • manifest-driven model card generation
  • Hugging Face upload support for converted MLX/CoreML artifacts
  • protected hf-publish environment policy
  • HF_WRITE_TOKEN handling guidance

Packaging now includes the manifest, gate baseline, policy/schema JSON, LICENSE, and NOTICE so release artifacts have the metadata they need.

CLI, Docs, And Examples

The openmed CLI is now packaged and includes benchmark and calibration commands. The benchmark command supports SHIELD suite runs, multi-model JSON output, and adversarial re-identification mode.

New and refreshed documentation includes:

  • de-identification cookbook notebook
  • clinical NER families example
  • project FAQ
  • mkdocstrings API reference
  • compliance posture
  • release streams and channels
  • HF token policy
  • dependency and secret-handling docs
  • device-tier SLOs
  • status and leaderboard pages

Security And CI

v1.6.0 expands release hardening:

  • protected HF publish environment
  • dependency license policy gate
  • pip-audit gate with time-boxed ignores
  • gitleaks CI/pre-commit secret scanning
  • grouped GitHub Actions Dependabot updates
  • repo policy checks
  • Ruff-based Python lint/format
  • Swift swift-format scripts and CI gate
  • release-gates workflow

The old duplicate .github/workflows/release.yml workflow was removed. PyPI publishing now goes through the guarded tag/manual publish.yml workflow.

Fixes

  • REST/MCP PII schemas now accept ar, ja, and tr.
  • Privacy-filter trust_remote_code allowlist matching is case-insensitive.
  • Feb 29 date shifting clamps safely when keep_year=True targets a non-leap year.
  • REST requests are capped by OPENMED_SERVICE_MAX_TEXT_LENGTH and return 422 on oversized text.
  • BatchProcessor.iter_process now honors batch_size while preserving order.
  • Duplicate benchmark fixture IDs are rejected.
  • Duplicate benchmark CLI registration was fixed.
  • Release gates skip cleanly when no candidate report exists.
  • User-controlled HTML formatter fields are escaped.
  • Long non-ASCII/CJK clinical text is no longer rejected as suspicious content.
  • Reversible remove mappings and repeated same-type placeholders now work with keep_mapping=True.

Migration Notes

  • deidentify(..., audit=True) returns an audit report. Leave audit=False when callers expect DeidentificationResult.
  • deidentify(..., keep_mapping=True) can now produce unique repeated placeholders such as [NAME_2]. Snapshot tests may need updates. Without keep_mapping, legacy repeated mask placeholders are preserved.
  • Prefer method="shift_dates" in new code. shift_dates remains available as a compatibility alias.
  • REST clients sending more than OPENMED_SERVICE_MAX_TEXT_LENGTH characters now receive a 422 unless the limit is raised.
  • Full SHIELD and DUA corpora require approved or user-supplied access paths; restricted rows are not vendored.
  • Use .github/workflows/publish.yml for PyPI release operations.
  • Model publishing requires HF_WRITE_TOKEN in the protected hf-publish environment.
  • Local Python formatting is now Ruff-based: make format, make lint, and make format-check.
  • Swift formatting uses make format-swift and make lint-swift.

PR Review Summary

Reviewed range: v1.5.5..master

  • Last release: v1.5.5 at adff5fe
  • Reviewed head: origin/master at 3b62f2e
  • Merged PRs reviewed: 69
  • Direct first-parent release-prep commits reviewed: 7 (f654531, .gitignore; 8f398cc, CHANGELOG.md; baad551, CHANGELOG.md; 0769007, docs/swift-openmedkit.md; 0190625, docs/website/index.html; 0b9bdd8, openmed/__about__.py; 3b62f2e, README.md)
  • Aggregate diff: 371 files changed, 40,102 insertions, 4,854 deletions

Major PR groups:

Open Items Before Tagging

  • Reconcile the website language-support copy: PR #404 advertises 16-language PII support, while core/service/README/FAQ currently enumerate 12 PII languages (en, fr, de, it, es, nl, hi, te, pt, ar, ja, tr).
  • Confirm the v1.6.0 release-gate candidate bundle has required metadata, calibration evidence, G8 span fixtures, and quantization evidence.
  • Previously identified release blockers are now resolved by merged PRs: #554 / #529 rejects empty audit HMAC keys; #550 / #530 rejects duplicate FHIR ResourceType/id values; #559 / #513 shifts default-model lowercase date labels by canonical label.
  • Decide whether #68 is required by release policy. GitPython is present in uv.lock through the docs extra, not imported by runtime code.
  • #556 was merged and adds FHIR Bundle empty-input and dangling-reference regression coverage. Treat #553 as a useful non-blocking FHIR refactor unless its helper extraction is required for the release branch.
  • Run the full project test/lint/release-gate suite before tagging if you want coverage beyond the targeted release-prep gates listed below.

Verification For This Draft

  • Re-audited against current origin/master on 2026-06-22 after PRs #559, #554, #550, and #556 plus the direct release-prep commits through 3b62f2e were merged. The latest release tag is still v1.5.5, reviewed head is 3b62f2e, and the first-parent range contains 76 commits: 69 merged PRs plus direct release-prep commits f654531, 8f398cc, baad551, 0769007, 0190625, 0b9bdd8, and 3b62f2e.
  • Four parallel subsystem reviews were completed: privacy/de-identification, evaluation/release gates, docs/CLI/service, and CI/release/MLX.
  • The pasted GitHub release-note list, live gh pr list metadata, and this draft resolved to the same 64 merged PR numbers before PR #551. After PR #551, #559, #554, #550, and #556, live GitHub metadata and this draft resolve to the same 69 merged PR numbers, with no missing or extra PRs.
  • A changed-area coverage check confirmed explicit release-note coverage for the privacy pipeline, evaluation harness, release gates, datasets, clinical interop, model manifest/publishing, CLI/service, docs/examples, website, security/CI, Swift/MLX, tests, packaging, and the direct .gitignore commit.
  • PR #551 was reviewed separately: CI passed, focused cardiology label/routing tests passed locally, Ruff passed on changed files, and the runtime probe confirmed cardiology zero-shot labels plus private cardiology routing while public model suggestions retain the intended fallback behavior.
  • PRs #559, #554, #550, and #556 were reviewed separately with parallel focused checks. method="shift_dates" now uses canonical date labels, de-identification audit signing rejects empty HMAC keys, FHIR Bundle assembly rejects duplicate ResourceType/id values, and empty/dangling FHIR Bundle edge cases now have regression coverage.
  • Latest origin/master CI for 3b62f2e passed.
  • python3 scripts/release/check_release_version.py --version 1.6.0 passed after the local version bump.
  • uv run --extra docs mkdocs build --strict passed.
  • uv run --extra dev ruff check . passed.
  • uv run --extra dev pytest tests/unit -q passed: 1,615 passed, 1 skipped.
  • uv run --with build python -m build passed and produced 1.6.0 artifacts.
  • uv run --with twine twine check dist/* passed.
  • git diff --check passed after the changelog, version, and release-note edits.

What's Changed

New Contributors

Full Changelog: v1.5.5...v1.6.0

Contributors

gaoflow, maziyarpanahi, and 16 other contributors
Assets 2
Loading