fa-redact v0.4.0

fa-redact v0.4.0

Production-ready Persian PII detection with stronger NER, long-document support, privacy-safe serialization, batch processing, and evidence-based performance validation.

v0.4.0 is a major step forward for fa-redact, expanding the library beyond deterministic Iranian identifier detection into a more complete, production-oriented toolkit for Persian privacy and de-identification workflows.

Highlights

🧠 Validated Persian PERSON NER

Persian named-entity recognition has been extensively evaluated across independent synthetic and clinical-style Persian datasets.

The existing PEYMA-based PersianNERDetector remains the reference PERSON backend, while the new ONNXPersianNERDetector provides an optional alternative for environments where PyTorch is not desirable.

Both backends support:

• exact source character offsets
• deterministic inference
• long-document processing
• conservative PERSON reconstruction
• Persian name-component handling
• local-only model execution

No model downloads or network resolution are performed automatically.

📄 Robust long-document NER

NER processing now supports documents beyond the model sequence limit through deterministic overlapping sliding windows.

The implementation preserves exact source offsets, deduplicates overlapping predictions, and carefully reconstructs entities crossing window boundaries without incorrectly merging unrelated names.

⚡ Evidence-based performance profiling

This release includes extensive CPU performance and memory profiling across deterministic detectors and both supported NER backends.

The zero-dependency deterministic core remains extremely lightweight, while the benchmark results provide practical guidance for choosing between PEYMA/PyTorch and TookaBERT/ONNX depending on deployment requirements.

Rather than introducing speculative optimizations, the project keeps performance changes evidence-driven.

🏢 Iranian Legal Entity National ID

A new opt-in detector and validator are available for Iranian legal-entity national identifiers:

• IranianLegalEntityIDDetector
• is_valid_iranian_legal_entity_id()
• entity type: IR_LEGAL_ENTITY_ID

Validation is performed entirely offline using structural and checksum validation, with support for ASCII, Persian, Arabic-Indic, and mixed-script digits.

The detector remains opt-in and does not alter the default detector set.

📦 Streaming batch processing

New helpers make it easier to process large collections without loading everything into memory:

• detect_many()
• redact_many()
• report_many()

They work lazily with any Python iterable, including generators and one-shot streams.

🔐 Privacy-safe serialization

New serialization helpers allow detection metadata and reports to be safely exported without leaking detected PII values.

Structured detection serialization intentionally excludes raw and normalized values, while report serialization provides aggregate metadata suitable for logging, analytics, and audit pipelines.

🧩 Zero-dependency core preserved

Despite the expanded functionality, the base installation still has:

0 mandatory runtime dependencies.

pip install fa-redact==0.4.0

Optional Persian NER backends remain explicitly installable:

pip install "fa-redact[ner]==0.4.0"

or:

pip install "fa-redact[onnx]==0.4.0"

This keeps deterministic PII detection lightweight while allowing ML functionality only where it is actually needed.

Compatibility

• Python 3.10+
• deterministic core remains dependency-free
• default detector behavior remains backward compatible
• ML-based PERSON detection remains opt-in
• no model artifacts are bundled with the package
• no automatic network model downloads

What remains unchanged

The default detection pipeline continues to focus on the three core Iranian identifiers:

• Iranian National ID
• Iranian mobile numbers
• Iranian IBAN

Additional detectors and ML-based PERSON recognition remain explicit opt-ins, keeping the default behavior predictable, lightweight, and safe.

---

Full technical details, benchmark methodology, validation results, and individual changes are available in the project CHANGELOG.md and research/ documentation.

Thanks to everyone following and testing the project. Feedback, bug reports, and real-world integration experiences are very welcome.

fa-redact v0.3.0

fa-redact v0.3.0 expands the privacy-first Persian/Iranian PII toolkit with reporting, CLI workflows, structured-record helpers, experimental opt-in Persian PERSON NER, and conservative clinical redaction profiles.

Highlights

• Privacy-safe reporting: DetectionReport, detection_report(), report_detections(), aggregate/value-free reporting without storing detected PII values
• Command-line interface: fa-redact detect, fa-redact report, fa-redact redact, stdin/stdout and file workflows
• Structured data helpers: detect_fields(), redact_fields(), report_fields(), explicit field selection, record-wide referential consistency
• Experimental Persian PERSON NER: opt-in PersianNERDetector, caller-supplied local Hugging Face-compatible model, no automatic model download, optional fa-redact[ner], exact source-character offset preservation, fail-loud long-text behavior
• Clinical redaction profiles: ClinicalRedactionProfile, ClinicalTextTemplate, clinical_profile(), templates: outpatient_note, discharge_summary, referral_letter, institution-specific IDs through explicit PatternRule, PERSON detection remains explicit opt-in

Stable defaults

The global default detector set remains:

• Iranian National ID
• Iranian mobile number
• Iranian IBAN

Email, bank-card, pattern, and Persian NER detectors remain opt-in.
Base installation still has zero mandatory runtime dependencies.

Important limitations

• PersianNERDetector is experimental.
• PEYMA benchmark results are news-domain research evidence, not clinical accuracy guarantees.
• No automatic personal-name detection occurs in clinical profiles unless a person detector is explicitly supplied.
• No universal MRN/Patient ID patterns are built in.
• No generic address or date/DOB detector is included.
• No patient/doctor role inference is performed.
• No FHIR/HIS integration is included.
• ClinicalRedactionProfile does not guarantee complete de-identification, HIPAA Safe Harbor compliance, GDPR compliance, medical-device compliance, or absence of residual identifiers.

Full details are available in CHANGELOG.md.

fa-redact v0.2.0

fa-redact v0.2.0

Second public alpha release of fa-redact, a privacy-first, zero-runtime-dependency Python toolkit for Persian/Iranian PII detection, redaction, and pseudonymization.

Highlights

• Iranian IBAN / Sheba validation and detection
• Conservative ASCII email validation and opt-in detection
• Conservative 16-digit Bank Card / PAN validation and opt-in detection
• Configurable PatternRule / PatternDetector support for institution-specific identifiers such as MRNs and Patient IDs
• Explicit detection conflict-resolution policies: reject, longest, and priority
• Improved release artifact and version-consistency verification

Compatibility

• Python 3.10+
• Zero runtime dependencies
• Existing Iranian National ID and mobile-number behavior remains available
• Iranian IBAN is included in the default detector set
• Email detection remains opt-in
• Bank Card / PAN detection remains opt-in
• Configurable institutional identifiers remain opt-in
• Default conflict policy remains reject
• detect() continues to preserve raw overlaps and duplicate detections

Important limitations

• fa-redact does not provide complete automated clinical de-identification or anonymization.
• Email validation is syntactic only and does not verify mailbox existence.
• Iranian IBAN validation verifies format and checksum only; it does not verify account existence or status.
• Bank Card / PAN validation uses the Luhn checksum only and does not verify issuer, ownership, card status, or Iranian origin.
• PatternDetector rules are institution-specific trusted configuration; there is no universal MRN or Patient ID format.
• Non-reject conflict policies are explicit heuristics and can discard overlapping detections.
• Using fa-redact does not by itself establish GDPR, HIPAA, or other regulatory compliance.
• PseudonymizationSession.mapping contains original sensitive values and must be protected accordingly.

Installation

pip install fa-redact

See the bilingual README and CHANGELOG for detailed API usage and limitations.

fa-redact v0.1.0

fa-redact v0.1.0

First public alpha release of fa-redact, a privacy-first Python toolkit for Persian/Iranian PII detection, redaction, and pseudonymization.

Highlights

• Position-preserving Persian and Arabic-Indic normalization
• Iranian National ID checksum validation and detection
• Iranian mobile-number validation and detection using the bundled 2026 CRA numbering-plan snapshot
• High-level detect() API
• Typed placeholder-based redact() API
• Stateful PseudonymizationSession with stable aliases and local restoration
• Session-local sensitive mapping for AI/LLM workflows
• Structural custom-detector support
• Fully typed package with py.typed
• Zero runtime dependencies
• Python 3.10–3.13 CI coverage

Privacy and scope

The default detector set currently covers Iranian National IDs and Iranian mobile numbers only.

fa-redact does not yet provide complete clinical-text de-identification or full anonymization, and use of the package does not by itself establish GDPR, HIPAA, or other regulatory compliance.

PseudonymizationSession.mapping contains original PII and must be treated as sensitive data.

Installation

pip install fa-redact

See the repository README and CHANGELOG for details.