Releases: DataFog/datafog-core
Release list
DataFog Core 0.4.1
DataFog Core 0.4.1 adds three default credential detectors across Rust, Python, Node.js, and browser WASM:
API_KEY: documented GitHub token formats and Stripe secret/restricted keys.BEARER_TOKEN: explicit Authorization headers, quoted JSON headers, and immediate structured Authorization fields; findings cover the token only.CREDENTIAL_URI: scoped password-bearing PostgreSQL connection URIs; findings cover the complete original URI.
Rust and Python capabilities now report 23 supported entities and 14 default text detectors. Capability contract version 1, public binding signatures, finding schemas, exceptions, and original-input byte/code-point/UTF-16 offsets remain unchanged. German detectors and UUID retain their existing opt-in activation. Detection is lexical; no provider authentication or network checks occur.
Packages:
The DataFog Python 4.9 capability adapter consumes these labels without a duplicate inventory update. Its default backend remains unchanged. Legacy overlap resolution still runs before label filtering: API_KEY can suppress a same-span BEARER_TOKEN, BEARER_TOKEN can suppress JWT, and a containing CREDENTIAL_URI can suppress an inner key/JWT. Native datafog.v5 retains the candidates and supports selecting an individual label before transformation overlap resolution.
Validation includes Rust formatting, strict Clippy and all-feature workspace tests; installed Python 3.10–3.14 and source-distribution rebuild; installed Node and real Chromium WASM; Mintlify content/anchors; 352 Python 4.9 adapter/compatibility tests, 10 credential-overlap fixtures and 60 native transformations against the exact candidate. Clean registry installations were verified after publication.
Release source: 175e67548f654c740352a7d909ae55d376dfcb0d. Implementation and candidate PRs: #33, #34, #35, #36. Detailed scopes and integration evidence.
DataFog Core 0.4.0
DataFog Core 0.4.0 adds runtime capability discovery and expands native detection across Rust, Python, Node.js, and browser WASM.
- Rust and Python
capabilities()expose the actual detector registry: 20 supported entities, 11 default text entities, locale additions, and activation metadata. - Added JWT, complete PEM private keys, context-labeled US routing numbers and NPI, plus opt-in UUID detection.
- Seven German entity types remain locale opt-in. Unsupported explicit locales now raise the existing configuration error; German aliases and base-only
en-US/frare documented. - The 0.4.x compatibility policy preserves existing binding signatures, required result fields, exceptions, and offset semantics. Additive entities/locales/APIs are compatible; detector improvements can change findings. Breaking binding changes are reserved for 0.5.0.
Packages:
The Python capability adapter is tracked separately in DataFog Python PR #179. This Core release does not change Python's default backend. The legacy Python overlap policy can retain PHONE over a same-span NPI; native Core scanning retains both findings.
Validation: Rust formatting, strict Clippy, all-feature workspace tests; installed Python 3.10–3.14 wheels and source rebuild; installed Node and Chromium WASM suites; Mintlify content/anchors; 352 Python adapter checks against both candidate and published wheels. Registry-only Rust, Python, Node, and Chromium WASM installs passed smoke verification. Release source: 133bcef.
DataFog Core 0.3.0
DataFog Core 0.3.0 adds structured PERSON discovery and improves performance
when processing many findings. Rust, Python, Node.js, and browser WASM share
the detection and stateless protection behavior.
Structured PERSON support
- Discover explicit name fields such as
first_name,last_name, and
fullNamewithout a model or dictionary download. - Supply concrete JSON Pointer mappings for fields that automatic discovery
leaves unresolved, such as/customer/name. - Scan JSON string values with the original seven detectors and return each
finding with its field path. Protect the findings with the existing policies. - Use structured pseudonymization, tokenization, and restoration through
Rust, Python, and Node provider integrations. Browser WASM retains its
existing restriction on provider-backed operations.
PERSON detection uses field context. It does not recognize arbitrary names in
prose, and passing serialized JSON to scan(text) does not enable discovery.
See Discover and protect person fields.
Performance
- Index duplicate findings and resolve ordinary overlaps with ordered interval
selection. Built-in findings use an O(m log m) selection path, where m is the
finding count. - Reuse lazy text indexes during validation and byte/code-point/UTF-16 range
conversion, maintain running output positions, and avoid copying a whole
field for every Node transformation record. - Expose Rust's reusable
TextIndexfor converting multiple ranges from the
same string. See the Rust reference.
On one local macOS ARM64 benchmark, selection of 4,096 disjoint findings fell
from 60.3 ms to 0.56 ms. Separately, the bookkeeping changes reduced a complete
Node structured scan-and-protect request with 1,024 findings in one Unicode
field from 242.3 ms to 3.2 ms. These measure different stages and baselines;
they are not a universal speedup guarantee. Short-field workloads changed
little, and one sparse scan-only case was about 10% slower.
The selection benchmark notes
and bookkeeping benchmark notes
include inputs, methodology, and limitations.
Compatibility and publishing
Existing validation, offset semantics, transformation policies, and finding
preferences are preserved. Caller-supplied overlapping findings that mix
scored and unscored confidence can retain the original quadratic selection
algorithm to preserve its behavior. See Findings and ranges.
The Node and WASM release workflows publish to npm from GitHub Actions using
trusted publishing. Node builds cover macOS ARM64/x64, Linux GNU ARM64/x64,
and Windows x64. Node.js 24.x remains required.