v1.5.0 — Auto-config verification layer
Highlights
Auto-config verification layer — new preflight + postflight modules give dedupe_df(df) with no args a runtime-safe warranty. Fixes the DBLP-ACM-style crash where auto-config emitted matchkeys referencing __title_key__ without enabling config.domain. Also addresses silent model downloads, useless exact matchkeys on unique IDs, and over-confident weighting of fuzzy fields.
What's new
Preflight — 6 checks before auto-config returns
- Column resolution — verifies every matchkey / blocking reference resolves; auto-repairs
config.domainfor domain-extracted column refs. - Cardinality bounds — drops exact matchkeys on columns with cardinality ≥ 0.99 (always unique) or < 0.01 (always same).
- Block-size sanity — warns when block-size distribution is pathological (P99 > 5000 or median < 2).
- Remote-asset demotion —
embedding/record_embedding/ cross-encoder rerank are demoted unlessallow_remote_assets=True. Offline CI is now safe by default. - Low-confidence weight cap — fields with profile confidence < 0.5 can't exceed weight 0.5.
Emits ConfigValidationError on unrepairable issues. PreflightReport attached to the returned config.
Postflight — 4 signals after scoring, before clustering
- Score histogram + bimodality detection → optional threshold nudge.
- Blocking recall estimate (gated, currently deferred).
- Preliminary cluster sizes with bottleneck-pair diagnostics.
- Threshold-band overlap → advisory to try
--llm-autoon ambiguous data.
PostflightReport attached to DedupeResult / MatchResult. Stable signals schema enforced by PostflightSignals TypedDict.
New kwargs
auto_configure_df(..., strict=False)— suppresses postflight adjustments for deterministic parity runs.auto_configure_df(..., allow_remote_assets=False)— opt-in to embedding / rerank (default off).
Classifier smarts
- Cardinality ≥ 0.95 →
identifier, notphone/zip/numeric. - New
yearcol_type routed to blocking, not scoring. - New
multi_namecol_type for comma/semicolon-delimited author-style fields. - Low-confidence fields cap at weight 0.3 in weighted matchkeys.
Breaking changes
ConfigValidationError—auto_configure_dfmay now raise on unrepairable issues.dedupe_df/match_dfzero-config paths let it propagate. Catch and inspecterr.report.findingsif you want a partial config.- Remote-asset scorers demoted by default. Pass
allow_remote_assets=Trueto keepembedding/record_embedding/rerank=Truein auto-generated configs. match_dfgained a zero-config code path it didn't have before (required for postflight on the match pipeline).
Examples
examples/verification_inspection.py— end-to-end walkthrough of preflight findings and postflight signals.examples/strict_mode_parity.py— deterministic parity runs withstrict=True.
Verification
- 1394 tests pass in the default suite.
- 4 integration tests (marked
@pytest.mark.benchmark) pass: DBLP-ACM regression, NCVR no-useless-matchkeys, Abt-Buy offline, frame-shape pin. --check-floorsgate: DBLP-ACM F1 0.97 ≥ 0.70, Abt-Buy no-LLM 0.44 ≥ 0.42, NCVR synth 1.00 ≥ 0.98.
Install
```bash
pip install --upgrade goldenmatch
```
Thanks
Review loop caught 13 spec + 6 plan + 11 implementation issues pre-merge. Post-merge review added 6 critical fixes + types/docs/tests polish. Full details in PRs #44 and #45.