Skip to content

goldenmatch v2.7.0

Choose a tag to compare

@github-actions github-actions released this 02 Jul 16:41
Immutable release. Only release title and notes can be modified.
999875d

Added

  • Group/list-attribute demotion for EXACT matchkeys on single-source person data — now DEFAULT ON. A shared GROUP/LIST/FACILITY value — a shared switchboard phone line, a mailing-list / campaign identifier (tl_id), a facility NPI, a role email inbox — is not a person-identity claim: as an exact matchkey (exact_phone, exact_tl_id) it force-merges every DIFFERENT person sharing it into one mega-cluster. autoconfig_discriminative.should_demote_attribute_field demotes such an exact matchkey to blocking-only when the LARGE shared-value groups do NOT co-agree on the person name. Design: (1) group-size-aware — co-agreement is measured over large shared-value groups only (≥10 records), so a real personal id that only ever groups a person's few duplicates is KEPT (insufficient support), while a campaign list / facility id / role inbox that groups many different people is demoted — this also stops a mostly-unique column (a tl_id at 0.53 cardinality) from being rescued by its many small same-person groups averaging the signal up (measured: big-group name-power 0.01 vs small-group 0.80); (2) person-name basket — co-agreement is measured against person-name columns only, not the broad #1351 identity basket (polluted on real data by constant metadata mis-typed as identifier).
    • Scoped to EXACT uses only, deliberately. The accuracy sweep (scripts/autoconfig_quality) showed that demoting the same attribute from a weighted fuzzy contributor regressed corruption-heavy data: febrl3 F1 0.99→0.86, ncvr_synthetic 0.96→0.95 — its synthetic addresses also collide across people, but with the names corrupted the weighted address field is load-bearing for matching true duplicates. Restricting to exact matchkeys keeps that recall (a soft contributor never force-merges) while still killing the hard force-merges.
    • Default flipped ON in 2.7.0. The exact-only sweep showed zero F1 change across the whole corpus (febrl3 0.9921→0.9921, ncvr_synthetic 0.9636→0.9636, anchors unchanged — flag-on == flag-off on every dataset), while it fixes real-world group-attribute over-merges: measured on a real MJH dermatology list (19,278 rows), the shared-clinic-phone and campaign-tl_id exact matchkeys are both dropped, biggest cluster 70→6, clusters recovered 11,347→16,509. Kill-switch: GOLDENMATCH_ATTRIBUTE_DEMOTION=0 restores the pre-2.7.0 behavior. (PRs #1368, #1370.)