goldenmatch v3.6.0
Changed
-
Zero-config now routes probabilistic-shaped datasets to Fellegi-Sunter
by default (#1874). A dataset with no surviving strong-identity exact
matchkey (identifier/email/phone) and 2+ fuzzy fields is served by the
EM-weighted FS path instead of exact+weighted matchkeys -- measured F1
lifts on error-heavy PII (historical_50k 0.62 -> 0.78) with no regression
where a strong key survives. Kill-switch:
GOLDENMATCH_AUTOCONFIG_ROUTE_PROBABILISTIC=0. -
Config-healer loop cost is now bounded and tunable (#1404).
heal(and
thereview_config/suggest_from_resultverify path it drives) could run up
tostep_cap × (1 + max_verify)full pipeline passes per call — ~45 on the
defaults — plus a full goldencheckblocking_riskvariant scan (O(distinct²)
per string column) re-run on the unchanged frame every iteration. Three cost
levers, all defaulting to byte-identical behavior:- The goldencheck variant scan is memoized for the whole heal loop via a
newvariant_risk_cache()scope (core/suggest/adapter.py) keyed on the
data-column set — it runs once over the frame instead of once per iteration.
Output is unchanged; only the cost moves. - Verify fan-out is tunable:
review_config/suggest_from_result/heal
takemax_verify, andGOLDENMATCH_SUGGEST_MAX_VERIFYsets it globally
(default 8). Since the healer applies only the top surviving suggestion,
max_verify=1verifies just that candidate — the cheapest mode. - Marginal-gain early-stop:
heal(min_health_gain=…)/
GOLDENMATCH_HEAL_MIN_HEALTH_GAINstops the loop once cluster-health gain
flattens (fail-open — a result without clusters never triggers a stop). Off
by default.GOLDENMATCH_HEAL_STEP_CAPalso exposes the outer cap.
- The goldencheck variant scan is memoized for the whole heal loop via a
Added
DedupeResult.identity_summary(#1913). The per-run identity-
resolution summary (entities created/absorbed/merged) is now surfaced on
the public result --Nonewhen identity resolution is disabled. Backs the
in-Postgresgm_resolvewrite path ingoldenmatch-pg.- Small-N Fellegi-Sunter routing floor (#1947). Below
GOLDENMATCH_FS_ROUTE_MIN_ROWSrows (default 500) a probabilistic-shaped
dataset stays on the robust weighted path -- FS EM is data-starved at small
N and under-merges fuzzy-close variants.0disables the floor. Every
dataset that validated the FS default is far above it, so routing there is
unchanged.
Fixed
- Net-zero-evidence filter kills scale-growing FS over-merge (#1899,
default ON). Pairs whose only agreement is on absent (unobserved) fields
no longer accrue spurious match weight; ported to the numpy and native
kernels. - FS at 1M rows (#1896). A pairs-budget blocking gate stops a low-
cardinality pass from compounding into a megablock, and an Arrow pair-
stream plus EM block-sample cut the FS memory peak. - Unobserved
record_embeddingmasked out of the weighted score
(#1859). A record with no embedding no longer contributes max agreement. - Arrow-native auto-config blocking profile emitter (#1946). The
controller's sample iterations no longer force an arrow->polars round trip
just to count rows, so zero-config runs correctly (no degraded RED-sentinel
config) on a base, polars-free install. Byte-identical with polars present.