goldenmatch v3.4.0
Immutable
release. Only release title and notes can be modified.
Added
- Embeddings as first-class Fellegi-Sunter scorers.
embedding/
record_embeddingfields train and score on the FS path (vectorized EM
E-step + block scoring); they were unusable before (both EM training and the
scalar scorer raisedUnknown scorer). Model-backed scorers always run
vectorized —GOLDENMATCH_FS_VECTORIZED=0only affects string scorers. The
TUI engine now routes probabilistic matchkeys through the block-scorer
selector, so it gets native/vectorized FS too.
Fixed
- Term-frequency (
tf_adjustment) now applies on the scalar FS path, matching
the vectorized path — the same config no longer scores differently depending on
route (a model-backed scorer orGOLDENMATCH_FS_VECTORIZED=0forces scalar). - Distributed and chunked lanes score probabilistic matchkeys. They
previously dropped FS pairs silently, then (mid-cycle) failed loudly. Both
lanes now score FS against ONE sharedEMResult: the distributed driver
trains once before dispatch (or loadsmk.model_pathdriver-side;
GOLDENMATCH_DISTRIBUTED_FS_TRAIN_ROWSbounds the training sample), and the
chunked lane trains once on the first chunk. The loud
NotImplementedErrorremains only for the bare scoring kernel invoked with
no model source. - Memory-bounded FS scoring for strategy-generated blocks. Probabilistic
matchkeys withlsh/ann/learned/canopy/sorted_neighborhood
blocking now score throughscore_probabilistic_external_blocks(one block
resident at a time, exclude-set Arc handle built once, oversized auto-split,
native/vectorized selection) instead of the batched scorer's all-units
accumulation.GOLDENMATCH_FS_DEFAULT_BUCKET=0still selects the legacy
batched scorer and now logs a warning (documented in tuning). - The columnar opt-in no longer demotes FS to the batched path. The
columnar branch is structurally weighted-only, so probabilistic matchkeys
keep the memory-bounded bucket route underGOLDENMATCH_COLUMNAR_PIPELINE=1. - FS bucket lane auto-splits oversized blocks (default
skip_oversized=Falsepath) instead of scoring them whole, and the
vectorized scorer refuses impossible dense allocations with an actionable
error (GOLDENMATCH_FS_VEC_MAX_ELEMS). - Missing values are unobserved evidence in FS scoring (null field values
carry no likelihood contribution instead of folding into total disagreement);
persisted v1 models without a training manifest are rejected on reuse. - Multi-pass EM conditions per pair and blocking fields keep the fixed
prior (repairs a recall collapse on near-unique blocking fields). - Per-field blocking transform chains (
BlockingKeyConfig.field_transforms);
the Splink converter maps mixed SUBSTR blocking rules exactly.