Repository navigation
Nomenklatura 4.16.0
This release is mostly performance work in the logic-v2 name scoring path, one scoring fix around stopword detection, and a retrain of both packaged models under scikit-learn 1.9.1.
Name scoring performance
name_matchcaches the consolidated name sets per entity (#363). Consolidation — the O(n²) containment pass that drops names subsumed by longer ones — ran again for every candidate pair, so a heavily merged entity paid for it once per query that pulled it in.entity_names_consolidatedmemoizes it alongsideentity_names; the literal-match early exit still uses the raw, unconsolidated set. Callers must not mutate the returned sets.- Per-name extras weights are precomputed once instead of rescanned per unmatched part.
weight_extra_matchwalked every span of the name on each call, which on a long single-name query (58 parts, 114 spans) is 716 calls and 81k span visits per candidate — nearly all returning 1.0.extra_match_weightsnow folds the weights into a dict keyed by span parts once per name, andweight_extra_matchbecomes a STOP-tag check plus a dict lookup. Spans whose category leaves the weight at 1.0 are omitted, so most person names yield an empty table. Scores are unchanged across all 826name_benchcases; the long-name fixture drops from 4.7 ms to 0.6 ms per candidate compare, ordinary inputs are flat. - The scoring parameters (
nm_extra_query_name,nm_extra_result_name,nm_family_name_weight) and theCompareConfigare read once per candidate and passed intomatch_name_symbolic, rather than re-read from theScoringConfigfor every name pair. weight_extra_matchcompares span parts by tuple equality instead of hashing the tuples. Spans and alignments share the sameNamePartobjects, so equality short-circuits on identity, and a hash collision can no longer apply the wrong weight.
Stopword detection fix
logic-v2name scoring uses theSTOPpart tag instead ofrigour.text.is_stopword. The wordlist lookup normalized its list with the slug profile while part forms come from name normalization, so diacritic stopwords such as "für" or "à" were never recognized. TheSTOPtag is assigned during analysis with matching normalization and defers to property-derivedGIVEN/FAMILYandLEGALtags, so a token that a property claims as part of the name is no longer discounted as filler. This changes scores for names containing diacritic stopwords and for property-tagged tokens that the wordlist previously treated as stopwords.
Models
regression-v1ander-unstableare retrained and repickled under scikit-learn 1.9.1, which removes theInconsistentVersionWarningraised when loading the 1.9.0 pickles. Coefficients move slightly: at most 0.020 forregression-v1and 0.0011 forer-unstable.
Compatibility
ScoringConfig.__hash__is removed. It serialized the model to JSON on every call and had no callers; without it, a non-frozen pydantic model is unhashable, so any cache keyed on aScoringConfignow fails loudly instead of paying for a JSON dump per lookup.match_name_symbolicno longer takes aScoringConfig. It takes theCompareConfigplus the three float weights directly.weight_extra_matchtakes the weight table fromextra_match_weightsinstead of theName.
Full changes: 4.15.0...4.16.0