Skip to content

Nomenklatura 4.16.0

Choose a tag to compare

@pudo pudo released this 23 Sep 12:29
· 55 commits to main since this release

This release is mostly performance work in the logic-v2 name scoring path, one scoring fix around stopword detection, and a retrain of both packaged models under scikit-learn 1.9.1.

Name scoring performance

  • name_match caches the consolidated name sets per entity (#363). Consolidation — the O(n²) containment pass that drops names subsumed by longer ones — ran again for every candidate pair, so a heavily merged entity paid for it once per query that pulled it in. entity_names_consolidated memoizes it alongside entity_names; the literal-match early exit still uses the raw, unconsolidated set. Callers must not mutate the returned sets.
  • Per-name extras weights are precomputed once instead of rescanned per unmatched part. weight_extra_match walked every span of the name on each call, which on a long single-name query (58 parts, 114 spans) is 716 calls and 81k span visits per candidate — nearly all returning 1.0. extra_match_weights now folds the weights into a dict keyed by span parts once per name, and weight_extra_match becomes a STOP-tag check plus a dict lookup. Spans whose category leaves the weight at 1.0 are omitted, so most person names yield an empty table. Scores are unchanged across all 826 name_bench cases; the long-name fixture drops from 4.7 ms to 0.6 ms per candidate compare, ordinary inputs are flat.
  • The scoring parameters (nm_extra_query_name, nm_extra_result_name, nm_family_name_weight) and the CompareConfig are read once per candidate and passed into match_name_symbolic, rather than re-read from the ScoringConfig for every name pair.
  • weight_extra_match compares span parts by tuple equality instead of hashing the tuples. Spans and alignments share the same NamePart objects, so equality short-circuits on identity, and a hash collision can no longer apply the wrong weight.

Stopword detection fix

  • logic-v2 name scoring uses the STOP part tag instead of rigour.text.is_stopword. The wordlist lookup normalized its list with the slug profile while part forms come from name normalization, so diacritic stopwords such as "für" or "à" were never recognized. The STOP tag is assigned during analysis with matching normalization and defers to property-derived GIVEN/FAMILY and LEGAL tags, so a token that a property claims as part of the name is no longer discounted as filler. This changes scores for names containing diacritic stopwords and for property-tagged tokens that the wordlist previously treated as stopwords.

Models

  • regression-v1 and er-unstable are retrained and repickled under scikit-learn 1.9.1, which removes the InconsistentVersionWarning raised when loading the 1.9.0 pickles. Coefficients move slightly: at most 0.020 for regression-v1 and 0.0011 for er-unstable.

Compatibility

  • ScoringConfig.__hash__ is removed. It serialized the model to JSON on every call and had no callers; without it, a non-frozen pydantic model is unhashable, so any cache keyed on a ScoringConfig now fails loudly instead of paying for a JSON dump per lookup.
  • match_name_symbolic no longer takes a ScoringConfig. It takes the CompareConfig plus the three float weights directly.
  • weight_extra_match takes the weight table from extra_match_weights instead of the Name.

Full changes: 4.15.0...4.16.0

Relevant pull requests: #367, #368, #369, #370, #373.