Skip to content

Version 1.2.2

Choose a tag to compare

@vthorsteinsson vthorsteinsson released this 15 May 00:48
· 80 commits to main since this release

Compound-Word Splitting

Eliminate duplicate finality emits in the C++ DAWG navigator.
navigate_from_edge was calling accept(matched, true) twice at every
embedded | finality marker — once via the look-ahead in the main loop
body, and again at the explicit | branch (with a third firing site in
the trailing block when the input ran out mid-edge). FindNavigator's
accept is idempotent, but CompoundNavigator::accept spawns a fresh
recursive search per fire, so each duplicate produced a duplicate
combination in the output list. As a concrete measurement,
forsætisráðherrafundur returned 23 raw splits with 10 duplicates,
and ríkisstjórnarfundur returned 10 with 4. After the fix the
look-ahead is the single canonical accept site, the | branch and
trailing block just advance/return, and the same inputs now produce
13 / 13 and 6 / 6 splits — distinct counts unchanged, duplicates gone.

Drop the now-unneeded seen dedup in
Wordbase._iter_legal_compound_splits.
It existed solely to absorb
the C++ double-fire and became dead code with the fix above.

Refactor slice_compound_word to early-return from a shared
generator
rather than materialising the full candidate list and
discarding all but element zero.

Documentation

  • Expand the slow-path comment in Bin._last_part_is_defective with a
    concrete málið example, explaining why the per-lemma lookup_id()
    loop is necessary even after the surface-form fast path proves
    bi-numericality.
  • Note in serialize_combinations (dawgdictionary.cpp) why the
    std::string → char[] copy is unavoidable — no release() API on
    std::string, small-string optimisation often inlines the buffer
    into the struct, and the compiler can't elide across a destructor —
    and is not worth optimising on this hot path.

Compatibility

No public API changes. Same wheel set as 1.2.1 — CPython 3.9+ via
abi3 (Linux x86_64, macOS x86_64 & arm64, Windows AMD64) plus
PyPy 3.11 wheels for the same four platforms, and a source
distribution.