Repository navigation
Version 1.2.2
Compound-Word Splitting
Eliminate duplicate finality emits in the C++ DAWG navigator.
navigate_from_edge was calling accept(matched, true) twice at every
embedded | finality marker — once via the look-ahead in the main loop
body, and again at the explicit | branch (with a third firing site in
the trailing block when the input ran out mid-edge). FindNavigator's
accept is idempotent, but CompoundNavigator::accept spawns a fresh
recursive search per fire, so each duplicate produced a duplicate
combination in the output list. As a concrete measurement,
forsætisráðherrafundur returned 23 raw splits with 10 duplicates,
and ríkisstjórnarfundur returned 10 with 4. After the fix the
look-ahead is the single canonical accept site, the | branch and
trailing block just advance/return, and the same inputs now produce
13 / 13 and 6 / 6 splits — distinct counts unchanged, duplicates gone.
Drop the now-unneeded seen dedup in
Wordbase._iter_legal_compound_splits. It existed solely to absorb
the C++ double-fire and became dead code with the fix above.
Refactor slice_compound_word to early-return from a shared
generator rather than materialising the full candidate list and
discarding all but element zero.
Documentation
- Expand the slow-path comment in
Bin._last_part_is_defectivewith a
concretemáliðexample, explaining why the per-lemmalookup_id()
loop is necessary even after the surface-form fast path proves
bi-numericality. - Note in
serialize_combinations(dawgdictionary.cpp) why the
std::string → char[]copy is unavoidable — norelease()API on
std::string, small-string optimisation often inlines the buffer
into the struct, and the compiler can't elide across a destructor —
and is not worth optimising on this hot path.
Compatibility
No public API changes. Same wheel set as 1.2.1 — CPython 3.9+ via
abi3 (Linux x86_64, macOS x86_64 & arm64, Windows AMD64) plus
PyPy 3.11 wheels for the same four platforms, and a source
distribution.