Skip to content

0.3.0

Choose a tag to compare

@iliaal iliaal released this 10 Jul 02:17
· 122 commits to master since this release
0.3.0
e4fd2ea

Fixed

  • double_metaphone(): four rules now follow the published Philips algorithm (thumb 0M|TM, high/hugh H, jose HS|HS, mooch MK).
  • double_metaphone(): leading/trailing whitespace (any byte <= U+0020) is trimmed, so "\tOtto" and "\nKNIGHT" keep their word-start anchors.
  • double_metaphone(): -gier uses the French soft-G branch anywhere, not only word-final, so brigier codes PRJ|PRJR and angiera codes ANJR|ANJR.
  • double_metaphone(): malformed UTF-8 folds byte-wise instead of swallowing the letters after a stray lead byte.
  • UTF-8 decoding (all encoders): overlong, UTF-16 surrogate, and above-U+10FFFF sequences fold the lead byte to Latin-1 rather than decoding as a code point.
  • bmpm() and dm_soundex(): U+1E9E (ẞ) folds to ß, so STRAẞE encodes like Straße.
  • bmpm_match(): token intersection handles the (remainder)-(combined) prefix group syntax, so bmpm_match("van Smith", "Smith") is true.
  • bmpm() / bmpm_match(): a forced $language is honoured inside the prefix/d' split (deliberate divergence from Commons Codec).
  • bmpm() / bmpm_match(): the invalid-$language exception preview is binary-safe (non-printables as ?) and bounded to 32 bytes.
  • bmpm(): leading/trailing control characters (<= U+0020) are trimmed.
  • dm_soundex_match(): two inputs that match no rule no longer compare equal via the padded "000000" sentinel.
  • U+0130 (İ) lower-cases to i in bmpm() and dm_soundex(), so İstanbul matches istanbul.

Security

  • match_rating() and match_rating_compare() no longer write past the clean buffer on a bare Latin-1 0xDF that expands to SS.
  • bmpm() and bmpm_match() reject input over 4096 bytes with a ValueError, matching the Daitch-Mokotoff cap.

Changed

  • The five comparison helpers' parameters are named $a/$b (affects named-argument calls only).
  • composer.json license is the SPDX expression BSD-3-Clause AND Apache-2.0 (was a disjunctive array).
  • The README BMPM token-splitting recipe splits on (, ), |, - with PREG_SPLIT_NO_EMPTY, matching bmpm_match()'s tokenizer.
  • Performance: bmpm() decodes input once and pre-parses rules at init; Double Metaphone, Daitch-Mokotoff, and NYSIIS drop per-call allocations.

For contributors

  • macOS CI builds with --enable-phonetic-dev, enforcing the same warning-clean build as Linux.
  • Generated BMPM table decoding fails module init if a checked-in table exceeds the C buffer caps.
  • CI preflights extension_loaded("phonetic") and fails on any skipped/borked/leaked test, so a silent load failure can't pass green.
  • scripts/pie-smoke.sh treats a pie install failure as fatal (opt out with ALLOW_MANUAL_FALLBACK=1).