Skip to content

v1.5.0 — TranscriptionValidator: grapheme clusters, Google/TTS, Wikimedia strip preset

Choose a tag to compare

View all tags
@joshdaugherty joshdaugherty released this 13 Apr 00:26
· 10 commits to main since this release
v1.5.0
520e540

Highlights

This release bumps dataset_version / npm 1.5.0 with no allowlist row changes (schema_version remains 1.0.0). The PHP TranscriptionValidator API gains optional extended grapheme cluster validation (ICU via ext-intl), optional Google/TTS normalization aligned with Wikimedia IPAValidator, and a slash/bracket-only delimiter preset matching Wikimedia $stripRegex.

Added (PHP / Composer)

  • TranscriptionValidator::SEGMENT_GRAPHEME_CLUSTER — walk the post-pipeline string with IntlBreakIterator::createCharacterInstance(); each scalar inside each cluster must still be allowlisted (Option A). Use TranscriptionValidator::graphemeSegmentationAvailable() to detect ext-intl. New segmentationMode parameter on fromDisk() and the constructor (inserted before $validateSchema). composer.json suggest includes ext-intl.
  • googleTtsNormalization — Google/TTS branch: parentheses removal, modifier-letter → ASCII map, then strip combining marks U+0300–U+036F; requires wikimediaLegacyAscii. applyGoogleTtsNormalization() for reuse in tests.
  • STRIP_DELIMITERS_WIKIMEDIA_SLASH_BRACKETS and wikimediaSlashBracketStripScalarSet() — strip only / [ ] before normalization.

Changed

  • meta.dataset_version set to 1.5.0 in data/inventory.json, data/inventory.phonetic-strict.json, and data/normalization.json (fields stay aligned).
  • Compare scripts use User-Agent ipa-unicode-inventory-compare/1.5.
  • TranscriptionValidator: removed dead mb_str_split() === false checks (PHP 8.0+ always returns an array).

Install

  • Composer: composer require joshdaugherty/ipa-unicode-inventory (tag v1.5.0 after publish).
  • Node (maintainers / CI): npm ci / 1.5.0 in package.json.

Assets checklist

Attach npm run build outputs for consumers who do not use Node (per README / CONTRIBUTING), e.g. minified inventories, PCRE fragments, manifest.json, and generated php/AllowedCodePoints*.php, plus docs/mediawiki-parity.md (and log) if your release workflow does not already upload them.

Full changelog

See CHANGELOG.md — section [1.5.0] - 2026-04-16.

Assets 2
Loading