Skip to content

v2.0.0: Table-Driven O(log n) Architecture

Choose a tag to compare

View all tags
@SCKelemen SCKelemen released this 16 Dec 20:20
· 58 commits to main since this release
v2.0.0
539a623

Performance Improvements

Version 2.0.0 focuses on performance optimization while maintaining 100% conformance with Unicode standards.

Table-Driven Binary Search

All packages now use table-driven O(log n) binary search for character classification, replacing sequential O(n) checks:

  • UAX #9: Bidi class lookup optimized with 3,060 precomputed ranges from DerivedBidiClass.txt
  • UAX #29: Unified packed data structure with 4,673 ranges encoding all three break types (grapheme, word, sentence) in 16-bit format

Performance: Character classification now runs at ~60-100 ns/op with 0 allocations on Apple M4 Pro.

Generated Unicode Data

All Unicode property data is now generated directly from official Unicode 17.0.0 data files:

  • Download from unicode.org during build
  • Parse property files (DerivedBidiClass.txt, GraphemeBreakProperty.txt, etc.)
  • Generate optimized Go code with binary search tables
  • Ensures correctness and synchronization with Unicode standard

Single-Pass API

UAX #29 provides a new FindAllBreaks() API that computes grapheme, word, and sentence boundaries in a single traversal.

Conformance

Maintains 100% Unicode conformance on all official test suites:

  • UAX #9: 513,494/513,494 tests passing
  • UAX #14: 19,338/19,338 tests passing
  • UAX #29: 3,222/3,222 tests passing (766+1944+512)
  • UTS #51: 5,223/5,223 tests passing
Assets 2
Loading