Repository navigation
0.1.4
Another performance release on the parsing path. No behavioral changes; same 275 tests pass; RBS still validates.
Changed
-
Parsing is ~36% faster again on top of 0.1.3. Combined with the chars-Array fix in 0.1.3, parsing is ~31× faster than 0.1.0.
Avo bench (278 KiB / 67k tokens / 2080 rules):
tokenize parse_stylesheet 0.1.3 406 ms 477 ms 0.1.4 261 ms 286 ms Two independent wins:
CSS::CodePointspredicates use ASCII lookup tables. The four predicates (digit?,hex_digit?,ident_start_code_point?,ident_code_point?) now use frozen 128-element boolean tables keyed byc.ord; non-ASCII (c.ord >= 0x80) is always treated as ident-cp per spec. Replaces a chain ofString#<=>range tests that were 12.8% of tokenize time.ident_code_point?drops to ~4%.Positionis built lazily. The tokenizer plantsstart_offset/end_offset/ a shared@newlinesreference on each Token via the newToken#assign_source!;Token#positionmaterializes thePositionData on first read viabsearch_index. Eliminates ~67kPositionallocations from the hot path along with the per-token line/column work. Worst case (every token's position queried) tokenize takes 319 ms — still 22% under the 0.1.3 baseline.
-
Internal: small simplifications.
CodePointstable builders share abuild_tablehelper.Token#inspectno longer triggers lazyPositionmaterialization as a side effect. Tokenizer caches@chars.lengthas@length. Eager nil-init of the lazy-position fields removed for cleaner object shape (better for YJIT inline caches).
Removed
Token#assign_position!was never called outside its own definition. Hand-built tokens with a knownPositionshould pass it throughToken.new(:type, value, position: pos)instead. The RBS sig now advertisesToken#assign_source!in its place.
Compatibility
Same as 0.1.0 — Ruby 3.4+, no mandatory runtime dependencies.