Repository navigation
Fix slow semantic coloring on long lines, including minified files. Token conversion now reuses its UTF-16 scan position while preserving Unicode byte positions, overlapping starts, and skipped spans.
A benchmark with 8,000 tokens on a 40 KB line fell from roughly one second to 0.38 ms on an M1 Max. This measures token conversion on that machine.
Verified with unit tests on Linux, macOS, and Windows, race tests on Linux and macOS, the protocol checker, and end-to-end editor tests. Update the Language Server extension from rhun’s Extensions page.