v0.10.6
Fixed
-
Inline-walker columns are now UTF-16 code units, matching what LSP
clients expect by default. Two cursor walkers —
lex_core::lex::ast::inline_positions::position_at(semantic tokens- document links) and
lex_analysis::inline::ReferenceWalker::position_at
(find_references+goto_definition) — were accumulating
column += ch.len_utf8()for each char as they walked through a
line. For any char wider in UTF-8 than in UTF-16 (notably→at 3
bytes / 1 unit,§at 2 bytes / 1 unit, non-BMP emoji at 4 bytes /
2 units), every subsequent inline token's column was shifted right
bylen_utf8 - len_utf16. In VSCode this surfaced as the
open-backtick of an inline code span landing on the next glyph
(theeofSetupinstead of the`), and as
find_references/goto_definitionjumping to the wrong column on
any line that contained a→before the reference. Switched both
walkers tolen_utf16. Byte-levelRange::spanvalues are
unchanged — they remain UTF-8 byte offsets, which is correct.
Caveat for follow-up: the rest of the AST still records
Position.columnas a UTF-8 byte offset (see
SourceLocation::byte_to_position, and the deliberate slicing-by-bytes
inlex_lsp_core::available_actions::label_from_diagnostic_range).
Block-level ranges sent to LSP that span content with non-ASCII
characters (e.g., a session title containing a→) therefore still
have a similar shift in their end columns. The deeper fix is to
convert UTF-8 byte columns to UTF-16 at the LSP wire boundary; this
PR is scoped to the inline-walker bug that was visible in editor
rendering of paragraph text. - document links) and