Skip to content

fix(editor): two Monaco defaults that only hurt non-Latin scripts (#393) - #546

Merged
PathGao merged 1 commit into
masterfrom
fix/cjk-editor-defaults
Aug 8, 2026
Merged

fix(editor): two Monaco defaults that only hurt non-Latin scripts (#393)#546
PathGao merged 1 commit into
masterfrom
fix/cjk-editor-defaults

Conversation

@PathGao

@PathGao PathGao commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Two items from the audit in #393. Neither is a bug in Markpad — both are Monaco
defaults that are right for code in English and wrong for prose in a script
that behaves differently.

The space bar under a CJK IME is outlined

#462 turned off unicodeHighlight.ambiguousCharacters and closed #186 and #94
with it. The boxes came off the fullwidth punctuation and stayed on U+3000
IDEOGRAPHIC SPACE — which is what the space bar produces under every Chinese
and Japanese IME. Same reporters, same typing, same hover offering an Adjust
settings
button that leads nowhere in standalone Monaco.

That character is caught by invisibleCharacters, a different flag, and
allowedLocales cannot narrow it: strings.js getData() flattens every
locale bucket into one 465-entry set, so U+3000 is in it unconditionally. Only
allowedCharacters reaches it.

The flag itself stays on, deliberately — an NBSP after - stops a list from
parsing, and nobody types one on purpose. Worth recording that the comment
justifying it was wrong in the other direction: it claimed the highlighter
"never fires on something typed on purpose", and U+3000 is exactly that.

Word-wise navigation treats a Chinese clause as one word

⌥←/→, double-click-to-select and ⌥⌫ split on wordSeparators. A language that
puts no spaces between its words supplies none, so the whole clause is one
word: ⌥→ crosses the sentence, double-click selects it, ⌥⌫ deletes it.

wordSegmenterLocales switches on Intl.Segmenter, which knows where the
words are. 这是一个用于测试的段落 goes from one word to seven.

It reads like a whitelist and is closer to a switch. ICU dispatches its
dictionary breaking by SCRIPT, not by this list:

script in the list? segmented?
Chinese, Japanese yes yes
Thai, Khmer, Lao, Burmese, Tibetan no yes

and ['zh'], ['ja'] and ['zh','ja'] produce identical output — including on
Han text, where the two dictionaries might have been expected to differ. Pinned
in a test, because the obvious tidy-up is to trim the list to match the comment,
which would change nothing while making the code claim something false.

Space-delimited languages lose nothing. Korean, Vietnamese, Russian and English
segment word-for-word identically to splitting on whitespace — which is why
this can be on for everyone rather than keyed to the UI language. That would be
the wrong key anyway: it is the document's script that decides, not the
language of the menus.

Tests

Driven through Monaco's own code rather than by asserting the options are
spelled correctly — UnicodeTextModelHighlighter.computeUnicodeHighlights for
the first, getMapForWordSeparators (the function wordOperations.js and
cursorWordOperations.js both call) for the second. Each fixture is also
checked to still reproduce the original bug once the fix is removed, so a
passing test cannot mean the fixture stopped exercising anything.

Verified

By hand on macOS: the IME space is no longer outlined, ⌥←/→ and double-click
move by word in Chinese prose, and English in the same document behaves exactly
as before.

The segmentation results above were measured in Node's ICU. Dictionary breaking
is standard across ICU builds, but it was not exercised inside WKWebView,
WebView2 or WebKitGTK.

…scripts

Both come out of the audit in #393. Neither is a bug in Markpad — they are
Monaco defaults that are right for code in English and wrong for prose in a
script that behaves differently.

## The space bar under a CJK IME is outlined

#462 turned off `unicodeHighlight.ambiguousCharacters` and closed #186 and
#94 with it. The boxes came off the fullwidth punctuation and stayed on
U+3000 IDEOGRAPHIC SPACE, which is what the space bar produces under every
Chinese and Japanese IME — the same reporters, the same typing, the same
hover offering an **Adjust settings** button that leads nowhere in standalone
Monaco.

That character is caught by `invisibleCharacters`, a different flag, whose
465-entry set cannot be narrowed by `allowedLocales`: `strings.js` getData()
flattens every locale bucket into one set, so U+3000 is in it unconditionally.
Only `allowedCharacters` reaches it.

The flag itself stays on, and that is deliberate. An NBSP after `-` stops a
list from parsing, and nobody types one on purpose. Worth recording that the
comment justifying it was wrong in the other direction: it claimed the
highlighter "never fires on something typed on purpose", and U+3000 is exactly
that.

## Word-wise navigation treats a Chinese clause as one word

⌥←/→, double-click-to-select and ⌥⌫ split on `wordSeparators`. A language that
puts no spaces between its words supplies none, so the whole clause is one
word: ⌥→ crosses the sentence, double-click selects it, ⌥⌫ deletes it.

`wordSegmenterLocales` switches on `Intl.Segmenter`, which knows where the
words are.

It reads like a whitelist and is closer to a switch. ICU dispatches its
dictionary breaking by SCRIPT, not by this list: Thai, Khmer, Lao, Burmese and
Tibetan all segment without being named, and `['zh']`, `['ja']` and
`['zh','ja']` produce identical output — including on Han text, where the two
dictionaries might have been expected to differ. Pinned in a test, because the
obvious tidy-up is to trim the list to match the comment, which would change
nothing while making the code claim something false.

Space-delimited languages lose nothing: Korean, Vietnamese, Russian and
English segment word-for-word identically to splitting on whitespace. That is
why this can be on for everyone rather than keyed to the UI language — which
would be the wrong key anyway, since it is the document's script that decides,
not the language of the menus.

## Tests

Driven through Monaco's own code rather than by asserting the options are
spelled correctly: `UnicodeTextModelHighlighter.computeUnicodeHighlights` for
the first, `getMapForWordSeparators` — the function `wordOperations.js` and
`cursorWordOperations.js` both call — for the second. Each fixture is also
checked to still reproduce the original bug once the fix is taken away, so a
passing test cannot mean the fixture stopped exercising anything.

Verified in Node's ICU. The dictionary breaking is standard across ICU builds,
but it was not exercised in WKWebView, WebView2 or WebKitGTK.
@PathGao
PathGao merged commit 3a3bd55 into master Aug 8, 2026
4 checks passed
@PathGao
PathGao deleted the fix/cjk-editor-defaults branch August 8, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

there are highlighted boxes in v2.6.11

1 participant