Add an option to toggle the width of ambiguous-width characters

[tats-u]

This bug-tracker is monitored by Windows Console development team and other technical types. We like detail!

If you have a feature request, please post to the UserVoice.

Important: When reporting BSODs or security issues, DO NOT attach memory dumps, logs, or traces to Github issues. Instead, send dumps/traces to secure@microsoft.com, referencing this GitHub issue.

Please use this form and describe your issue, concisely but precisely, with as much detail as possible

• Your Windows build number: (Type ver at a Windows Command Prompt)
  10.0.16299.371

• What you're doing and what's happening: (Copy & paste specific commands and their output, or include screen shots)

1. Set font to ”MS Gothic” or a Japanese monospace font
2. Type ”■square□square” or another text that contains ambiguous-width characters like ”■” and ”┌”
3. Ambiguous-with characters lap over the next characters to them

• What's wrong / what should be happening instead:
  Some characters like ”■” and ”┌” has the same width as ASCII characters in latin fonts but double width in Japanese (and maybe some other languages) fonts. These characters are called ambiguous-weighted.

Other terminal emulator has the option to toggle the width of them but Windows Console doesn't.

EDIT: also see #16779

[omar2205]

Any updates on this issue?

[zadjii-msft]

Nope. We'll make sure to update this thread when there is. In the meantime, might I recommend the Subscribe button?

That way you'll be notified of any updates to this thread, without needlessly pinging everyone on this thread ☺️

[pwang2]

I was also pointed here when investigating a nerdfont icon cutoff case. ryanoasis/nerd-fonts#795

can we have the s_wideAndAmbiguousTable here to be configurable/overridable in settings.json

terminal/src/types/CodepointWidthDetector.cpp

Line 27 in c62705e

static constexpr std::array<UnicodeRange, 295> s_wideAndAmbiguousTable{

[j4james]

Unless I've misread something, the character discussed in that nerd-fonts issue was U+FD42, which is defined as neutral (which is essentially narrow), rather than ambiguous width, so this option wouldn't help. Also note that U+FD42 is in the Arabic Presentation Forms-A block, and is defined as ARABIC LIGATURE RADI. I really don't understand how Nerd Fonts feel entitled to redefine large chunks of Unicode and then still complain when something doesn't work.

[pwang2]

I don't think we need be nerve tense on this use of code block here. Nerd fonts is also an open source project which favored by a lot of people like me to get our nerdy life a little better giving this context that it is an aggregation of other open source fonts like Mdi, awesomefonts etc. It is not a problem of who get entitled, but just with a positive hope that things could be moving in a convergent way.

It is sad war day for the world. May the world peace as it deserves.

[Finii]

@j4james

I really don't understand how Nerd Fonts feel entitled

Well, the glyphs have been placed in PUA in error several years ago when it seemed to not matter. There are efforts to move the glyphs to an appropriate region. This legacy is hard to tackle, though, with a user base that expects glyph codes to not change.
But it is the top priority for the next major release.

and then still complain when something doesn't work

'Nerd Font' is not complaining. Often it is the user that uses the wrong 'flavor' of patched font that causes issues; with all the terminal applications behaving differently it is hard to determine if the patching itself is the problem or some setting, or ...
And then I poke in different applications' code to see how the fonts need to be patched differently to work on the majority.

Maybe you are familiar with Windows Terminals code? What I fail to find out, can the font supply some information here:
https://github.com/microsoft/terminal/blob/main/src/types/CodepointWidthDetector.cpp#L445
And I do not know if _pfnFallbackMethod is set to anything useful (and to what?) in production builds?

Maybe you have a minute for some answers :-) Thank you.

[j4james]

But it is the top priority for the next major release.

That's good to know. Thank you.

Maybe you are familiar with Windows Terminals code? What I fail to find out, can the font supply some information here:

No, I don't think that would help you. First off that only applies to ambiguous width characters, so it wouldn't effect something like U+FD42. And even if you were using an ambiguous width character, that fallback functionality is only used in conhost (the old Windows console), for legacy compatibility reasons.

A standard terminal shouldn't be varying the width of a character based on the font, because terminal applications would then have no way of knowing how much space a string of text was going to take. For example, an application wouldn't be able predicate when text is going to wrap, or how much to move the cursor when editing. If you don't get widths right, lots of things will break.