Option to use system's widths in GTK3 GUI

[robert-ross]

GVim uses its own version of functions such as wcwidth. In many cases that's better but sometimes it's more important to be consistent with other programs such as VTE3 which is commonly used by several terminals. In particular, some text files that are properly aligned when using Vim in a terminal become messed up in GVim and vice-versa. Please add an option such as "systemwidth" that can be enabled on a modline to avoid disruption.

[robert-ross]

Oops, s/modline/modeline/.

[brammool]

GVim uses its own version of functions such as wcwidth. In many cases that's better but sometimes it's more important to be consistent with other programs such as VTE3 which is commonly used by several terminals. In particular, some text files that are properly aligned when using Vim in a terminal become messed up in GVim and vice-versa. Please add an option such as "systemwidth" that can be enabled on a modline to avoid disruption.

This is possible, but the choice has to be made at runtime. Define "USE_WCHAR_FUNCTIONS" when building.

…

-- It is too bad that the speed of light hasn't kept pace with the changes in CPU speed and network bandwidth. -- <wietse@porcupine.org> /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[chrisbra]

This is possible, but the choice has to be made at runtime.

you mean compile-time?

[brammool]

> This is possible, but the choice has to be made at runtime. you mean compile-time?

Yes. (I should read back what I write...)

…

-- BEDEVERE: And what do you burn, apart from witches? FOURTH VILLAGER: ... Wood? BEDEVERE: So why do witches burn? SECOND VILLAGER: (pianissimo) ... Because they're made of wood...? "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[robert-ross]

Thanks for the fast response. This request is for being able to choose at runtime since the default (without USE_WCHAR_FUNCTIONS) is usually a good thing. I could switch off between two gvim binaries on my own machines but requiring that would be onerous for other people to read the text files.

[brammool]

Thanks for the fast response. This request is for being able to choose at runtime since the default (without USE_WCHAR_FUNCTIONS) is usually a good thing. I could switch off between two gvim binaries on my own machines but requiring that would be onerous for other people to read the text files.

Before we look into making this a runtime option, can you please check that enabling the feature (and recompiling) has the desired effect? I don't think this build configuration has been tested much.

…

-- The acknowledged parents of reengineering are Michael Hammer and James Champy. When I say they're the "parents" I don't mean they had sex - and I apologize for making you think about it. I mean they wrote the best-selling business book _Reengineering the Corporation_, which was published in 1993. Businesses flocked to reengineering like frat boys to a drunken cheerleader. (This analogy wasn't necessary, but I'm trying to get my mind off that Hammer and Champy thing.) (Scott Adams - The Dilbert principle) /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[robert-ross]

Will do.

[robert-ross]

It didn't have the desired effect but it did help me determine why GVim and terminal Vim had different widths. In my case, characters that were introduced in Unicode 12 differed since I'm using the latest stable glibc release and glibc 2.29 only supports Unicode 11. Thus characters such as U+1F6D5 appear narrower in terminal Vim than GVim. A workaround could append a filler or space when wcwidth(c) is -1 yet c is in Vim's doublewidth. I also found differences in older characters but only in scripts that I'm not qualified to assess. For example the sequences U+0020 U+093F and U+0915 U+093F both appear wider in terminal Vim than GVim but I have no idea which rendering is preferable or how to make the widths consistent. I could attach an example text file if that would help.

[robert-ross]

After updating locale data files and running locale-gen, I'm getting excellent results from wcwidth, iswprint, etc. Now, USE_WCHAR_FUNCTIONS fixes issues with U+3248 to U+324F, U+32FF, and U+4DC0 to U+4DFF where the width should be 2 but commands such as 'echo strwidth("\u32FF")' return 1. Please update Vim's internal tables so USE_WCHAR_FUNCTIONS isn't necessary. U+32FF was a recent addition from Unicode 12.1 so it was probably going to be corrected eventually anyway. The other ranges don't satisfy Markus Kuhn's comments in http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c but his mk_wcwidth and GNU's wcwidth return 2 for them.

[chrisbra]

Unicode 12.1 is very recent, and hasn't been applied to Vim yet. However patch 8.1.1157 added support for Unicode 12. So what exactly are you missing?

[k-takata]

Unicode 12.1 added only one character U+32FF SQUARE ERA NAME REIWA.
I don't know about the rest of the characters: U+3248 to U+324F and U+4DC0 to U+4DFF.

[robert-ross]

The first range was added to Unicode 5.2 based on ARIB STD B24 (the Japanese Association of Radio Industries and Businesses). The last range was added to Unicode 4 to encode Yijing (the ancient Chinese text known as I Ching). It's reasonable to assign these characters a width of 1 but that's a misfit, at least on GNU/Linux systems.

[brammool]

After updating locale data files and running locale-gen, I'm getting excellent results from wcwidth, iswprint, etc. Now, USE_WCHAR_FUNCTIONS fixes issues with U+3248 to U+324F, U+32FF, and U+4DC0 to U+4DFF where the width should be 2 but commands such as 'echo strwidth("\u32FF")' return 1. Please update Vim's internal tables so USE_WCHAR_FUNCTIONS isn't necessary. U+32FF was a recent addition from Unicode 12.1 so it was probably going to be corrected eventually anyway. The other ranges don't satisfy Markus Kuhn's comments in <http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c> but his mk_wcwidth and GNU's wcwidth return 2 for them.

The table we use is http://www.unicode.org/Public/UNIDATA/EastAsianWidth.txt 3248 to 324F are ambiguous width, correct. 32FF is new and double width, I'll add it. 4DC0 to 4DFF are marked "N" which is single width, correct. I consider the Unicode specification the authorative version. If it is not correct this should be brought up with the Unicoce consortium. Otherwise it is a bug in wcwidth.

…

-- I AM THANKFUL... ...for a lawn that needs mowing, windows that need cleaning and gutters that need fixing because it means I have a home. /// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[robert-ross]

Agreed. There are some minor issues with defining USE_WCHAR_FUNCTIONS (need to #define _XOPEN_SOURCE on Linux before #include <wchar.h>, compiler warning about the doublewidth variable being unused, etc.) but nothing major. If desired, I'll write another ticket.

[chrisbra]

okay, closing then.