Weechat corrupts on Zero width character \u200b

[AIndoria]

Bug summary

Weechat corrupts itself on Windows Terminal (ssh'd to Weechat 3.4.1 running on screen on Ubuntu 20.04) seeing zero width character, \u200b (Refer to this code to see an example. Examples of corruption can be seen here, here, and here.

It remains even when you switch to another buffer, characters often remain on screen unless /redraw is pressed, but will do it again unless user clears the buffer by /buffer clear and then /redraw(or Ctrl+L).

Steps to reproduce

1.  Find a word with a ZWC like \u200b (or in this case, ・゜゜・。。 ​ ・゜゜\​_ö< quack​! )
2.  scroll up or down and/or type something/change buffers.
3.  Magic corruption.

Is there a way I can replace what appears in buffer as a temp solution BEFORE it is displayed to me while this gets fixed?

---

• WeeChat version: 3.4.1
• OS, distribution and version: Windows 10 21H2 ssh session into Ubuntu 20.04
• Terminal: Windows Terminal Preview Version: 1.13.10395.0, sshed into fish shell
• Terminal multiplexer (screen/tmux/…/none): screen

[flashcode]

Hi,

This is similar to the question in #1669.
Most of times, the problem is not WeeChat itself (which relies on glibc to know the char width), but either glibc returning wrong info or the terminal not displaying properly these chars.

I'll make some tests anyway to check if there's a problem in WeeChat.

[AIndoria]

Hm, do you know if I can use a script/setting to block ZWCs before they appear in the buffer for me? Suggestions?

Most of times, the problem is not WeeChat itself (which relies on glibc to know the char width), but either glibc returning wrong info or the terminal not displaying properly these chars.

I'm unsure if I can get Windows Terminal to change how it displays characters.

[flashcode]

You can remove such chars with a simple trigger like this:

/trigger add remove_zwc modifier weechat_print "" "/\u200b/"

This replaces \u200b with nothing (just add any char after the last slash if you need a replacement char/string).

By the way, are you able to reproduce the problem by printing yourself something like this?

/print -escape test\u200bhere

For me a space is displayed, but no screen corruption (in the terminals I tested: terminator and Windows Terminal).

[AIndoria]

Yep, the

/print -escape test\u200bhere

corrupts it. See:

I'll check out the trigger to remove the ZWC, ty.

[flashcode]

@AIndoria: I wrote a specification that solves this issue and some other with Unicode chars: https://specs.weechat.org/specs/2022-003-fix-unicode-display.html

Please tell me what you think about the proposed changes before I implement them.

I can make them available on a testing branch before merging into master.

[flashcode]

I pushed the branch unicode-fixes for tests: https://github.com/weechat/weechat/tree/unicode-fixes

Please ping me if you find differences with the specification or display bugs (chat and bars).

[AIndoria]

Thanks! I'll take a look :D

[trygveaa]

@flashcode: Not sure if it's worth fixing, but to be completely correct, I think WeeChat should break the line at a zero with space, if there's not enough space left for the next word. Otherwise, it looks good. I haven't verified the corruption though, as I don't have Windows Terminal to reproduce it.