Invisible character from YouTube chat copying

[alexherbo2]

Copy this line.  There is a character between [​].

[Screwtapello]

hexdump -C says:

00000000  43 6f 70 79 20 74 68 69  73 20 6c 69 6e 65 2e 20  |Copy this line. |
00000010  20 54 68 65 72 65 20 69  73 20 61 20 63 68 61 72  | There is a char|
00000020  61 63 74 65 72 20 62 65  74 77 65 65 6e 20 5b e2  |acter between [.|
00000030  80 8b 5d 2e 0a                                    |..]..|
00000035

The bytes between [] are E2 80 8B, which is the UTF-8 encoding of U+200B ZERO WIDTH SPACE. The h and l keys in Kakoune navigate by Unicode code-point, but not all code-points are visible on-screen, so sometimes the cursor disappears when a zero-width string is selected. Unicode combining characters (which are also zero-width) have the same effect: try pasting Z̭̯̹͈͢a̜̰͇̘̿̎͋ͥḽ̝̺̳̩̩̗g͎̺̩̞o̧ ̤̭̀ͬͧͅt͚̯͉̃̇̇ĕ̟͇͑xt into Kakoune and see what happens.

[lenormf]

Related #1447

[oc34n-1]

​

[Icantjuddle]

OOC what is "correct" behavior here?

[Screwtapello]

I think the expectation (at least in this comment field, and GUI text editors in general) is that nobody will ever edit text that contains a sequence of code-points they couldn't type from their regular keyboard. Once you start getting into the weirder parts of Unicode control sequences, like "right-to-left override" or emoji that are ligatures of other emoji, or (on some platforms) once you involve characters beyond the Basic Multilingual Plane, all bets are off and all kinds of weirdness can occur.

Kakoune's behaviour here isn't the weirdest thing I've seen, and it allows surgical editing of individual code-points which is probably what you want in a programming-oriented editor.

Possibly the nicest behaviour would be to temporarily expand zero-width characters to take an entire terminal cell (padding them with a space character) when selected so they'd be visible, but I have no idea how complex that would be to implement, and it would be a pretty niche feature.