Backspace doesn't handle zero width joiners (and combining marks) correctly.

[brandf]

• VSCode Version: Code 1.10.2 (8076a19, 2017-03-08T14:02:52.799Z)
• OS Version: Windows_NT ia32 10.0.14393
• Extensions:

Extension	Author	Version
cpptools	ms-vscode	0.9.2

---

Steps to Reproduce:

1. paste this emoji: 👩‍👩‍👧‍👧
2. hit backspace

Expected: deletes entire emoji (like it does in this html form).
Result: deletes the right kid, leaving you with 👩‍👩‍👧‍.

I've had this same bug in my own code once. It's because javascript has derpy non-utf-8 strings. Single utf-8 characters will be multiple javascript string characters. I don't know the correct terminology, but just play with it and see what I mean. I found the correct way to delete character off javascript strings to be:

                    var chars = Array.from(text);
                    chars.pop();
                    text = chars.join('');

[brandf]

There are a number of articles about this topic, for example: https://mixmax.com/blog/unicode-woes-in-javascript

Sounds like anything in the Basic Multilingual Plane is ok because it's represented as a single 16 bit code point, but characters from the Supplementary Multilingual Plane are encoded as surrogate pairs, which is represented as two 'indexes' in Javascript strings.

[CoenraadS]

Maybe useful: https://github.com/orling/grapheme-splitter

(Happened to read https://news.ycombinator.com/item?id=13830177 a couple of days ago)

[brandf]

Having looked into emoji closer, it looks like the case above has to do with emoji's that use zero width joiners:
http://unicode.org/emoji/charts/emoji-zwj-sequences.html

The family 👩‍👩‍👧‍👧is 4 emoji 👩 👩 👧 👧 combined with zwj's (200d) into a single glyph:
http://unicode.org/emoji/charts/emoji-zwj-sequences.html#1f469_200d_1f469_200d_1f467_200d_1f467

If you take off the last utf-16 character you get a smaller family 👩‍👩‍👧‍+ a dangling zwi which is likely ignored:
http://unicode.org/emoji/charts/emoji-zwj-sequences.html#1f469_200d_1f469_200d_1f467

Given that VSCode's text stack is combining the sequence into a single glyph, the correct behavior is for backspace to remove the whole zwj sequence.

[brandf]

I just noticed something...in a Chrome textarea if you have "👩‍👩‍👧‍👧" and you backspace it will remove the whole glyph. The same thing in Edge just removes a kid. I would call that an Edge bug, the same as this VSCode bug. The Chrome behavior is more in line with the user intent since the zwj sequence was combined into a single glyph.

Note that Array.from handles zwj by giving an empty string:
Array.from("👩‍👩‍👧‍👧👩‍👩‍👧‍👧")
["👩", "‍", "👩", "‍", "👧", "‍", "👧", "👩", "‍", "👩", "‍", "👧", "‍", "👧"]

So it's pretty easy to get the correct behavior by simply pop'ing additional items until it's not empty.

Unfortunately combining marks can't be handled this way, and exhibit a similar behavior. Chrome backspaces them 'correctly' and Edge and VSCode don't.

For example:

Z͑ͫ̓ͪ̂ͫ̽͏̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ͫ͗͢L̠ͨͧͩ͘G̴̻͈͍͔̹̑͗̎̅͛́Ǫ̵̹̻̝̳͂̌̌͘!͖̬̰̙̗̿̋ͥͥ̂ͣ̐́́͜͞

requires like 50 backspaces to delete in Edge and VSCode but only 5 in Chrome.

[brandf]

Or maybe this is a Chrome bug, or they all have a bug...hard to say, unicode is hard. Consider this:

The string: "\u{1f469}\u{200d}\u{2764}\u{fe0f}\u{200d}\u{1f48b}\u{200d}\u{1f468}" is typically joined into a single icon: 👩‍❤️‍💋‍👨

So it feels like a single backspace would delete the whole thing. But note that the text stack wasn't required to combine them like that...it could have just concatenated the individual emoji.

In fact it will do just that if you reverse the man/woman (at the time of this writing) "\u{1f468}\u{200d}\u{2764}\u{fe0f}\u{200d}\u{1f48b}\u{200d}\u{1f469}" looks like 4 individual emoji on most systems: 👨‍❤️‍💋‍👩.

In that case, it feels like backspace should just remove the woman (and yet in Chrome it deletes all 4. In this case VSCode and Edge apparently do the 'right' thing.

I think the bestest behavior would be if backspace behaved dependent on if the zwj sequence was actually combined or not. In the first case backspace should remove the whole sequence, and in the second case it should only remove up to the last zwj. In the future if the text stack decides to join the second case like it does the first, then backspace should remove the whole thing.

[rebornix]

Duplicate of #7218.