Problems with certain emojis

[rightaway]

When I paste emojis into the terminal and delete them it mostly works fine, but with some emojis like the flag emojis (like 🇦🇫 🇦🇽 🇦🇱 🇩🇿 🇦🇸) pasting it pastes extra space characters, and trying to delete them results in some strange display issues, and having to press delete multiple times to delete a single emoji, and then I can't get to the beginning of the line to delete everything.

[kovidgoyal]

This is likely caused by issues in your shell not kitty. Try running cat and then pasting your flag emojis and deleting them, they will work fine. If not let me know.

[maximbaz]

It is reproducible in cat:

[kovidgoyal]

OK I'll take a look at it when I have a moment

[rightaway]

@maximbaz Thanks for double checking. How did you take that screen video?

[maximbaz]

Using https://github.com/phw/peek, but be aware of this bug phw/peek#331

[Luflosi]

I can also reproduce this bug. I think the unicode flag symbols consist of two special unicode characters next to each other. This would explain, why it's possible to press backspace twice and why there is a weird character when pressing backspace once.

[kovidgoyal]

Yeah I looked at it, its not realy a bug. flags are two unicode symbols. So when you pres backspace one of them get deleted, leaving the other behind. Sadly the unicode committee did not make flags combining symbols so that is expected behavior.

[maximbaz]

Pressing backspace twice is understandable, but see my gif, I cannot get rid of (A) emoji no matter how many times I press backspace

[kovidgoyal]

OK I'll open it again, this looks like its going to go on all night :)

[Snuggle]

This happens with other emojis too. It causes a ton of issues when needing to use emojis in a terminal. I can't backspace either, and some of the emojis don't actually exist if I try to echo them etc.

Here's a recording showing flag emojis at first and then normal single-character emojis

[kovidgoyal]

@Snuggle like I said initially, dont use a shell. Different shells have widely arying support for emoji. Try to reproduce the problem in cat. Flags are not emoji and have special problems. As far as I know all actual emojis work fine in kitty.

[ssoriche]

I get it with ✊in fish, cat, bash, weechat, and irssi. ☁️ Takes 2 presses of backspace is ok, but after the first press of backspace on ✊moves the cursor, after that nothing happens and the ✊ is still displayed.

[kovidgoyal]

Works fine for me in vim. But sure, I can probably look into making the erase_char operation detect wide characters and act intelligently on it. I just hope doing that does not screw it up in vim. Probably have to make it zero both cells and then move one cell back. That should work in both cat and vim.

[maximbaz]

The issue with flag and fist emojis is fixed on master, thanks 😉

I was wondering if in 7f0674a you should have removed removed all wide characters before the cursor, no only one wide character before the cursor.

---

Also if you are not sick of emojis yet, can you have a look at these ones?

• 👨‍👦 (U+1F468, ‍U+200D, U+1F466) - kitty requires pressing backspace twice even though there is a joiner
• 👨‍👧‍👦 (U+1F468, ‍U+200D, U+1F467, ‍U+200D, U+1F466) - similarly requires 3 times to press backspace

[kovidgoyal]

Why would I want to remove all wide characters? Pressing backspace once should remove one wide character, not all of them.

kitty does not support joiners beyon treating them as basic combining characters.

[blahgeek]

Commit 7f0674a breaks backspace for unicode characters in NeoVim for me:

With content “你好” in neovim, pressing backspace once would delete two instead of one character.

[kovidgoyal]

Still works in vim. This is what I was basically afraid of. There is no standard that I can find about what backspace is supposed to do with wide characters. Which means that no matter what choice I make, something will not work.

Presumably neovim expect the terminal to do what kitty did before this patch. I'm not sure how vim is able to handle both behaviors, probably it manages cursor position manually when dealing with wide characters to get around differences in terminal behavior.

Maybe @justinmk would like to comment on what neovim does.

[kovidgoyal]

Actually, I think I am going to revert this patch. Most terminal applications seem to expect the terminal emulator to not special case backspacing of emoji. They want to handle it themselves. That means that whether backspacing any particular wide character works or not depends entirely on whether the application you are running inside the terminal knows that the character is an wide character.

For instance without this patch:

with the raised fist emoji: backspacing works for me with up-to-date bash and zsh and vim and nvim
but not in cat and fish. Which tells me that fish is using an outdated wide character database.

With the patch, it works in vim and cat and fish but not in zsh, bash and nvim.

with the flag on the other hand, without the patch: it works in vim and nvim but not in any of the shells or cat. You will simply have to wait for the shells to update their wide character database to include the flags.

Given that terminal applications want to handle backspacing themselves, I dont think this is an issue that can be resolved in the terminal emulator.

[toonn]

Hmm, is this an instance of the same issue? Running cat (in fish, in kitty, on mac os). I insert an emoji 😄, then backspace, then insert another 🎄, the smiley remains visible, only when pressing enter the christmas tree becomes visible on the next line.

[kovidgoyal]

yeah cat is dumb, so it will never know how to backspace emoji