Box characters shown in composed Unicode sequences (e.g. in reactions)

[FrostyX]

It will probably be easier to just show you

What are they? At first I thought they are broken/missing user avatars but now it seems more like broken emojis?

The event looks like this:

#s(ement-event "$NktDguy2XVMKK7hOmENVtHCKaUnXmmOXfy7c_L7oxtU" #s(ement-user "@frostyx:matrix.org" nil nil "#f5d223" nil nil nil nil) ((body . "found it") (msgtype . "m.text")) 1679239320863 "m.room.message" ((age . 728) (transaction_id . "1627272823-1679239320")) nil nil ((reactions #s(ement-event "$It6A-6p5khKl_NGo0E_ToZA60MsxN8IJLfECuGZBYtU" #s(ement-user "@frostyx:matrix.org" nil nil "#f5d223" nil nil nil nil) ((m\.relates_to (event_id . "$NktDguy2XVMKK7hOmENVtHCKaUnXmmOXfy7c_L7oxtU") (key . "👍️") (rel_type . "m.annotation"))) 1679257345246 "m.reaction" ((age . 725) (transaction_id . "1627272843-1679257345")) nil nil nil) #s(ement-event "$hbatsB3HF6sG3bS73jY90RN6r_4uAC66s91gBO0iUn0" #s(ement-user "@frostyx:matrix.org" nil nil "#f5d223" nil nil nil nil) ((m\.relates_to (event_id . "$NktDguy2XVMKK7hOmENVtHCKaUnXmmOXfy7c_L7oxtU") (key . "👍") (rel_type . "m.annotation"))) 1679255972991 "m.reaction" ((age . 1306) (transaction_id . "1627272842-1679255972")) nil nil nil) #s(ement-event "$4hQQK_FLlITrY7tPvqPyu_phkCo0oRAOGM3na3PGbXw" #s(ement-user "@alphapapa:matrix.org" nil nil "#91ec55" nil nil nil nil) ((m\.relates_to (event_id . "$NktDguy2XVMKK7hOmENVtHCKaUnXmmOXfy7c_L7oxtU") (key . "👍️") (rel_type . "m.annotation"))) 1679239346101 "m.reaction" ((age . 983)) nil nil nil))))

It is displayed correctly here, but in my *Messages*, there are the squares as well

I did the Emoji configuration from readme

(setf use-default-font-for-symbols nil)
(set-fontset-font t 'unicode "Noto Emoji" nil 'append)

didn't help

[alphapapa]

If you use C-u C-x = you can see more information about the characters. It appears to happen for "composed" Unicode sequences (e.g. a hand with a skin-color variation selector). I don't know if this can be solved here, or if it's an issue to be solved in Emacs. If you'd like to investigate, it would be appreciated. :)

[RedChops]

FWIW, I cannot make emoji that have this variation from within Emacs. These characters only seem to occur on messages received. The only difference between a character I'd make which lacks the strange extra box or spaces, or one that is received that has them is the composition with 0xFE0F, Variation Selector-16:

             position: 2972 of 9597 (31%), column: 2
            character: 👍 (displayed as 👍) (codepoint 128077, #o372115, #x1f44d)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x1F44D
               script: emoji
               syntax: w 	which means: word
             category: .:Base
             to input: type "C-x 8 RET 1f44d" or "C-x 8 RET THUMBS UP SIGN"
          buffer code: #xF0 #x9F #x91 #x8D
            file code: #xF0 #x9F #x91 #x8D (encoded by coding system utf-8-unix)
              display: composed to form "👍️" (see below)

Composed with the following character(s) "️" using this font:
  ftcrhb:-GOOG-Noto Color Emoji-regular-normal-normal-*-18-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 128077 569 22 0 23 17 5 [0 0 136]]
with these character(s):
  ️ (#xfe0f) VARIATION SELECTOR-16

Character code properties: customize what to show
  name: THUMBS UP SIGN
  general-category: So (Symbol, Other)
  decomposition: (128077) ('👍')

Here is a ‘button’ button labeled ‘👍️ (1)’.


There are text properties here:
  action               ement-room-reaction-button-action
  button               (t)
  category             default-button
  face                 ement-room-reactions-key
  follow-link          t
  help-echo            [Show]

Also interesting is that in Emacs, that text looks like this:

So, I'm leaning more towards this being an Emacs display issue, but I don't really know where I'd start to try and figure out what can be done about it.

[alphapapa]

So, I'm leaning more towards this being an Emacs display issue, but I don't really know where I'd start to try and figure out what can be done about it.

Thanks. I'd recommend looking at the emacs-devel archives, because, e.g. Emacs 29 has additional support for emoji-related things, so there's likely to be relevant discussion there. The Emacs bug tracker will likely have more, too.

[Stebalien]

It's variation selector 16, or FE0F.

It looks like emacs should be downloading https://www.unicode.org/Public/15.0.0/ucd/emoji/emoji-variation-sequences.txt and adding those to the compose table, but it isn't.

This comment in the source makes me think emoji support isn't quite done.

This code seems to be manually adding FE0F support for some emoji, but not the ones in question...

[Stebalien]

Fix: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63731

[alphapapa]

@Stebalien You're like a bug-eating machine lately!

[Stebalien]

Well, in this case, it looks like Eli and Robert have taken it over with a better and much simpler patch. But yeah, that's what vacations are for.

[rpluim]

For those of you not wanting to wait for the emacs-29 release, putting the following in your init file should fix things:

  (set-char-table-range
   composition-function-table
   '(#xFE0E . #xFE0F)
   `(["\\c.[\ufe0e\ufe0f]" 1 compose-gstring-for-graphic]))

[alphapapa]

@rpluim Thank you, that works great! I'll add it to the documentation as well.