Consider providing a way to keep emoji sequences joined

[drott]

(Spun off from https://crbug.com/774302)

The sequence of used fonts in Chrome is the font-family: font stack, after which emoji segmentation is used to determine a fallback priority, and use an emoji font as the first attempted fallback font for an emoji sequence, then perform other system fallback. This allows specifying a custom emoji font in the font stack.

If a font in the font stack has glyph coverage for symbols that are part of an emoji sequence, they get shaped with the font appears earliest in the font stack. For example, Arial has coverage for the male and female sign characters.

Thus, emoji sequences are broken up as HarfBuzz does not consider those sequences a full grapheme cluster. In Chrome, we rely on HarfBuzz clusters as the unit of fallback. However, if HarfBuzz breaks up the emoji sequence, this fallback mechanism is suboptimal.

For example:

$ ./hb-shape /usr/share/fonts/truetype/msttcorefonts/Arial.ttf \
`../test/shaping/hb-unicode-encode U+1F481,U+1F3FB,U+200D,U+2642,U+FE0F`
[.notdef=0+1536|.notdef=0+1536|space=0+0|male=3+1536|space=3+0]

So, HarfBuzz returns two clusters, starting at character index 0 and at character index 3.

Whereas, when shaped with an emoji font:

$ ./hb-shape ~/.local/share/fonts/NotoColorEmoji.ttf  \
`../test/shaping/hb-unicode-encode U+1F481,U+1F3FB,U+200D,U+2642,U+FE0F`
[gid2101=0+2550]

The shaping result comes back with one cluster, starting at character index 0.

Keeping in mind what the user expects to see, for shaping fallback we should consider emoji sequences as a whole unit of fallback, even though they are not necessarily defined as grapheme clusters by Unicode (to be clarified).

We should probably not break into multiple HarfBuzz clusters all those sequences that are defined as an xpicto-sequence in UTR #29 Text Segmentation.

[behdad]

After talking to Dominik, I agreed implementing xpicto-sequence in HarfBuzz makes sense.

[khaledhosny]

HarfBuzz does not currently know anything about grapheme clusters and HarfBuzz clusters only map glyphs back to input characters and the grouping depends on the layout process not the character properties (mostly).

I think Chrome should do the check for grapheme clusters itself, i.e. if a character is unsupported, go back and forward to the grapheme cluster boundaries and try the next fallback font for the full grapheme cluster not just that specific character. This way Chrome has full control on how fallback happens and it can tailor Emoji handling if needed. This would allow Chrome to switch to HB_BUFFER_CLUSTER_LEVEL_MONOTONE_CHARACTERS in the future (which allows things like coloring combining marks differently from their bases, which I think not possible in Chrome right now) without breaking the fallback logic.

[khaledhosny]

Oh, didn’t see @behdad’s comment before posting mine.

[behdad]

Khaled, I agree with you, and I was reluctant to allow this. However, this solves another problem for us that I didn't have a good answer: currently if an emoji sequence happens in RTL context, we flip it and break the ligature formation. For this reason, NotoColorEmoji includes GSUB rules for both directions. If we form a grapheme of these (which, after reviewing new UTR#51, is not that complicated), then they'll flip as a unit and things will apply fine.

We already attach marks to base. Already handle Emoji_Modifier=Yes (Fitzpatrick). I think we already do jamo, though looks like not in hb-ot-shape.cc level (so RTL jamo will be wrong). Should fix that. Adding xpicto-sequence is the big missing part. While I'm grumpy about having to import a table and maintain it, it's not a big deal, and will solve things for all clients.

[khaledhosny]

Would it still work with HB_BUFFER_CLUSTER_LEVEL_MONOTONE_CHARACTERS? I feel that we are attaching more (conflicting?) meanings to HarfBuzz clusters, may be it is time for having another public field in hb_glyph_info_t? (not sure it is still possible or even a good idea at this point)

[behdad]

Would it still work with HB_BUFFER_CLUSTER_LEVEL_MONOTONE_CHARACTERS?

No, there won't be any attaching there, unless it's RTL and we need to flip.

hb_glyph_info_t? (not sure it is still possible or even a good idea at this point)

Unfortunately that's not possible. I'm open to using cluster value differently though.

[drott]

Do you have any updates − Can we finish this in A Coruña? It'd be ideal to push it before the next branch point. Thank you!

[behdad]

Yes, let's figure it out next week.

…

On Wed, Sep 26, 2018 at 5:48 AM, Dominik Röttsches ***@***.*** > wrote: Do you have any updates − Can we finish this in A Coruña? It'd be ideal to push it before the next branch point. Thank you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1159 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIuK9bryU42WZK2PoNqqvfu_ef-wPhMks5ue01pgaJpZM4Wndvu> .

-- behdad http://behdad.org/