[css-fonts-4] font-variant-emoji lacks fallback algorithm or outcome definition

[drott]

https://drafts.csswg.org/css-fonts-4/#font-variant-emoji-prop

This property is expected to affect font fallback; however, the exact nature of the interaction of font fallback with font-variant-emoji is explicitly unspecified. However, a variation selector must be included in a previous cluster as defined by the cluster matching section above.

Then further below:

text Code points are rendered as if U+FE0E VARIATION SELECTOR-15 was appended to every Presentation Participating Code Point.
emoji Code points are rendered as if U+FE0F VARIATION SELECTOR-16 was appended to every Presentation Participating Code Point.

I might be missing something, but I don't think Unicode defines the fallback algorithm for VS-15 and VS-16. To my understanding it's only well-defined for what glyphs should be selected within a font according to cmap format 14 in combination with the use of variation selectors.

Unclear sentence

"However, a variation selector must be included in a previous cluster as defined by the cluster matching section above"

I fail to understand what that means - any pointers?

Lack of algorithmic description or precise outcomes

So, if the font-variant-emoji property itself does not specify the fallback but relies on

"as if U+FE0F VARIATION SELECTOR-(15|16) was appended to every Presentation Participating Code Point"

we don't have any well-defined behavior here.

Can we find algorithmic proposals for fallback in combination with variation selectors? I think that would be a valuable addition to the spec.

If we can't succeed or agree on that, can we define a set of clear testable outcomes (preconditions and postconditions) or further details on how fallback should change with respect to this property?

[shivamidow]

I want to add a demo to Dominik's description, which is not directly related to font-variant-emoji. However, font-variant-emoji may follow the same font fallback algorithm of VS-15/16 handling, so it would help understand the current issue.

Please try https://jsfiddle.net/92jg6dLw/ with Blink, Gecko, and WebKit on Mac. The demo contains two smile emoji Unicode code points with VS-16, and we expect to see two color emojis. The result, interestingly, varies depending on user agents. The used fonts per engine are as follows.

• Blink: Times New Roman and Times New Roman
• Gecko: Noto Color Emoji and Apple Color Emoji
• WebKit: Apple Color Emoji and Apple Color Emoji

Gecko was my expectation initially, but I found other rendering engines also had a reason for their behavior while perusing the related spec.

Gecko follows the general rule of the fallback glyph lookup. Since Times New Roman does not support the variant smile emoji glyph, Gecko falls back to the next font, Noto Color Emoji, in the given font-family for the first emoji. Gecko applies the same rule to the second glyph, so it chose the system fallback font, Apple Color Emoji. This behavior may meet most web developers' expectations.

WebKit seems to follow the Cluster matching rule.

For sequences containing variation selectors, which indicate the precise glyph to be used for a given character, user agents always attempt system font fallback to find the appropriate glyph before using the default glyph of the base character.

So, Mac's system fallback emoji font, Apple Color Emoji, is selected for two emojis regardless of the font-family.

Blink's behavior might be the least wanted result in the user's view. And many related bug reports have been filed on crbug.com. It ignores VS-16 and uses the base codepoint glyph instead as the primary font does not support emoji variations. This might be based on the following description of the spec.

Variation selectors other than FE0E VARIATION SELECTOR-15 and U+FE0F VARIATION SELECTOR-16 must not have any affect on font selection. If one of these variation selectors is present, but unsupported by the font previously selected, the variation selector is ignored.

I am trying to fix Blink to fall through the font-family and system fallback fonts as Gecko does, but I found the spec is inconsistent on the font fallback algorithm, as Dominik described. For examples,

This property is expected to affect font fallback; however, the exact nature of the interaction of font fallback with font-variant-emoji is explicitly unspecified. However, a variation selector must be included in a previous cluster as defined by the cluster matching section above. Therefore, the presence of a variation selector is treated as ancillary data to inform the font fallback routine.

So... does the spec specify font-variant-emoji's font fallback or not? Does the font-variant-emoji fallback follow the cluster matching rule?

Note: Different platforms have different conventions about how to handle emoji presentation sequences. A cross-platform UA might wish to follow the conventions of each individual platform, or it might wish to use the same approach on all platforms. When tasked with a request for an emoji style rendering, a UA might wish to disregard fonts which do not include color tables. A different UA might wish instead to use the same mechanical cluster fallback algorithm that it would use for any arbitrary cluster.

This sounds more puzzling to me. So, the spec says there is no rule for emoji font fallback, and it is up to UAs; they could follow system conventions or decide the fallback at their discretion. In other words, not specified?

What is the expected behavior for the demo? Personally, I lean toward Gecko's behavior. However, as I read so far, WebKit's behavior looks like a correct implementation. Assuming font-variant-emoji follows the cluster matching rule for VS-15/16, font-variant-emoji also implements WebKit's behavior, right? The spec needs to clarify this.

[shivamidow]

Personally, I lean toward Gecko's behavior. However, as I read so far, WebKit's behavior looks like a correct implementation.

Let me correct myself. Gecko's behaviors seem still correct. The algorithm below clarifies the font fallback procedure. The user font-family list should be respected first.

[drott]

"However, a variation selector must be included in a previous cluster as defined by the cluster matching section above"

I fail to understand what that means - any pointers?

Looking closer at UAX 29 I think what this means is that a variation selector extends a grapheme cluster and does not cause a cluster break, because extended grapheme cluster = [...] precore* core postcore*... and postcore is a production for [Extend ZWJ SpacingMark] and Extend here are all codepoints which have the character property Grapheme_Extend=yes. So those sequences are covered.

[svgeesus]

@drott we didn't get to this at TPAC. where are we on Lack of algorithmic description or precise outcomes, do you have a concrete and testable suggestion here?

[drott]

here are we on Lack of algorithmic description or precise outcomes, do you have a concrete and testable suggestion here?

Did you mean to ask "where are we...?"?

I find the wording:

"This property is expected to affect font fallback; however, the exact nature of the interaction of font fallback with font-variant-emoji is explicitly unspecified."

too broad and I think we otherwise already have tighter rules in the 5.3 cluster matching section, which @shivamidow also points out.

What I established in the comments above from UAX 29 is that the variation selectors belong to the clusters according to Unicode cluster rules and we specify cluster matching as an addition to fallback in the specification. (The integration of cluster matching with fallback and font matching algorithm could be improved in the spec, as a side note).

So IMHO a good outcome would be if we could lift the requirements here to saying something like: The property causes font fallback to behave as if the respective VS was appended (as in the values description), then font fallback, including the cluster fallback rules is performed, which must take the presentation preference / "appended" variation selectors into account.

[drott]

PS: I imagine, roughly, what should become testable:

• The primary font lacks emoji presentation but has text presentation in CMAP, but a fallback web font in the font-family list supports emoji => fallback font preferred.
• The primary font lacks text presentation but has emoji presentation in CMAP, but a fallback web font in the font-family list supports text presentation => fallback font preferred.

WPT testing is not very good at interacting with system fonts or having definitions for a fixed set of those. So anything where the fallback descends into system fonts is harder to test.

[svgeesus]

So IMHO a good outcome would be if we could lift the requirements here to saying something like: The property causes font fallback to behave as if the respective VS was intended (as in the values description), then font fallback, including the cluster fallback rules is performed, which must take the presentation preference / "appended" variation selectors into account.

That sounds good to me. Want to make a pull request?

[svgeesus]

Added the suggested text.