Add an option to prefer decomposed forms

[khaledhosny]

As discussed on this Twitter thread.

Alternatively we can just prefer the form used in input if the font supports the characters, if this one form renders suboptimaly we can say it is a font bug.

[behdad]

Jonathan and I are discussing this now. I'm inclined to just change the default to prefer decomposed. Initially when we did this, it was to improve shaping with SBL Hebrew and older Latin fonts that lacked GPOS mark positioning. Maybe we can continue precomposing if font doesn't have GPOS. Just exposing more options is not best solution if there's no clear criteria for how the client of the library should set that option.

[khaledhosny]

I agree about not having an option.

[behdad]

@jfkthame any preference here?

[jfkthame]

Generally, I think I'd favor preferring the form as provided in the input, so that as far as possible harfbuzz stays out of the way of whatever the client & font are trying to do, rather than introducing additional magic.

But of course there are exceptions, where harfbuzz steps in to do something that wasn't explicitly requested by either the client or the font, but in practice often helps: fallback mark positioning is a great example. And precomposing accented letters if the font lacks GPOS seems like it fits in that category; it's basically a better alternative to fallback mark positioning.

When a font has a GPOS table, though, we should just use it and not try to second-guess the designer. If the GPOS table then fails to position accents well, that's a font bug.

[behdad]

Generally, I think I'd favor preferring the form as provided in the input, so that as far as possible harfbuzz stays out of the way of whatever the client & font are trying to do, rather than introducing additional magic.

I think it serves us better if HarfBuzz renders canonically-equivalent sequences the same independent of the font.

[jfkthame]

I think it serves us better if HarfBuzz renders canonically-equivalent sequences the same independent of the font.

Usually, I'd agree with that, but I don't see it as an absolute rule that is entirely the responsibility of the shaping engine.

(For example, there are the CJK Compatibility Ideographs at U+F900, which are canonically equivalent to other chars in the Unified block, but where a distinction in rendering must be maintained if we're to support text mapped from certain legacy encodings.)

For the case of Latin accents, I'm torn between the desire to render canonically equivalent sequences identically, the desire to allow clients to achieve effects such as mark coloring (dependent on decomposed rendering), and the desire to render decent-looking results with as many fonts as possible (often dependent on precomposed).

Consider me confused & conflicted.... :\

[behdad]

Consider me confused & conflicted.... :\

Same here. But yeah, in the interest of mark coloring, I like going in that direction.

[khaledhosny]

I think something like the following would be a good compromise:

• If input uses decomposed form, and the characters are supported by the font, and the font has a GPOS table, use them.
• Else, use the composed form, if supported by the font.

If after that canonically equivalent forms render differently, then it is a font bug (which can still happen with the current scheme, as seen in #1092).

[behdad]

So, basically: if GPOS available, prefer decomposed. Else, prefer composed. I think that makes sense. That said, our fallback positioning also kicks in if GPOS is not available. So, maybe always decomposed is fine...

[jfkthame]

That said, our fallback positioning also kicks in if GPOS is not available. So, maybe always decomposed is fine...

I don't think so; although fallback positioning is (much) better than nothing for otherwise-unsupported combinations, it's unlikely to be acceptable as a substitute for precomposed glyphs provided by the font.

[behdad]

Ok, but then can we agree on decomposed if font has GPOS? Has GPOS and mark feature?

[jfkthame]

Yes, I think that's reasonable. IMO "has GPOS and mark feature" would be a better condition than just "has GPOS", as it seems plausible there'll be fonts that include kerning (and newer tools may have put the kern table into GPOS rather than a legacy 'kern' table), but no mark positioning has been implemented.

[dscorbett]

That would break fonts that assume either that the shaper uses NFC, or that the shaper might not normalize but that most text is already in NFC. For example, here are U+0123 LATIN SMALL LETTER G WITH CEDILLA and U+0386 GREEK CAPITAL LETTER ALPHA WITH TONOS in Noto Sans when the default shaper uses HB_OT_SHAPE_NORMALIZATION_MODE_DECOMPOSED.

[behdad]

That would break fonts that assume either that the shaper uses NFC, or that the shaper might not normalize but that most text is already in NFC. For example, here are U+0123 LATIN SMALL LETTER G WITH CEDILLA and U+0386 GREEK CAPITAL LETTER ALPHA WITH TONOS in Noto Sans when the default shaper uses HB_OT_SHAPE_NORMALIZATION_MODE_DECOMPOSED.

What's wrong with that?!!