Broken Kannada shaping?

[brawer]

For NotoSerifKannada-Regular.ttf, HarfBuzz emits broken output while CoreText output looks good for the same font. This might be a bug in HarfBuzz. However, it might also be that CoreText has a bug in its Kannada shaping; I don’t know if Noto’s font designers have tested their font on multiple OpenType implementations or just on CoreText. So it might well be that the font (and CoreText) are buggy. If somebody who knows more about Kannada shaping could analyze this, it would be great.

I’ve added this as a test case to Unicode’s text rendering test suite, using CoreText to generate the expected renderings. Should it turn out that the HarfBuzz implementation is correct and CoreText (and the Noto font) are both wrong, I’ll of course adjust the test suite.

See https://github.com/googlei18n/noto-fonts/issues/759 for the corresponding bug report on the NotoSerifKannada font.

[KrasnayaPloshchad]

I’m sure this is a bug in HarfBuzz, see the discussions on notofonts/noto-fonts#341.

[lianghai]

I wrote GSUB rules for all Noto Serif Indic fonts, and I can confirm it's a HarfBuzz issue. HarfBuzz's handling of the "GDEF" table seems to be nonstandard, which leads to this issue.

Noto Serif Kannada relies on lookupflag IgnoreMarks; for some lookups to work properly. The lookup of rules sub knKA knmI by knKI; etc has lookupflag IgnoreMarks; too (for skipping Nukta), however HarfBuzz wrongly considers vowel signs are marks (a derivation from UCD?) despite that in the "GDEF" table vowel signs are not defined as GDEF marks.

A patched Noto Serif Kannada with a minimal change (setting the <LookupFlag value="8"/> in <Lookup index="4"> to <LookupFlag value="0"/>) demonstrates how this works:
NotoSerifKannada-Regular-patched.ttf.zip

[lianghai]

@KrasnayaPloshchad I don't think this issue is related to notofonts/noto-fonts#341 though.

[lianghai]

While patching SamsungOne fonts I've noticed HarfBuzz's exact issue seems to be:

When a Unicode-mapped glyph's GDEF class is undefined in the GDEF table, HarfBuzz defaults its GDEF class to a value derived from UCD (eg, Mn becomes GDEF class 3 "mark glyph").

However, according to the OT spec, https://www.microsoft.com/typography/otspec/GDEF.htm —

The font developer does not have to classify every glyph in the font, but any glyph not assigned a class value falls into Class zero (0).

— Which is indeed the common behavior of text engines (including earlier versions of HarfBuzz, eg, 0.9.40 and 1.0.5, according to Samsung's report). With the expected behavior of defaulting to 0, any glyph not assigned a class value in the GDEF table should not be ignored by lookupflag IgnoreMarks; (ignoring class 3).

Now the workaround for me is to patch the fonts by assigning non mark glyphs the class value of 1 "base glyph", like what happened (or was intended?) to have been done in Nirmala UI.

[behdad]

@lianghai The GDEF behavior you mention is definitely not expected. HarfBuzz only fills in GDEF classes from Unicode data if there is no GDEF table present...

Ugh. Now I see that this was changed in March, in: 69f9fbc
I suppose I can special-case Indic-like shapers to disable this and use the previous logic.

[lianghai]

@behdad Ah, I see… Though, given the clearly defined behavior of "any glyph not assigned a class value falls into Class zero (0)" in the OT spec, should the special case be actually Hebrew that does synthesize GDEF glyph class (instead of Indic-like shapers that do not)?

[behdad]

@lianghai Yeah, I'm going to revert this, and make hebrew shaper ignore GDEF and GPOS (and use fallback for both) if GPOS Hebrew script system is not found.