Kannada JIHVAMULIYA and UPADHMANIYA insert dotted circles

[roozbehp]

According to Unicode, these should display fine just appearing by themselves. But HarfBuzz inserts dotted circles after them. Here's the result of:

./test/shaping/hb-unicode-encode 0CF1 20 0CF2| ./util/hb-view --font-file=$HOME/src/noto-fonts/unhinted/NotoSansKannada-Regular.ttf --output-file=kannada.png

[jfkthame]

The same will presumably apply to the equivalent Brahmi signs U+11003 and U+11004, as these also have Indic_Syllabic_Category=Consonant_With_Stacker, which is not being handled correctly by harfbuzz. (See hb-ot-shape-complex-indic-private.hh, which treats it as OT_Repha, with a suspicious-looking TODO comment!)

[jfkthame]

BTW, does the Noto Kannada font have any support for these characters (besides just rendering simple glyphs)? From a quick look, I'm not seeing any sign of the expected ligatures for things like <U+0CF2, U+0CAB>.

[behdad]

From shaping point of view looks to me like just treating them as consonant is best. I'll commit that for now.

[roozbehp]

@jfkthame There's a bug open for complex behavior of these in Noto Sans Kannada at https://github.com/googlei18n/noto-fonts/issues/459

[dscorbett]

This change breaks the latest development version of FreeSerif. FreeSerif uses 'psts' to substitute consonants with their subjoined forms following jihvamuliya and upadhmaniya, but now that Consonant_With_Stacker is treated as Consonant, 'psts' does not get applied.

[behdad]

This change breaks the latest development version of FreeSerif.

Humm....

FreeSerif uses 'psts' to substitute consonants with their subjoined forms

That should happen in 'pstf', not 'psts'.

following jihvamuliya and upadhmaniya, but now that Consonant_With_Stacker is treated as Consonant, 'psts' does not get applied.

[dscorbett]

'pstf' applies to “Halant + Consonant combinations”, but these characters subjoin following consonants without any halant. So is 'pstf' appropriate?

[behdad]

Why does 'psts' not apply?!

[behdad]

We apply 'psts' globally.

[dscorbett]

Is it global to the cluster or global to the whole input string?

[behdad]

Is it global to the cluster or global to the whole input string?

Global to the syllable. Though, backtrack/lookahead can span across syllable boundaries.

[behdad]

Ok so the problem I suppose is that now the stacked component ends up in a separate cluster...

[behdad]

So we do need a better solution for these, as they behave more like "consonant halant". I suppose we can hack the grammar. @jfkthame WDYT?

[behdad]

@dscorbett do you have tests you can contribute to HarfBuzz perhaps?

[dscorbett]

I could make a pull request using record-test.sh, but FreeSerif is GPL-licensed and I don’t know any other font that tries to handle these characters. Would that be okay?

[behdad]

Yes, GPL is fine for the tests. I was thinking maybe you have a test suite that failed...

[JelleBosmaMT]

FYI Starting with the design proposal for Noto Sans Kannada update:

With a calt or abvs lookup I can get a subscript consonant after JIHVAMULIYA and UPADHMANIYA in Mac Safari and Chrome browsers, reproducing the samples given by Unicode. With blwf it doesn't work. In Windows it doesn't work at all. Which is not a problem for HarfBuzz.