Implement kern (and may kerx?) cross-stream to support GeezaPro Arabic diacritics

[ebraminio]

From https://chromium-review.googlesource.com/c/chromium/src/+/1275945/33/third_party/WebKit/LayoutTests/platform/mac/fast/text/font-fallback-expected.png

util/hb-view /System/Library/Fonts/GeezaPro.ttc الأَبْجَدِيَّةـالعَرَبِيَّة --shaper coretext

util/hb-view /System/Library/Fonts/GeezaPro.ttc الأَبْجَدِيَّةـالعَرَبِيَّة --shaper ot or (with or without --font-funcs=ot)

On 10.13 machine,

CT:

OT:

3	en	Geeza Pro Regular; 14.0d1e4; 2017-12-05

bash-3.2$ shasum /System/Library/Fonts/GeezaPro.ttc 
ab26ea45dcaa5e1c5a958e42af10e10d330e7334  /System/Library/Fonts/GeezaPro.ttc

[ebraminio]

Still arguable maybe, I don't know

[behdad]

For a font that has no mark positioning data, this is really good already.

[ebraminio]

Replaced with high quality images. Maybe kasrah can be put a little higher?

[jfkthame]

From a brief look at the Geeza Pro on my 10.13 system, it appears to include a number of cross-stream kerning subtables (in a 'kern' table, not 'kerx'). I'm guessing these are intended to adjust the height of diacritics; but are they supported by HB yet?

[ebraminio]

Oh thanks @jfkthame, pretty sure that is the issue and Kern table doesn't show any indication of handling cross-stream AFAICS.

diff --git a/src/hb-ot-kern-table.hh b/src/hb-ot-kern-table.hh
index e361330b..9ab64af7 100644
--- a/src/hb-ot-kern-table.hh
+++ b/src/hb-ot-kern-table.hh
@@ -209,9 +209,10 @@ struct KernSubTableFormat1
     };
 
     inline driver_context_t (const KernSubTableFormat1 *table_,
-                            AAT::hb_aat_apply_context_t *c_) :
+                            AAT::hb_aat_apply_context_t *c_, bool is_cross_stream_) :
        c (c_),
        table (table_),
+       is_cross_stream (is_cross_stream_),
        /* Apparently the offset kernAction is from the beginning of the state-machine,
         * similar to offsets in morx table, NOT from beginning of this table, like
         * other subtables in kerx.  Discovered via testing. */
@@ -258,7 +259,9 @@ struct KernSubTableFormat1
          int v = *actions++;
          if (idx < buffer->len && buffer->info[idx].mask & kern_mask)
          {
-           if (HB_DIRECTION_IS_HORIZONTAL (buffer->props.direction))
+           bool is_horizontal = HB_DIRECTION_IS_HORIZONTAL (buffer->props.direction);
+           if (is_cross_stream) is_horizontal = !is_horizontal;
+           if (is_horizontal)
            {
              buffer->pos[idx].x_advance += c->font->em_scale_x (v);
              if (HB_DIRECTION_IS_BACKWARD (buffer->props.direction))
@@ -284,16 +287,16 @@ struct KernSubTableFormat1
     const UnsizedArrayOf<FWORD> &kernAction;
     unsigned int stack[8];
     unsigned int depth;
+    bool is_cross_stream;
   };
 
-  inline bool apply (AAT::hb_aat_apply_context_t *c) const
+  inline bool apply (AAT::hb_aat_apply_context_t *c, bool is_cross_stream) const
   {
     TRACE_APPLY (this);
-
     if (!c->plan->requested_kerning)
       return false;
 
-    driver_context_t dc (this, c);
+    driver_context_t dc (this, c, is_cross_stream);
 
     AAT::StateTableDriver<AAT::MortTypes, EntryData> driver (machine, c->buffer, c->font->face);
     driver.drive (&dc);
@@ -483,12 +486,12 @@ struct KernSubTable
     }
   }
 
-  inline void apply (AAT::hb_aat_apply_context_t *c, unsigned int format) const
+  inline void apply (AAT::hb_aat_apply_context_t *c, unsigned int format, bool is_cross_stream) const
   {
     /* TODO Switch to dispatch(). */
     switch (format) {
     case 0: u.format0.apply (c); return;
-    case 1: u.format1.apply (c); return;
+    case 1: u.format1.apply (c, is_cross_stream); return;
     case 2: u.format2.apply (c); return;
     case 3: u.format3.apply (c); return;
     default:                    return;
@@ -528,6 +531,9 @@ struct KernSubTableWrapper
   inline bool is_supported (void) const
   { return !(thiz()->coverage & T::CheckFlags); }
 
+  inline bool is_cross_stream (void) const
+  { return !(thiz()->coverage & T::CrossStream); }
+
   inline bool is_horizontal (void) const
   { return (thiz()->coverage & T::Direction) == T::CheckHorizontal; }
 
@@ -541,7 +547,7 @@ struct KernSubTableWrapper
   { return is_supported () && is_horizontal () ? get_kerning (left, right) : 0; }
 
   inline void apply (AAT::hb_aat_apply_context_t *c) const
-  { thiz()->subtable.apply (c, thiz()->format); }
+  { thiz()->subtable.apply (c, thiz()->format, is_cross_stream ()); }
 
   inline unsigned int get_size (void) const { return thiz()->length; }

And that made result a little different or better,

Before:

After:

But the way HB's buffer is ordered, I guess Behdad is better to have a look at this.

[behdad]

From a brief look at the Geeza Pro on my 10.13 system, it appears to include a number of cross-stream kerning subtables (in a 'kern' table, not 'kerx'). I'm guessing these are intended to adjust the height of diacritics; but are they supported by HB yet?

Interesting. Never saw a font with those. I'll implement that.

[behdad]

But the way HB's buffer is ordered, I guess Behdad is better to have a look at this.

That wouldn't be enough. Also, have to disable fallback mark positioning. I'll take a look.

[behdad]

I hope we don't have to look into kern table to decide if fallback mark positioning should be applied. Currently, looks like we might need to.

[behdad]

Also, we apply kern table if GPOS kern is missing. That can interact badly with mark positioning. Oh well.

[behdad]

Maybe kasrah can be put a little higher?

We decided to not move bottom marks up. That helped with Hebrew.

[ebraminio]

util/hb-shape /System/Library/Fonts/GeezaPro.ttc الأَبْجَدِيَّةـالعَرَبِيَّة --shaper coretext

[u0629.final.tehMarbuta=26+713|u064e_u0651.shaddaFatha=23@0,-200+0|u064a.medial.yeh=23+736|u0650.kasra=21@0,290+-80|u0628.initial.beh=21+856|u064e.fatha=19@0,-570+-200|u0631.final.reh=19+1102|u064e.fatha=17@0,-200+-200|u0639.medial.ain=17+938|u0644.initial.lam=16+514|u0627.final.alef=15+647|u0640.tatweel=14+448|u0629.final.tehMarbuta=13+712|u064e_u0651.shaddaFatha=10@0,-200+0|u064a.initial.yeh=10+735|u0650.kasra=8@0,570+-79|u062f.final.dal=8+1192|u064e.fatha=6@0,-160+-290|u062c.medial.jeem=6+1358|u0652.sukun=4@0,-200+0|u0628.initial.beh=4+283|u064e.fatha=2@0,120+120|u0644_u0623.isolated.lamHamzaOnAlef=1+1162|u0627.alef=0+646]

[behdad]

Okay. After a long day of trial and error, my PR gets most of this right. Just the u064e_u0651.shaddaFatha doesn't get positioned, and I don't see in the font tables how it should...

There's a few finishing touches that need to be made. I'll finish tomorrow.

[behdad]

Okay! With help from Ned, I fixed the remaining mystery, as well as finish the finishing touches. It renders fine now. There's a tiny difference visible compared to CoreText, but I don't think that's worth tracking down...

[behdad]

[behdad]

The marks on Lam-Alef (right-most two stacked marks) don't touch the Sokun mark to their left in our rendering, whereas they do in CoreText's. Not sure why. Ours is better anyway....

[behdad]

And hb-shape output:

[u0629.final.tehMarbuta=26+713|u064e_u0651.shaddaFatha=23@0,-200+0|u064a.medial.yeh=23+656|u0650.kasra=21@80,290+80|u0628.initial.beh=21@-80,0+576|u064e.fatha=19@200,-570+200|u0631.final.reh=19@-200,0+702|u064e.fatha=17@200,-200+200|u0639.medial.ain=17@-200,0+738|u0644.initial.lam=16+515|u0627.final.alef=15+647|u0640.tatweel=14+449|u0629.final.tehMarbuta=13+713|u064e_u0651.shaddaFatha=10@0,-200+0|u064a.initial.yeh=10+656|u0650.kasra=8@80,570+80|u062f.final.dal=8@-80,0+822|u064e.fatha=6@290,-160+290|u062c.medial.jeem=6@-290,0+1069|u0652.sukun=4@0,-200+0|u0628.initial.beh=4+656|u064e.fatha=1@-252,120+-252|u0644_u0623.isolated.lamHamzaOnAlef=1@120,0+1282|u0627.alef=0+647]

[avidrissman]

Does this help with the Hebrew vowels as I pointed out in #1264?

…

On Nov 7, 2018, at 1:40 PM, Behdad Esfahbod ***@***.***> wrote: Closed #1356 via 385f78b. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[behdad]

Does this help with the Hebrew vowels as I pointed out in #1264?

Possibly. Someone with Mac needs to test.

[ebraminio]

Thanks Behdad, looks cool now, specially sharing the logic part.

No that doesn't have cross stream, harfbuzz doesn't shape it incorrectly AFAICT also, something wrong with Chrome itself, limited to 10.10 and 10.11, most likely till we find shaping difference on HarfBuzz itself also. I will check it again however.

[behdad]

something wrong with Chrome itself

Dominik and I now have a theory about what's wrong in Blink. He's working on it.