calt substitution problem

[rajeeshknambiar]

I’m not sure if this qualifies as a bug report yet; so discussing here first.

I have a contextual substitution which replaces r3cil glyph with dotreph in the attached font (RIT-Rachana):

lookup calt_dot_reph {
    sub r3cil' [... l3l3 ...] by dotreph;  #full list of context glyphs can be found in otf
} calt_dot_reph;

hb-shape indeed seems to substitute it correctly

hb-shape --font-file RIT-Rachana-Regular.otf "ര്‍ല്ല"   #r3  xx zwj l3 xx l3 → r3cil l3l3 → dotreph l3l3
[dotreph=0+0|l3l3=3+1450]

but hb-view is missing the dotreph glyph, only l3l3 glyph is displayed.

hb-view --font-file RIT-Rachana-Regular.otf "ര്‍ല്ല" -o /tmp/hb-test.png   #expects dot reph above l3l3

Actual result:

Expected result:

Font:
RIT-Rachana-Regular.zip

What am I missing?

[khaledhosny]

The glyphs is there but too far to the left, and hb-view bounds calculation misses this. Try:

hb-view --font-file RIT-Rachana-Regular.otf "ര്‍ല്ല"  --margin=10,10,10,200

If mark attachment should happen here, for some reason it isn’t working.

[rajeeshknambiar]

Ok, so the glyph is present. But GPOS doesn't seem to work, even if mark positioning is defined in the font (abvm). hb-shape -V shows

...
trace: end table GSUB   buffer: [dotreph=0|l3l3=3]
trace: start table GPOS buffer: [dotreph=0+0|l3l3=3+1450]
trace: start lookup 0   buffer: [dotreph=0+0|l3l3=3+1450]
trace: end lookup 0     buffer: [dotreph=0+0|l3l3=3+1450]
trace: start lookup 1   buffer: [dotreph=0+0|l3l3=3+1450]
trace: end lookup 1     buffer: [dotreph=0+0|l3l3=3+1450]
trace: start lookup 2   buffer: [dotreph=0+0|l3l3=3+1450]
trace: end lookup 2     buffer: [dotreph=0+0|l3l3=3+1450]
trace: start lookup 3   buffer: [dotreph=0+0|l3l3=3+1450]
trace: end lookup 3     buffer: [dotreph=0+0|l3l3=3+1450]
trace: start lookup 4   buffer: [dotreph=0+0|l3l3=3+1450]
trace: end lookup 4     buffer: [dotreph=0+0|l3l3=3+1450]
trace: end table GPOS   buffer: [dotreph=0+0|l3l3=3+1450]
[dotreph=0+0|l3l3=3+1450]

[dscorbett]

It isn’t working because dotreph is not being reordered towards the end of the syllable. That is because HarfBuzz doesn’t recognize it as a reph. The reph should be encoded as U+0D4E MALAYALAM LETTER DOT REPH, not as <U+0D30, U+0D4D, U+200D>. The Unicode Standard, version 15.0, chapter 12 says “The sequence <ra, virama, ZWJ> is not used to represent the dot reph, because that sequence has considerable preexisting usage to represent the chillu form of ra, prior to the encoding of the chillu form as a distinct character, U+0D7C U+0D7C MALAYALAM LETTER CHILLU RR.”

If the text follows Unicode’s recommendation, this font works:

$ hb-shape RIT-Rachana-Regular.otf -u '0D4E 0D32 0D4D 0D32'
[l3l3=0+1450|dotreph=0@196,0+0]

[behdad]

The shaper moves dotreph to after the baseglyph, but not the chillu-ra.

[rajeeshknambiar]

The shaper moves dotreph to after the baseglyph, but not the chillu-ra.

Ok, that explains.

Is there an advisable method to display a conjunct with dotreph positioned properly using calt for chillu-ra?

[behdad]

Is there an advisable method to display a conjunct with dotreph positioned properly using calt for chillu-ra?

Since the glyph will be before the base, you cannot use mark-positioning. Your best bet is to draw the glyph in a way that it just is positioned in roughly the right place I'm afraid...

[rajeeshknambiar]

The shaper moves dotreph to after the baseglyph, but not the chillu-ra.

Indeed, this happens at reordering indic final step:

trace: end lookup 10    buffer: [dotreph=0|l3l3=1]
trace: start reordering indic final     buffer: [dotreph=0|l3l3=1]
trace: end reordering indic final       buffer: [l3l3=0|dotreph=0]             ← reordered  here
trace: start lookup 11  buffer: [l3l3=0|dotreph=0]
trace: end lookup 11    buffer: [l3l3=0|dotreph=0]
trace: end table GSUB   buffer: [l3l3=0|dotreph=0]
trace: start table GPOS buffer: [l3l3=0+1450|dotreph=0+0]
trace: start lookup 0   buffer: [l3l3=0+1450|dotreph=0+0]
trace: end lookup 0     buffer: [l3l3=0+1450|dotreph=0+0]
trace: start lookup 1   buffer: [l3l3=0+1450|dotreph=0+0]
trace: attaching mark glyph at 1 to glyph at 0  buffer: [l3l3=0+1450|dotreph=0+0] ← mark attachment
trace: attached mark glyph at 1 to glyph at 0   buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: end lookup 1     buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: start lookup 2   buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: end lookup 2     buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: start lookup 3   buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: kerning glyphs at 0,1    buffer: [l3l3=0+1450|dotreph=0@1646,0+0]       ← kerning
trace: kerned glyphs at 0,1     buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: end lookup 3     buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: start lookup 4   buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: end lookup 4     buffer: [l3l3=0+1450|dotreph=0@1646,0+0]
trace: end table GPOS   buffer: [l3l3=0+1450|dotreph=0@1646,0+0]

[rajeeshknambiar]

Is there an advisable method to display a conjunct with dotreph positioned properly using calt for chillu-ra?

Since the glyph will be before the base, you cannot use mark-positioning. Your best bet is to draw the glyph in a way that it just is positioned in roughly the right place I'm afraid...

It looks like reordering could work with chaining substitution (ref. Example 5 in Section 5.f.i of https://adobe-type-tools.github.io/afdko/OpenTypeFeatureFileSpecification.html#5.f.i). I presume ka.pas_cakra.ns in that example is removed/substituted away.

I could get the reordering work (change r3cil l3l3 to l3l3 dotreph), but the final l3l3 is not removed. Not sure whether it is an issue with Fontforge (which is what merges the feature file). These are the lookups I am trying to apply:

lookup REMOVE_l3l3 {
    sub r3cil l3l3 by r3cil;
} REMOVE_l3l3;

lookup REORDER_reph_l3l3 {
    sub r3cil by l3l3 dotreph;
} REORDER_reph_l3l3;

lookup calt_dot_reph {
    sub r3cil' lookup REMOVE_l3l3 lookup REORDER_reph_l3l3 l3l3' ;
} calt_dot_reph;

feature calt {
    lookup calt_dot_reph;
} calt;

And the shaping becomes:

[l3l3=0+1450|dotreph=0@196,0+0|l3l3=3+1450]    # l3l3 at the end is not substituted away

May be @simoncozens have some ideas?

[simoncozens]

Try this (untested)

lookup KILL_l3l3 {
   sub l3l3 by NULL;
} KILL_l3l3;

    sub r3cil' lookup REMOVE_l3l3 lookup REORDER_reph_l3l3 l3l3' lookup KILL_l3l3 ;

[simoncozens]

Wait, we're all making this too difficult. Looks like what you want to do is substitute the r3cil by l3l3 and l3l3 for dotreph. So surely the code is

lookup swap {
    sub r3cil by l3l3;
    sub l3l3l by dotreph;
} swap;

lookup calt_dot_reph {
    sub r3cil' lookup swap l3l3' lookup swap ;
} calt_dot_reph;

[rajeeshknambiar]

And... that is just perfect (barring a minor typo l3l3l).

...
trace: end lookup 10    buffer: [r3cil=0|l3l3=3]
trace: start reordering indic final     buffer: [r3cil=0|l3l3=3]
trace: end reordering indic final       buffer: [r3cil=0|l3l3=3]
trace: start lookup 12  buffer: [r3cil=0|l3l3=3]
trace: recursing to lookup 11 at 0      buffer: [r3cil=0|l3l3=3]
trace: replacing glyph at 0 (single substitution)       buffer: [r3cil=0|l3l3=3]  ← (sequence now: r3cil l3l3)
trace: replaced glyph at 0 (single substitution)        buffer: [l3l3=0|l3l3=3]   ← r3cil replaced (sequence now: l3l3 l3l3)
trace: recursed to lookup 11    buffer: [l3l3=0|l3l3=3]
trace: recursing to lookup 11 at 1      buffer: [l3l3=0|l3l3=3]
trace: replacing glyph at 1 (single substitution)       buffer: [l3l3=0|l3l3=3]
trace: replaced glyph at 1 (single substitution)        buffer: [l3l3=0|dotreph=3] ← second l3l3 replaced (sequence now: l3l3 dotreph)
trace: recursed to lookup 11    buffer: [l3l3=0|dotreph=3]
trace: end lookup 12    buffer: [l3l3=0|dotreph=3]
trace: end table GSUB   buffer: [l3l3=0|dotreph=3]
trace: start table GPOS buffer: [l3l3=0+1450|dotreph=3+0]
trace: start lookup 0   buffer: [l3l3=0+1450|dotreph=3+0]
trace: end lookup 0     buffer: [l3l3=0+1450|dotreph=3+0]
trace: start lookup 1   buffer: [l3l3=0+1450|dotreph=3+0]
trace: attaching mark glyph at 1 to glyph at 0  buffer: [l3l3=0+1450|dotreph=3+0]
trace: attached mark glyph at 1 to glyph at 0   buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: end lookup 1     buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: start lookup 2   buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: end lookup 2     buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: start lookup 3   buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: kerning glyphs at 0,1    buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: kerned glyphs at 0,1     buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: end lookup 3     buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: start lookup 4   buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: end lookup 4     buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
trace: end table GPOS   buffer: [l3l3=0+1450|dotreph=3@1646,0+0]
[l3l3=0+1450|dotreph=3@196,0+0]

For a better understanding: does the lookup gets applied as follows?

1. r3cil' lookup swap → triggers lookup and replaces r3cil with l3l3.
2. l3l3' lookup swap → triggers lookup and replaces l3l3 with dotreph.
   Resulting in sequence l3l3 dotreph and GPOS processing takes care of the rest.

[simoncozens]

Exactly!

[rajeeshknambiar]

Many thanks, Simon.

One (possibly final question): similar substitution should take place for a number of conjuncts, the only option to make them all work is something as follows?

...
lookup swap_reph_y1y1 {
	sub r3cil by y1y1;
} swap_reph_y1y1;

lookup swap_reph_l3l3 {
	sub r3cil by l3l3;
} swap_reph_l3l3;

lookup swap_reph_v1v1 {
	sub r3cil by v1v1;
} swap_reph_v1v1;

lookup swap_reph {
    sub [k1k1 k3k3 ch1ch1 ch3ch3 nhnh th1th1 th3th3 th3th4 n1n1 p1p1 p3p3 m1m1 y1y1 l3l3 v1v1] by dotreph;
} swap_reph;

lookup calt_dot_reph {
    # swap r3cil with following conjunct, swap the conjunct with dotreph
    ...
    sub r3cil' lookup swap_reph_y1y1 y1y1'    lookup swap_reph ;
    sub r3cil' lookup swap_reph_l3l3 l3l3'    lookup swap_reph ;
    sub r3cil' lookup swap_reph_v1v1 v1v1'    lookup swap_reph ;

[simoncozens]

Sadly, yes.