luahbtex/harf mode vs. AAT font

[schlcht]

The following, compiled with luahbtex on a Mac, results in a completely blank pdf.

\nopagenumbers
\tracingoutput1
\showboxdepth\maxdimen
\showboxbreadth\maxdimen
\input luaotfload.sty
\font\1={HoeflerText:mode=harf;shaper=graphite2}
%\font\1={HoeflerText:mode=harf;shaper=coretext_aat}
\wlog{\1=\meaning\1}
\1 ABC
\bye

There is no error message, and the meaning of \1 is shown as expected, however neither is the font embedded in the pdf nor are the glyphs contained in the shipped out box, which are therefore even underfull (see attached test.log).

This happens with the shapers graphite2 and coretext_aat, but not with ot or if no shaper is specified. There also isn't any problem with luatex.

[zauguin]

I currently can't reproduce this because I don't have access to the font. Does something similar happen with any open font?

[zauguin]

@schlcht Have you verified that the shaper coretext_aat even exists? I thought that it was removed some time ago?

[schlcht]

@zauguin Do you have any suggestions for a free AAT font that I could use for testing?

As for coretext_aat, I had simply copied that from fontspec. Gleaning the shapers from harfbuzz, only ot and fallback work properly, while graphite2, uniscribe, directwrite and coretext (whatever they mean) silently suppress the output. Also, the same happens with a normal OpenType font, e.g. \font\1={LatinModernRoman:mode=harf;shaper=coretext}.

[zauguin]

I can't experiment with coretext anyway, but that sounds weird.

First, slightly off-topic: Have you tried if your AAT font works without specifying any shaper? Normally the default shaper should handle such fonts.

To get back to the actual issue, what do you get with

\font\1={LatinModernRoman:mode=harf;shaper=graphite2}

and

\font\1={LatinModernRoman:mode=harf;shaper=somethingweird}

?

[zauguin]

graphite2, uniscribe, directwrite and coretext (whatever they mean)

• graphite2: Use SIL Graphite2 for rendering. This open technology (which is roughly similar to AAT) requires special fonts and should be available on all platforms. Selected by default for compatible fonts.
• uniscribe and directwrite: Only available on Windows. Instead of HarfBuzz's own font shaper, use Windows system libraries. AFAICT only supports OpenType, not any advanced font technologies.
• coretext: Only available on MacOS. Instead of HarfBuzz's own font shaper, use Mac system libraries. (Probably) supports OpenType and AAT.
• ot: The default for non-Graphite fonts. Use HarfBuzz for font shaping. Provide cross-platform compatibility and supports OpenType and AAT fonts.

[schlcht]

Yes, loading HoeflerText (or any other font) without specifying any shaper works fine.

Loading LatinModernRoman only works with ot and fallback, all other shapers result in empty output, same if I request a non-existing one.

I did find a font that works with graphite2 (Charis SIL), though, proving that at least this shaper is not broken entirely.

[zauguin]

I just pushed 4d44dcd to dev. This adds warning messages if a incompatible shaper is passed. Also it provides slightly better fallback behavior.

@u-fischer How do we want to handle this? We could raise an error or just print a warning and don't shape the affected text. (Currently the warning option is implemented)
I think I prefer the error because nobody reads warnings anyway and there really isn't anything sensible we can do if the user requests a wrong shaper, but a warning feels more like the regular luaotfload approach.

[khaledhosny]

HarfBuzz will error if the specified shaper does not exist or not supported by the font. What is passed to HarfBuzz a list of shapers to be tried one by one (overriding the default order), and if the font does not support a shaper it will be skipped and if no shapers are left an error will happen and no output is given.

The code could work around this by allowing appending fallback to the list of shapers as this shaper never fails (but it is a dump shaper and does very little than mapping characters to glyphs).

[khaledhosny]

(ot shaper never fails too, the name is misnomer by now and should be seen as harfbuzz or default shaper).

[zauguin]

@khaledhosny

HarfBuzz will error if the specified shaper does not exist or not supported by the font. What is passed to HarfBuzz a list of shapers to be tried one by one (overriding the default order), and if the font does not support a shaper it will be skipped and if no shapers are left an error will happen and no output is given.

The code could work around this by allowing appending fallback to the list of shapers as this shaper never fails (but it is a dump shaper and does very little than mapping characters to glyphs).

I used another approach on purpose because I tried to avoid exactly that: I do not want a silent fallback to because I think that would give the impression that the specified shaper worked. So the current approach is to do the fallback manually and therefore allowing us to intercept it and inform the user what is going on.

[khaledhosny]

That is your call.

In hindsight, the shaper option was not the best idea. There are in reality two HarfBuzz shapers now, ot and graphite2 and the number is likely to go down not up (there is an ongoing effort to integrate Graphite in some form into OpenType).

HarfBuzz default shpers list gives priority to graphite2, so hybrid Graphite/OpenType fonts (like many of SIL’s Latin fonts) will use the former. XeTeX intentionally reorders the list so that Graphite is used only when explicitly requested (for compatibility with pre-HarfBuzz XeTeX since people complained about such fonts suddenly using Graphite) so luaotfload might want to do the same and change the default order and introduce a graphite option and deprecate (or discourage) the shaper option. XeTeX has something similar for AAT, but this does nothing now since native AAT support is done by the ot shaper which has heuristic to chose doing AAT or OpenType shaping, so luaotfload might want to add a aat option that does nothing for parity with XeTeX and in case HarfBuzz in the future make it possible to select AAT or OpenType explicitly (not very likely, though).

The other shapers (uniscribe, directwrite, and coretext) are not enabled in TeX Live builds as they are test shapers used mainly by HarfBuzz developer to compare these libraries with its own and are not meant to be used by end users (and not tested as such).

[u-fischer]

@u-fischer How do we want to handle this? We could raise an error or just print a warning and don't shape the affected text. (Currently the warning option is implemented)

Imho if text gets lost an error is the right thing to do.

[zauguin]

Imho if text gets lost an error is the right thing to do.

The new version no longer loses text, it just doesn't shape it and passes it on unchanged. But I'm also convinced by now that this should be an error.