Broken test SFNT-1

[brawer]

When a font contains both CFF and glyf tables, fontkit should use the sfntVersion field of the Offset Table to decide which one to use.

https://rawgit.com/unicode-org/text-rendering-tests/master/reports/fontkit.html#SFNT-1

[Pomax]

I'm pretty sure that the test in question made the wrong assumption about what should happen in this particular case. Looking at the OpenType definition for sfntVersion, we see this clarification:

OpenType fonts that contain TrueType outlines should use the value of 0x00010000 for the sfntVersion. OpenType fonts containing CFF data (version 1 or 2) should use 0x4F54544F ('OTTO', when re-interpreted as a Tag) for sfntVersion.

Rather than saying "which data to use based on the version", the only prescription here is for "which version to use, based on the data".

At best we can only conclude that, because the sfntVersion field cannot be two values at once, whatever software generated this font did so incorrectly, giving us a font that is inconsistent with respect to the spec, and so should not even be accepted as a valid font (and someone needs to fix it).

[twardoch]

I agree with Pomax’s interpretation of the OpenType spec. However, Fontkit.js claims to aim to supportnot only OpenType fonts but also TrueType fonts. The TrueType spec has a different spec for the scaler type field, which reads as if it allows hosting multiple outline flavors, but the scaler type field should be used to determine which outline flavor structures to pick.

So the font in question may not be a valid OpenType font but it may very well be a valid TrueType font and a valid sfnt-housed font.

https://developer.apple.com/fonts/TrueType-Reference-Manual/RM06/Chap6.html

The scaler type is used by OS X and iOS to determine which scaler to use for this font, that is, to determine how to extract glyph data from the font. Different font scalers wrap different font formats within the basic structure of a TrueType font; the scaler type in the offset subtable of the font's directory is used to indicate which scaler should be used with a particular font. (Non-TrueType fonts housed within the same structure as a TrueType font are referred to as "sfnt-housed fonts.")

The values 'true' (0x74727565) and 0x00010000 are recognized by OS X and iOS as referring to TrueType fonts. The value 'typ1' (0x74797031) is recognized as referring to the old style of PostScript font housed in a sfnt wrapper. The value 'OTTO' (0x4F54544F) indicates an OpenType font with PostScript outlines (that is, a 'CFF ' table instead of a 'glyf' table). Other values are not currently supported.

Fonts with TrueType outlines produced for OS X or iOS only are encouraged to use 'true' (0x74727565) for the scaler type value. Fonts for Windows or Adobe products must use 0x00010000.

[twardoch]

(sfnt-housed fonts with TrueType outlines that conform to the OpenType spec are known as TrueType-flavored OpenType fonts or as OpenType fonts with TrueType outlines. sfnt-housed fonts with TrueType outlines that conform to the TrueType spec are known as TrueType fonts. Fontkit.js claims to support both OpenType fonts and TrueType fonts.)

[Pomax]

Reading the feature list, I strongly suspect what happened instead is that someone made the intentional "mistake" of using the word OpenType simply to refer to otf files and the word TrueType simply to refer to ttf files, despite actually being an OpenType library only.

Presumably, because that's what the rest of world keeps (incorrectly, but entirely understandably) calling them. Outside of the typography world, no one calls ttf files "OpenType with TrueType outlines", they just call them "TrueType fonts" and search the web accordingly. I filed an issue to hopefully that get corrected enough to be both technically correct, while also keeping it clear that people who need a library for their .ttf fonts are still served by this software package.

[twardoch]

Some people use those terms this way, but Fontkit has a morx table parser, unlike the vast majority of toolkits that deal with sfnt-housed fonts. So it does indeed support aspects of the TrueType spec.

[twardoch]

I mean: it’d be desirable to clarify the terminology, since Fontkit is actually a much less “naive” toolkit than some people might think.

These days, sfnt-housed fonts are a bit like MKV or MOV containers. While those may contain multiple streams of audio and video, and the decoding app picks one, sfnt-housed fonts may contain multiple “streams” (data structures) of layout data and of imaging data.

The way I call this is that the “known” layout streams are:

• TrueType layout (kern table only)
• AAT layout (morx and kerx or kern), with optional variable data
• OpenType layout (GSUB and GPOS), with optional variable data
• SIL Graphite layout

The “known” imaging streams are:

• SVG with optional CPAL
• sbix
• CBDT
• CFF2 with variable data and optional color COLR/CPAL data
• glyf with optional variable data and optional color COLR/CPAL data
• CFF with optional color COLR/CPAL data
• EBDT or bdat

Both specs are written in an antiquated way, with the “scaler type” / sfntVersion field is used to pick between CFF and glyf, but there is no explicit info about what should happen if the font contains both, or none. A font may contain only CFF2 or only CBDT, but the spec is silent about these cases. SVG and sbix can only exist on top of monochrome outline imaging data, but it's also not clear what an app works do if, say, both SVG and CFF2 (and only those) exist in the font.

[twardoch]

But what I meant to say is that the Apple spec shows the spirit of that field. It was originally intended to pick the right scaler between several potentially co-existing imaging streams (glyph description formats). The major implementations still respect it — they consider a font with both glyf and CFF valid and use the sfntVersion girls to pick the right imaging stream.

[twardoch]

The logic of picking the other imaging streams, and the layout streams, is much more obscure and implemented inconsistently

[twardoch]

On top of that, we have different containers for the sfnt structure: otf, ttf, dfont, woff2, woff, otc, ttc — again, somewhat similar to audio/video where containers such as MKV, MOV, AVI are used only for packaging, and can hold video and audio streams (and also subtitles and other) in a variety of formats (codecs)

[Pomax]

no argument there, but as far as I can see from the README there is no solid claim to support TrueType-without-OpenType, it just reads like a generic description of an OpenType-only library, with some easy tables added because they were encountered in the wild, using incorrect text so that the layman user won't get confused.

Perhaps @devongovett or someone else in the FolioJS group can elucidate the intended support.

As for the original issue, the font in the test case would still be an invalid font. Reading the spec you linked to on Apple's dev website, we see:

The values 'true' (0x74727565) and 0x00010000 are recognized by OS X and iOS as referring to TrueType fonts. The value 'typ1' (0x74797031) is recognized as referring to the old style of PostScript font housed in a sfnt wrapper. The value 'OTTO' (0x4F54544F) indicates an OpenType font with PostScript outlines (that is, a 'CFF ' table instead of a 'glyf' table). Other values are not currently supported.

The crucial word here being that instead in the single-to-last sentence: by listing several cases but not listing the case of "truetype and CFF in the same font", a font that does this is not covered by the spec, which means it cannot be considered 100% spec compliant. As such, even if we assume Apple TrueType, that test case relies on an assumption of defined behaviour where there is none, and should be removed.

[twardoch]

But there is https://github.com/foliojs/fontkit/tree/master/src/aat — a full-blown TrueType AAT layout engine, which I believe was the first independent implementation of that layout system (now HarfBuzz being the 2nd)

[blikblum]

Only devongovett can clarify but i believe the intention is to support everything.

The parts that are not implemented is because lack of man power or just something that was overlooked

[Pomax]

Right, but this is tangential to the original issue, and should probably go into the README.md update issue instead (as that's about getting that clarification).

For this issue: given the text of both the OpenType and Apple TrueType specifications, any font that contains both glyf/loca tables and a cff (or cff2 when dealing with OpenType) table is not a spec-compliant font, and parser behaviour for them is undefined.

Any test that checks "whether a font engine does the right thing" on this kind of font is an invalid test, and should be removed from the test suite it is in.

@brawer is this enough to effect that, or would you like an issue filed for that over on https://github.com/unicode-org/text-rendering-tests ?

[devongovett]

There is spec compliance, and then there's the real world. Real world fonts are oftentimes not spec compliant, but users still expect them to work. text-rendering-tests is in many cases testing the behavior of these fonts at the edge cases. What does the parser do with an invalid font? Does it crash? Run out of memory? Behave inconsistently from other engines?

If we're inconsistent with the behavior of real world fonts in other engines, we should fix that so fontkit behaves as expected, regardless of spec compliance.