Problem with OpenType bitmap font (Ambrosia.otb)

[lemzwerg]

This zipped SFNT bitmap-only font, created a long time ago by George Williams with FontForge, can't be correctly disassembled with current git (1eba7b3).
I get (with paths shortened for readability)

Dumping "Ambrosia.otb" to "Ambrosia.ttx"...
Dumping 'GlyphOrder' table...
Dumping 'head' table...
Dumping 'hhea' table...
Dumping 'maxp' table...
Dumping 'OS/2' table...
Dumping 'hmtx' table...
Dumping 'cmap' table...
WARNING: corrupt 'loca' table, or wrong numGlyphs in 'maxp': 0 250
Dumping 'loca' table...
ERROR: An exception occurred during the decompilation of the 'glyf' table
Traceback (most recent call last):
  File ".../fontTools/ttLib/__init__.py", line 418, in __getitem__
    table.decompile(data, self)
  File ".../fontTools/ttLib/tables/_g_l_y_f.py", line 62, in decompile
    if len(data) - next >= 4:
UnboundLocalError: local variable 'next' referenced before assignment
Dumping 'glyf' table...
Dumping 'name' table...
Dumping 'post' table...
Dumping 'EBDT' table...
Dumping 'EBLC' table...
Dumping 'EBSC' table...
Dumping 'GDEF' table...
Dumping 'GPOS' table...
Dumping 'GSUB' table...

The glyf table is empty, and ttx doesn't like this while assembling.

Parsing 'GlyphOrder' table...
Parsing 'head' table...
Parsing 'hhea' table...
Parsing 'maxp' table...
Parsing 'OS/2' table...
Parsing 'hmtx' table...
Parsing 'cmap' table...
Parsing 'loca' table...
Parsing 'glyf' table...
Parsing 'name' table...
Parsing 'post' table...
Parsing 'EBDT' table...
Parsing 'EBLC' table...
Parsing 'EBSC' table...
Parsing 'GDEF' table...
Parsing 'GPOS' table...
Parsing 'GSUB' table...
ERROR: Unhandled exception has occurred
Traceback (most recent call last):
  File ".../fontTools/ttx.py", line 381, in main
    process(jobs, options)
  File ".../fontTools/ttx.py", line 355, in process
    action(input, output, options)
  File ".../fontTools/misc/loggingTools.py", line 372, in wrapper
    return func(*args, **kwds)
  File ".../fontTools/ttx.py", line 276, in ttCompile
    ttf.save(output)
  File ".../fontTools/ttLib/__init__.py", line 216, in save
    self._writeTable(tag, writer, done)
  File ".../fontTools/ttLib/__init__.py", line 649, in _writeTable
    self._writeTable(masterTable, writer, done)
  File ".../fontTools/ttLib/__init__.py", line 652, in _writeTable
    tabledata = self.getTableData(tag)
  File ".../fontTools/ttLib/__init__.py", line 663, in getTableData
    return self.tables[tag].compile(self)
  File ".../fontTools/ttLib/tables/_m_a_x_p.py", line 45, in compile
    self.recalc(ttFont)
  File ".../fontTools/ttLib/tables/_m_a_x_p.py", line 64, in recalc
    self.numGlyphs = len(glyfTable)
TypeError: object of type 'DefaultTable' has no len()

I don't know whether such a font without any data in the glyf table is supported except by FreeType, however, ttx should either support it or cleanly reject it.

[anthrotype]

UnboundLocalError is not good.
Does the spec say anything about such an empy glyf table?

[lemzwerg]

Here's a document w.r.t. bitmap-only SFNTs.

https://fontforge.github.io/bitmaponlysfnt.html

The X11 proposal is probably dead – it was suggested by Juliusz Chroboczek, cf.

https://www.irif.fr/~jch//software/xfree86-bitmap-fonts.html

but obviously never adapted.

IIRC, other people have added a single .notdef glyph to glyf to make MS accept the bitmap only font.

[anthrotype]

Thanks for the links.

Basically, if we exclude the *.otb X11 proposal which you say is dead, we are left with the Apple bitmap-only fonts, which have no glyf/loca tables (but havebdat, bloc, bhed tables, which fonttools does not support yet); and the Microsoft's bitmap-only fonts for which the glyf/loca tables are non-empty, and an extra EBSC table is needed (#685).

So, if I'm correct, we want fonttools to assert that glyf is not empty when it's there, right?

[lemzwerg]

Note that EBSC is an optional table.

I suggest that ttx can read OTB files but not write. And yes, for writing bitmap-only SFNT fonts I suggest that you add a glyf table with a single .notdef glyph in case the font doesn't contain it.

[anthrotype]

you mean, we should allow to decompile empty glyf tables (such as in OTB files), but then when we compile them back we should automatically add a .notdef glyph? I think the user should fix the latter.

[lemzwerg]

Exactly. Perhaps it would help if ttx emits a message that OTBs are dead.

[behdad]

you mean, we should allow to decompile empty glyf tables (such as in OTB files), but then when we compile them back we should automatically add a .notdef glyph? I think the user should fix the latter.

Right now fonttools is not in the business of enforcing anything on this level. I think it should stay that way.

[lemzwerg]

OK, then this font should rejected with a proper error message.

[joukewitteveen]

The X11 proposal is probably dead

I'd say it is not! With the recent release of Pango 1.44, it is practically the only way to distribute bitmap fonts on linux.

but obviously never adopted.

At least Harfbuzz, FreeType, and FontForge support it. Probably other projects do too.

It is true that it took quite some time to go from BDF to PCF to OTB, but we do seem to be getting there.

[khaledhosny]

I think the empty glyf/loca tables are complicating thins needlessly. They are not needed, HarfBuzz (and I think FreeType) will handle the fonts without these tables just fine and no other systems support these fonts any way. FontForge should either no added the empty table or have an option not to do so.

[justvanrossum]

Fixing the UnboundLocalError reveals that there's a lot of code in fonttools that assumes len(ttf['glyf']) == len(ttf.getGlyphOrder()), especially when compiling.

I have a rough patch that allows that assumption to be broken, but I tend to agree with @khaledhosny that the loca/glyf tables should simply not be there.

To allow this font to be dumped requires a small change to glyfTable.toXML(). I will make a PR for that.

[behdad]

Basically, just EBLC/EBDT and head, hhea, hmtx should be enough. "Empty" glyf/loca is wrong to me.

If each size of a font goes into its own OT face, it's easy. Multiple of those can be combined in the same TTC file. This should work just fine with fontconfig / Pango.

If multiple bitmap sizes are to go into the same OT face, that becomes more problematic as the hmtx table can represent extents of one only. It's not an issue for monospace fonts, but probably will be for others.

[lemzwerg]

Basically, just EBLC/EBDT and head, hhea, hmtx should be enough. "Empty" `glyf/loca` is wrong to me.

Indeed, I recommend at least a (scalable) `.notdef' glyph.

If each size of a font goes into its own OT face, it's easy. Multiple of those can be combined in the same TTC file. This should work just fine with fontconfig / Pango.

Yep.

If multiple bitmap sizes are to go into the same OT face, that becomes more problematic as the hmtx table can represent extents of one only. It's not an issue for monospace fonts, but probably will be for others.

I agree. While hmtx values can be scaled easily, this is not the case for bitmaps. Thus having different OT faces, you can select the bitmap font at the right pixel size, and you should get good visual results. Werner