how to get error message from hb_subset_repack_or_fail

[anthrotype]

Thanks to @qxliu76's work, we can now use the harfbuzz repacker when serializing GPOS/GSUB tables in FontTools (since version 4.33).
This is cool, however sometimes (esp. big fonts with large subtables) the hb_subset_repack_or_fail returns a nullptr to signal it couldn't complete successfully.
In uharfbuzz we raise a python exception which is then caught by FontTools and logged. However, uharfbuzz doesn't know what the actual problem for the nullptr was so it simply raises a generic RepackerError:
https://github.com/harfbuzz/uharfbuzz/blob/c11c7ba4ce6a67b72ac2d41a64f763f3b9d27129/src/uharfbuzz/_harfbuzz.pyx#L1219

It would be nice to give fonttools users more context as to why the hb-repacker failed and fonttools had to fall back to its pure-python serializer.

I see one can define the HB_DEBUG_SUBSET_REPACK macro and enable debugging messages, but they are a bit verbose, and i'm afraid to enable that in uharfbuzz all the time as it may slow down execution. Also hb-debug always prints to stderr if I'm not mistaken, so from python we would't be able to capture that and set it as exception object's string, but it'd just be outputted to the stderr.

Would it be possible to add some api to also return the error message from hb_subset_repack_or_fail when the latter fails?

[garretrieger]

I'm thinking we should have the harfbuzz api return an error code which will indicate the specific reasoning for the failure (ie. memory allocation failure vs repacker ran out of iterations etc). Since the repacker operates on a generic object graph getting more details specific than that is probably not helpful (eg. things like link from object x to object y overflowed) since users of fonttools won't be able to map object and subgraph id's to anything recognizable in the font.

[behdad]

A small error enum is fine, but I don't want to get into a huge one.