IFTB draft

[skef]

I went through my IFTB commits from earlier this year and reorganized them against the current main branch. Unfortunately the result doesn't quite build yet, I think because the separate IFTB document (which does build) isn't in W3C's bibliography.

A few notes:

1. I left the rangefile documented even though I think we should remove it, because I figure the documentation should generally track the reference implementation.
2. Some sections in Overview.bs have "XXX" notes indicating further changes that need to be made.
3. As I noted earlier this is all very rough.

---

Preview | Diff

[svgeesus]

Hi @skef could you please go to your W3C account and then link your GitHub account to it. That enables the IPR bot to recognize you. Thanks!

[garretrieger]

Here's some high level thoughts, we can likely discuss a lot of this during the upcoming TPAC meeting:

1. It’s likely that the memory usage for the unpacked font will be quite high (eg. Noto Serif SC base file is ~1 mb uncompressed). This is partially because the format retains gids and ends up with lots of zero’d out regions for glyphs that aren’t yet present. I wonder if there are some ways that we can avoid doing this? I suspect browsers are going to be pretty concerned about overall memory usage.

   a. Related to this there’s currently a requirement to change the first 4 bytes of the version tag on the font file before passing the font to a rendering process. This may end up necessitating a full copy of the fonts data prior to passing the font on to the rendering process further increasing memory usage. We might consider relaxing this requirement if the rendering process is aware of the IFTB version tag and capable of correctly processing the file.

2. The initial file can be large (even after compression, I saw 200k-400k in the fonts that are part of the demo). Part of this is because all non-glyph data is transferred in full as part of the initial file. I wonder if it might make sense to try and have data from additional tables that are indexed by glyph id sent in the chunk files (for example vmtx/hmtx). See a breakdown of table sizes in the initial file here: https://docs.google.com/spreadsheets/d/1AK43VvY4LAEuttNPLoXulmedjRs8y44A4ZtN0lcT2QU/edit?resourcekey=0-uRS36zkMiyBmi8fG_U70ZA#gid=0

3. Font collections: when developing patch subset a concern was raised about making sure it supported font collections. So we’ll possibly need to think about what it would take to support font collections in IFTB.

4. File extensions: would it make sense to have a recommended file extensions for the various new file formats (instead of reusing .otf and .woff2)?

   a. Related we almost certainly want to introduce a new format(...) for IFTB files in @font-faces. This would allow for feature detection in the browser (eg. client who doesn’t understand IFTB files will skip loading and fallback to the next URL).

5. The feature chunk mapping mechanism allows one to map from (feature, chunk idx) -> additional chunk idx. However, this is a bit limiting. If we had the ability to map from (feature, {chunk idx set}) -> additional chunk idx. Then it would be possible to conditionally include glyphs that are only needed when combinations of chunks are present (eg. ligature that can only be activated when glyphs from two different chunks are available). This would be optional, a simpler encoder could simply not utilize it (sticking with the simpler (feature, chunk idx) approach). An advanced encoder could likely utilize this to produce more granular chunks when encountering multi glyph substitution rules.

6. In the patch subset specification when referring to integrating with browsers we did so by referencing the fetch spec (eg. to describe how to make a request). In this spec since the integration with the browser is a lot simpler you might be able to get away with not even referencing browsers at all. Just focus on describing the operations that a client could do (initial request, extending the font via loading chunks).

[garretrieger]

You should remove Myles and add yourself as an editor.

[garretrieger]

Can you clarify here if chunks that are added by a feature are allowed to trigger further additions on later features? This process will need to be very precisely defined so that multiple implementations will always arrive at the same result.

[garretrieger]

[[IFTB]] won't work until we've commited the IFTB document. As a workaround you can instead use a regular old link <a href="IFTB.html">...</a> until the document is committed.

[garretrieger]

Brotli level 11 is an implementation detail of the current open source brotli encoder, but not actually something part of the brotli standard. Instead maybe just say maximum quality?

[garretrieger]

Agreed I think we'll end up with tech(incremental-patch) used with patch subset and format(new-iftb-format-name) for IFTB. Then there's no need for the incremental-auto/incremental-range mechanism and related text.

[skef]

I agree but am leaving this as-is for now.

[garretrieger]

Here and elsewhere Offset fields are used be sure to mention what those offsets are relative too.

[garretrieger]

To be as unambiguous as possible it's helpful to define the mapping to bits within a byte using the most and least significant bit (eg. the least significant bit is chunk 0, the most signifcant bit is chunk 7)

[garretrieger]

What is the encoding of the strings? (eg. ascii? utf-8?)

[skef]

I think we should adopt whatever currently makes sense from a w3c perspective. I believe URIs were ASCII for a long time but I don't know if that's still true. We should also be careful about the dollar signs.

[garretrieger]

https://datatracker.ietf.org/doc/html/rfc6570 is probably relevant here. A standardized way to have template parameters in a URL and it also talks about encoding in section 1.6.

[skef]

I looked through this. Given that IFTB would/will be a w3c spec maybe we should consult @svgeesus at a future meeting about the template format. The powers that be might prefer something more elegant than my dollar-sign expansion (although the spec allows for it).

For the time being I've been assuming the encoding will wind up being either ASCII or whatever corresponds to an "encoded" url.

[garretrieger]

It's should be OK to reference an IETF RFC in this spec, we reference quite a few in the patch subset specification. You can add reference like this: [[RFC6570]]. Agreed we can discuss this further at the next meeting.

[garretrieger]

I would clearly mark this part (the 3 options below) as being non-normative since it's just providing advice on how an encoder might be built and not actually laying out requirements that an encoder must follow.

[garretrieger]

Or another option this could be re-framed as things that the encoder may do which are valid. For example it's valid to place a glyph in more than one chunk if needed.

[garretrieger]

This section is probably also non-normative.

[garretrieger]

2. The initial file can be large (even after compression, I saw 200k-400k in the fonts that are part of the demo). Part of this is because all non-glyph data is transferred in full as part of the initial file. I wonder if it might make sense to try and have data from additional tables that are indexed by glyph id sent in the chunk files (for example vmtx/hmtx). See a breakdown of table sizes in the initial file here: https://docs.google.com/spreadsheets/d/1AK43VvY4LAEuttNPLoXulmedjRs8y44A4ZtN0lcT2QU/edit?resourcekey=0-uRS36zkMiyBmi8fG_U70ZA#gid=0

One additional thought regarding this: another approach that could be used to reduce the size of the initial file would be to split the font into a small number of subsets where possible and have each individual subset then augmented by IFTB (using unicode-range in CSS to select the subsets that are needed). For example in the CJK case you could (if it wouldn't break layout rules) split the font into a high usage and low usage subset. Then if a client doesn't need anything from the low usage subset it would only download and augment the high usage subset thereby saving from having to transmit all of the layout and metric information from the low usage glyphs. I don't think spec changes are needed to accommodate this, but probably worth mentioning in the section talking about optimizing the encoder.

[skef]

Starting to work through some of these comments ...

It’s likely that the memory usage for the unpacked font will be quite high (eg. Noto Serif SC base file is ~1 mb uncompressed). This is partially because the format retains gids and ends up with lots of zero’d out regions for glyphs that aren’t yet present. I wonder if there are some ways that we can avoid doing this? I suspect browsers are going to be pretty concerned about overall memory usage.

After thinking about this question for a while I've arrived at some philosophy, or perhaps ideology, for myself: The role or purpose of IFT is to try to improve the network transfer of font data. Accordingly, when considering local storage size we should generally be following developments in general font technology, or influencing such general development but not worry about or attempt unilateral improvements.

There are lots of potential ways of organizing a font file. If one were primarily concerned about persistent storage size, one could store the whole file compressed. If one were primarily concerned about in-memory size, one could store most of the per-glyph data together, perhaps on power-of-two boundaries, so that it could be easily paged in. If one were worried about both, one could do both: Store the per-glyph data in compressed, possibly page-aligned chunks, loading and decompressing only those chunks needed at a given time.

The general direction of font development in recent years has gone in the opposite direction. CFF had the advance in the glyph, glyf has phantom points. OpenType has moved much of this data into separate tables for better access during shaping. And although there is currently a component proposal to the ISO ad-hoc group for reducing file size, it chose to leave htmx and vtmx as they were.

But IFTB is saving substantial amounts of space -- that of any un-merged glyph data. And that it doesn't do so for other tables like hmtx and vmtx is pretty inherent to its GID-preservation-based design. Changing GIDs would (I think) require building knowledge of basically all the shaping tables back into the client-side, making it more complicated than range-request would have been. And experimenting with run-length-encoded h/vmtx, and therefore requiring shapers to be updated to support those formats, seems like beyond the scope of our project.

I suppose the counter-argument would be that web fonts are more ephemeral on a given system than system fonts, so there is more to save space. But I'm not convinced on that basis.

[skef]

Related to this there’s currently a requirement to change the first 4 bytes of the version tag on the font file before passing the font to a rendering process. This may end up necessitating a full copy of the fonts data prior to passing the font on to the rendering process further increasing memory usage. We might consider relaxing this requirement if the rendering process is aware of the IFTB version tag and capable of correctly processing the file.

You're right, this shouldn't be a requirement, nor (as I advocated several months ago) should recalculating the checksums be a requirement.

How about we change the language to suggest that clients be enhanced to use the version directly and ignore the checksums, but indicate that in contexts where that is not possible/desirable the client implementation can make the necessary adjustments?

[skef]

The initial file can be large (even after compression, I saw 200k-400k in the fonts that are part of the demo). Part of this is because all non-glyph data is transferred in full as part of the initial file. I wonder if it might make sense to try and have data from additional tables that are indexed by glyph id sent in the chunk files (for example vmtx/hmtx).

I am open to this discussion but we should think about the implications, not currently discussed in my added documentation (as far as I remember).

With IFT you have initial loads and augmentations. Each of these involves retrieving data you need to render the page correctly. There is an already well-established related question of what to do in the mean time. What many browsers do is render with fallback fonts, resulting in the oft-complained-about web font "flash" or "flicker".

If we continue to include the shaping data in IFTB browsers will have more rendering options, particularly when it comes to augmentation, because they will be able to arrive at the final page layout once they have the base file. Backup glyphs might then be temporarily coerced into the metrics of the IFTB font and replaced when loaded.

Beyond that specific consideration, I'm not sure IFTB will ever be the right format if you're worried about 40k here or 80k there out of initially huge font files like the Noto fonts, just because you will almost inevitably be loading many more glyphs than you'll need for rendering, which will overwhelm those numbers. And looking at your spreadsheet, The plurality of the initial compressed font size is still in the CFF (or glyf/gvar for ttf) table, which I suspect is mostly the result of my crappy encoder. We should think about these files as having a much smaller initial percentage of glyphs.

[skef]

Font collections: when developing patch subset a concern was raised about making sure it supported font collections. So we’ll possibly need to think about what it would take to support font collections in IFTB.

Seems like a huge mess but we can talk about it.

[skef]

File extensions: would it make sense to have a recommended file extensions for the various new file formats (instead of reusing .otf and .woff2)?

".otf" seems easy enough to replace if we want to. I'm more reluctant to move away from ".woff2" for the compressed files for two reasons:

1. WOFF2 already has a separate, internal version number and the currently IFTB use of woff2 is entirely spec-compliant.
2. I worry that if we introduce a new extension for the compressed version that there will be years and years of servers pointlessly gzipping (or whatever) compressed IFTB files followed by clients pointlessly un-gzipping them.

[skef]

The feature chunk mapping mechanism allows one to map from (feature, chunk idx) -> additional chunk idx. However, this is a bit limiting. If we had the ability to map from (feature, {chunk idx set}) -> additional chunk idx. Then it would be possible to conditionally include glyphs that are only needed when combinations of chunks are present (eg. ligature that can only be activated when glyphs from two different chunks are available). This would be optional, a simpler encoder could simply not utilize it (sticking with the simpler (feature, chunk idx) approach). An advanced encoder could likely utilize this to produce more granular chunks when encountering multi glyph substitution rules.

For "traditional" ligature scenarios I'm not sure I'm convinced of the benefit. Are there cases where you'll really be able to group enough related ligatures together to warrant an independent chunk? Or are you thinking it would be worth having single-glyph chunks just for this case?

Maybe looking at some specific ligature scenarios would help convince me.

OTOH I can see how something like this might be more directly relevant to emoji fonts. I haven't dug into emoji specifics very much in thinking about IFTB.

[skef]

I made some updates