File size diet? :) #46
Comments
|
hi Dave is this question related to Roboto only? Most glyphs that can be made as composites are made as so in Roboto. |
|
That is so. All of the floating diacritics are composites. There are no flipped ones I can see From looking at the composites in my inspector.
Glyphs like Ø , H-Bar, D and L w/bar, [Probably 20 glyphs] could be redone from whole additional contours to the source glyph’s contour with the addition of new glyphs for bars and the compositing code. I.e. a separate bar composites to an O, smaller than a new glyph with and O and a bar through it.
Then, the way I talked DC into this for Extremo, is that the file savings go up with each master, because the variations are reusing the O’s and bar’s gvars, and the composite data, more efficiently than a whole ‘nother set of gvars for an Ø contour.
After doing this for all the masters in the design space, a new variable could be given to Mike with a list of glyphs whose hints would be removed. These glyphs would then have the hinting of composites made and tested. Mike will correct me if needs be.
It could then go to Marc for retesting to make sure nothing went wrong with the old stuff, and that the new stuff is acceptable.
I’d suggest we’d need two-three weeks to complete this with what else is on the stove, but’s be glad to do so, if DC will give us schedule release from the original deadline.
|
|
It would be good to ship the fonts and then ship an update with a smaller file size in the following quarter. |
|
A survey of the target glyphs (red). And a couple of alts I think we're adding to Extremo (green).
|
|
The plan then is for Mike to finish and deliver a version of Classic.vf by the 6th of March. By Mid-march, we'll have a new source with additional composites and then, as schedule permits, or is required, Mike will produce a new hinted version that same as delivered but with these (27) red glyphs composited. |
|
Doesn’t HarfBuzz fall back to Unicode Normalization Form D (NFD) when an input code point is missing from the font’s from unicodedata import normalize
for codepoint in range(0, 0x110000):
c = chr(codepoint)
if c != normalize("NFD", c):
print("U+%06X" % codepoint) |
|
Doesn’t apply, Far as I can see |
|
I think there is definitely something fishy going on here :) In a cosmic coincidence, earlier this month I'd (a) learned that fontmake or at least vttLib doesn't build fonts that use flipped composites, and (b) requested a "Global Latin" glyph set definition from @moyogo (gftools/pull/177) to better define support for African languages that use an Extended Latin set. For example, there are 6 characters missing from the Google Fonts "Core" Latin set latin_unique-glyphs.nam that are needed to support Yoruba: @rsheeter did some research into this, and indeed, after adding those 6 characters to both Open Sans and Roboto's core subsets, which support 5 glyphs (no combining macron) for the 6 chars, then Roboto adds ~100 glyphs whereas Open Sans adds only 4!
Rod is wondering if Roboto has been built in a way that adds composed glyphs for things we nowadays expect to be done with the unicode composition that @brawer described. I am thinking that indeed this is the case, and that while a full ttfdiet may be too aggressive, a light diet may indeed yield significant file size savings. |
|
To give a specific example, if I subset to a, b, combining grave, combining accent I expected to see the combinations with grave/accent done by layout but Roboto does some via actual glyphs. |
|
I see now. This should work with Roboto, and its single set of combining accents, without a problem. As long as the optical centers of glyphs and diacritics are aligned and no glyph-specific horizontal or vertical refinements are required. I think that’s the way it’s always been, but perhaps type designers have not trusted the quality compared to refined glyph construction. So do we have refining capabilities for every single individual glyph? Moving on to more Complex requirements, e.g. in Roboto Extremo, there are five sets of combining accents to compensate for different design requirements of the cases, the above and below accents and those involved in stacking, all of which, once projected out into a larger design space, put too much stress on one accent working for all combinations. I like to know as much as possible about saving space as I can but on the other hand my goal is to produce the same quality Across the entire design space as exists in the default. |
|
To give a little more context, if I compare subsetting to latin with/without the six added codepoints for Yoruba (aside: actually only 4 are for Yoruba) the size delta for Roboto is way out of line with the rest of our library. +6 codepoints in latin for Open Sans adds 4 gids (230 => 234) and increases woff2 filesize by 100-200 bytes (1-2%). The woff2 filesize is the real concern. I'm hoping it's possible to adjust Roboto to perform closer to Open Sans when we add combining characters to the subset. |
Does Roboto perhaps implement combining characters with |
|
As far as I know, Roboto as delivered can go either way. We use an open source tool called Glyph Builder, by Ben Kiel I think, that synthesizes precomposed characters for the purpose of serving everyone. You may serve it to any system and it’ll work. Or you may subset it, removing all the composed accents, and serve it to have HarfBuzz’s fall back generate missing glyphs.
Santiago and I will meet this a.m., and update you all later today on providing the lightest Roboto by both methods discussed here, removal of composed glyphs, and addition of more composites that save glyph and gvar space.
Thanks.
|
|
I see https://benkiel.com/typeDesign/ has |
|
Yes, the latter is what we use! |
|
We are going through the steps of undoing the error we inherited on this.
|
Yes, but many applications (most actually) will check the |
|
Very curious about the actual space savings on a large font like Roboto, especially in WOFF format. If the improvement was huge, it might be worth building subsetted fonts for certain environments (eg. Google Fonts, which afaik can support browser-specific fonts); a large size difference might also be a reason for other browsers and apps to implement the same fallback logic like Chrome or LibreOffice. But if it’s only a small difference, it’s clearly not worth bothering. |
|
I've removed all composite glyphs which have a unicode decomposition for Roboto Regular Hinted. The file is around ~8% smaller when we factor in all other pyftsubset optimisations. For Chrome and modern browsers, the results are ok. There are some mark positioning issues but these can be solved.
Win 10 Chrome 70 Internet Explorer 11 is a mess.
Win 7 ie11 If we ignore the shifts, we can see the accented glyphs are using a fallback font. If we do go ahead with this diet idea, imo it has to be done by the gf backend as Sascha suggested and only served to particular browsers. I don't think @sannorozco and TN should be doing this manually. The benefit of making this server side is we can apply it to other families as well. |
|
I've made a minimal test case for those who are interested. OS X tests:
Chrome
Safari Win tests:
Chrome
Edge 80
Firefox
IE 11 Seems only Chrome and Edge are ok. |
|
@rsheeter please can you provide a way for @m4rc1e to reproduce the filesize increase you found when adding the Yoruba characters that appears unique to Roboto statics? The work being done in this thread is in response to that concern, but since this is taking much longer to complete than anticipated, I would like to decouple the two tasks:
|
|
On a 1/1 call just now, @m4rc1e also proposed that perhaps its better to do the diet'ing like https://github.com/twardoch/ttfdiet as a post-processing step, which is therefore applicable to any font project, and not in source files - since pyftsubset retains hinting So, we probably need to checkout the current master into a holding 'diet' branch, then revert to the last commit before the diet effort started, and continue from there. |
Ah, @rsheeter pointed out that the information required to do this is already on this thread, in 2 parts. I wrote,
and Rod wrote,
Rod also wrote earlier,
So, to move forward, I propose that @sannorozco and @dberlow take a step back and research how is Open Sans constructed so that adding the 6 characters does not increase the filesize by ~20%, and apply Marc's minimal test case to Open Sans to confirm that it renders as expected. Does that sound good? |
|
When i open Open Sans in Robofont, all the composites are there, and it has
a MUCH SMALLER glyph repertoire than Roboto.
So,
Where are the files that show Roboto increasing by 20% from the addition of
6 glyphs? who made them? How?
Santiago has dieted, and remapped the glyph indexes and it going to run a
test to make sure all the glyphs show up this time.
And, reverting to the pre-diet branch is going to lose the last few weeks
of Mike's efforts because we stopped optimizing on the pre-diet version as
it was being discarded as an option.
Our way to a complete source now is to replicate the composites in the
current version, add the hints for those composites back in, and go forward
from there?
The good news is, we can now rearrange the glyph indexes again if we need
to and that's open source.
Let us know, thanks.
…On Thu, Mar 26, 2020 at 11:32 AM Dave Crossland ***@***.***> wrote:
a way for @m4rc1e <https://github.com/m4rc1e> to reproduce the filesize
increase you found when adding the Yoruba characters that appears unique to
Roboto statics
Ah, @rsheeter <https://github.com/rsheeter> pointed out that the
information required to do this is already on this thread, in 2 parts. I
wrote,
For example, there are 6 characters missing from the Google Fonts "Core"
Latin set latin_unique-glyphs.nam
<https://github.com/googlefonts/gftools/blob/master/Lib/gftools/encodings/latin_unique-glyphs.nam>
that are needed to support Yoruba:
0x0300 COMBINING GRAVE ACCENT
0x0301 COMBINING ACUTE ACCENT
0x0303 COMBINING TILDE
0x0304 COMBINING MACRON
0x0309 COMBINING HOOK ABOVE
0x0323 COMBINING DOT BELOW
and Rod wrote,
for family in Roboto OpenSans; do
pyftsubset ${family}-Regular.ttf \
--unicodes="U+0061-0062,U+0x0300-0301" \
--output-file=${family}-Regular_combtest.ttf;
ttx -o ${family}-Regular_combtest.ttx ${family}-Regular_combtest.ttf;
done
Rod also wrote earlier,
+6 codepoints in latin for Open Sans adds 4 gids (230 => 234) and
increases woff2 filesize by 100-200 bytes (1-2%).
+6 codepoints in latin for Roboto adds 104 gids (248 => 352) and increases
woff2 filesize by 2400-2800 bytes (17-22%).
The woff2 filesize is the real concern
So, to move forward, I propose that @sannorozco
<https://github.com/sannorozco> and @dberlow <https://github.com/dberlow>
take a step back and research how is Open Sans constructed so that adding
the 6 characters does not increase the filesize by ~20%, and apply Marc's
minimal test case to Open Sans to confirm that it renders as expected.
Does that sound good?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#46 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAO5VDWSL56L6S4XHPOQRVLRJNYPVANCNFSM4K3QTFVQ>
.
|
|
Enclosed is a compare of the two repertoires.
…On Thu, Mar 26, 2020 at 4:56 PM David Berlow ***@***.***> wrote:
When i open Open Sans in Robofont, all the composites are there, and it
has a MUCH SMALLER glyph repertoire than Roboto.
So,
Where are the files that show Roboto increasing by 20% from the addition
of 6 glyphs? who made them? How?
Santiago has dieted, and remapped the glyph indexes and it going to run a
test to make sure all the glyphs show up this time.
And, reverting to the pre-diet branch is going to lose the last few weeks
of Mike's efforts because we stopped optimizing on the pre-diet version as
it was being discarded as an option.
Our way to a complete source now is to replicate the composites in the
current version, add the hints for those composites back in, and go forward
from there?
The good news is, we can now rearrange the glyph indexes again if we need
to and that's open source.
Let us know, thanks.
On Thu, Mar 26, 2020 at 11:32 AM Dave Crossland ***@***.***>
wrote:
> a way for @m4rc1e <https://github.com/m4rc1e> to reproduce the filesize
> increase you found when adding the Yoruba characters that appears unique to
> Roboto statics
>
> Ah, @rsheeter <https://github.com/rsheeter> pointed out that the
> information required to do this is already on this thread, in 2 parts. I
> wrote,
>
> For example, there are 6 characters missing from the Google Fonts "Core"
> Latin set latin_unique-glyphs.nam
> <https://github.com/googlefonts/gftools/blob/master/Lib/gftools/encodings/latin_unique-glyphs.nam>
> that are needed to support Yoruba:
>
> 0x0300 COMBINING GRAVE ACCENT
> 0x0301 COMBINING ACUTE ACCENT
> 0x0303 COMBINING TILDE
> 0x0304 COMBINING MACRON
> 0x0309 COMBINING HOOK ABOVE
> 0x0323 COMBINING DOT BELOW
>
> and Rod wrote,
>
> for family in Roboto OpenSans; do
> pyftsubset ${family}-Regular.ttf \
> --unicodes="U+0061-0062,U+0x0300-0301" \
> --output-file=${family}-Regular_combtest.ttf;
> ttx -o ${family}-Regular_combtest.ttx ${family}-Regular_combtest.ttf;
> done
>
> Rod also wrote earlier,
>
> +6 codepoints in latin for Open Sans adds 4 gids (230 => 234) and
> increases woff2 filesize by 100-200 bytes (1-2%).
>
> +6 codepoints in latin for Roboto adds 104 gids (248 => 352) and
> increases woff2 filesize by 2400-2800 bytes (17-22%).
>
> The woff2 filesize is the real concern
>
> So, to move forward, I propose that @sannorozco
> <https://github.com/sannorozco> and @dberlow <https://github.com/dberlow>
> take a step back and research how is Open Sans constructed so that adding
> the 6 characters does not increase the filesize by ~20%, and apply Marc's
> minimal test case to Open Sans to confirm that it renders as expected.
>
> Does that sound good?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#46 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAO5VDWSL56L6S4XHPOQRVLRJNYPVANCNFSM4K3QTFVQ>
> .
>
|
You can not attach files via email, as you tried here - you must visit the #46 page and drag and drop the files into your comment. |
The files were not shared, but the command line to reproduce them has been shared, so I am requesting that you/santiago (re)make them, so that you can be sure how they were made and fully investigate and compare.
Understood. Let's roll forwards!
After looking at the code (https://github.com/TypeNetwork/Roboto/blob/delivery-review/Scripts/mapper-VTT-gids.py) then this needs to be polished and packaged in order to be used again; I'll file a separate issue for that :) |
|
Where is,the command line please? Glyph repertoires below? Open Sans is top, Roboto below.
|
|
I explain how to construct the command here, #46 (comment) |
|
The original Roboto has a dedicated webfont family which is produced by post processing the master fonts. This is the version of Roboto we use on Google Fonts. If I run the post processing scripts which creates this family on our VFs, we get a file size of 900kb. The static ttfs we currently serve are 2.1mb. Imo, this is a massive win. The VFs also contain Mike's VTT work. |
I wonder if there are any opportunities for file size reduction by better use of components, ccmp diacritic glyph construction, etc?
The text was updated successfully, but these errors were encountered: