Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gen-metadata] More aggressively declare subsets in METADATA files #602

Open
davelab6 opened this issue Aug 18, 2022 · 7 comments
Open

[gen-metadata] More aggressively declare subsets in METADATA files #602

davelab6 opened this issue Aug 18, 2022 · 7 comments
Assignees

Comments

@davelab6
Copy link
Member

davelab6 commented Aug 18, 2022

fonts.google.com launched a new specimen page last week, that reuses the improvements developed for the fonts.google.com/noto section specimen pages last year.

However, this has highlighted a problem with Kumbh Sans and Nabla, where they had Google Fonts Latin Plus glyph set support, but their METADATA files didn't declare the math subset, and therefore the fonts API stripped those characters from the fonts served to the specimen page, which was trying to show the full glyph set. The Kumbh Sans designer reported this (google/fonts#5067) with a screenshot.

In google/fonts#5095 I add the math subset to the 2 families we've noticed this issue surfaced, and I believe the root cause is that the METADATA generator has a threshold for when it auto-adds subsets to a METADATA file, such that when a font has only 1% of the math subset's characters, it won't add it but when 100% of latin-core subset is in the font, it is added. I forget the details on the threshhold, I think it was set per subset.

It seems that we probably ought to be more aggressive about declaring subsets, like if even 1 character from a subset exists in any font in the family, it should be declared in METADATA, and then using comments to include the % the tool found to inform later hand curation of the METADATA file.

We probably then need to make a special version of the METADATA generator to lint all the existing library and then PR a bunch of updates to a bunch of families.

It would also be good for the linter to tell us what unicode characters are in any font in a family that are not in any declared subset encoding, and not in any encoding (https://github.com/googlefonts/glyphsets/tree/main/Lib/glyphsets/encodings)

@rsheeter
Copy link
Contributor

rsheeter commented Aug 22, 2022

Would be good to improve CJK detection as well. We have an ongoing issue where that tool adds many CJK subsets when only one is appropriate. The "I only have an hour" approach would be to simply prompt the user if this occurs and make them pick which one(s) they want. Given more time, identify characters that strongly suggest (high frequency, only in that script) support for specific CJKs and use them to autopick.

@m4rc1e
Copy link
Member

m4rc1e commented Aug 31, 2022

We discussed this issue in our team meeting last Friday.

We came to the conclusion that decreasing the number of glyphs needed in order to activate a subset won't work because users have the ability to search for fonts by subsets. If we start enabling subsets just because the font contains a single glyph within a given subset, users will get annoyed because the font doesn't fully support the subset fully.

Screenshot 2022-08-31 at 12 41 28

Language drop down enables users to search for fonts by subset

@nathan-williams Is there a reason why we're not using the font's cmap in order to construct the glyphs palette? if we simply used the cmap, we wouldn't get fallback glyphs.

Screenshot 2022-08-31 at 12 55 51

nabla on sandbox showing fallback glyphs because they don't exist in the font being used to display the glyphs

cc @chrissimpkins @davelab6

@davelab6
Copy link
Member Author

davelab6 commented Sep 8, 2022

However, the CMAP of the fonts in the API is a subset of the CMAP of the upstream fonts, because the upstream HAS the glyphs:

Screen Shot 2022-09-08 at 9 39 29 AM

But because the subset that contains the character isn't declared, the character is removed by the API and so unavailable to the Catalog:

Screen Shot 2022-09-08 at 9 39 44 AM

I personally believe the subsets that cover the glyphs in the font should be enabled, so that all glyphs can be accessed; the language dropdown should be fixed, so that filtering happens by actual script support, even if the subset is included - because the API deleting glyphs is a foundational issue with the API that should be fixed as a priority, and then the Catalog should accommodate a correct API.

But @vv-monsalve and @m4rc1e make a fair argument that fixing the Catalog glyph table preview issue by modifying the API subsets configured for families will break the Catalog language filter, and the Catalog language filter is a more important Catalog feature, so we should not modify the subsets as they currently work, and we should modify the Catalog glyph table preview functionality to fix its issue.

Taking a higher level view, I think there are 3 semantics which are being conflated in the GF system architecture:

  1. unicode-range subset definitions, which are called encodings in the github.com/googlefonts/glyphsets repo
  • These are long overdue for a detailed analysis and investigation, based on maturity of the glyphsets in that repo which are what type designers are expected to develop
  1. catalog unicode standard segmentation/grouping for the glyph table preview, which is in the github.com/googlefonts/lang repo developed by @twardoch
  1. actual language support of a family
  • I propose this should be addressed by advancing the shaperglot project and then integrating that into the GF back-end

We should bring these into harmony and use the appropriate data for the appropriate Catalog feature, but that's a larger (2023) effort.

Therefore I think the short term solution to the 'unavailable glyphs being seen' issue is not, as per this issue title, to _more aggressively declare subsets in METADATA files), but rather, the Catalog front-end code team (@nathan-williams :) should fix it in that code.

But to start on bringing those things into harmony, for (1) we should get better data on which characters exist in which font files that do not exist in any characters defined in the families' METADATA subset associations, and I'd like to ask @m4rc1e to gather that data in a Google Sheet

@davelab6
Copy link
Member Author

davelab6 commented Sep 8, 2022

Ha, just after posting this, I caught up on google chatrooms, and Nate said he already wrote a patch that excludes unavailable glyphs, which will be applied the next time such a family is pushed :)

Also @chrissimpkins noted the math subset was added to Nabla (here) - but it turns out that "per mille sign" glyph is not from the math subset..... its actually in Adlam!

So that seems weird and makes a detailed analysis and investigation of the subsets more important, although not urgent.

@rsheeter
Copy link
Contributor

rsheeter commented Sep 8, 2022

Please don't aggressively add subsets that are barely supported, it'll confuse all sorts of things.

@rsheeter
Copy link
Contributor

rsheeter commented Sep 8, 2022

And +1 for the FE to fix. Long term solution is to move away from subsets as they exist today entirely.

@nathan-williams
Copy link
Contributor

nathan-williams commented Sep 8, 2022

Is there a list of affected families? They will have to be version bumped once the fix is rolled out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants