Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new google fonts check: ensure families can render language sample text #3605

Closed
m4rc1e opened this issue Feb 10, 2022 · 7 comments
Closed
Assignees
Labels
New check proposal We expect new check proposals to include a detailed rationale description and a suggested check-id P2 Important
Milestone

Comments

@m4rc1e
Copy link
Collaborator

m4rc1e commented Feb 10, 2022

We recently added a lang package which is used to generate the new language fields in each metadata.pb file. We must check that the fonts can render each language's sample text, https://github.com/googlefonts/gftools/blob/main/Lib/gftools/lang/languages/aa_Latn.textproto.

These sample texts are a new feature. I think they work correctly for noto, https://fonts.google.com/noto/specimen/Noto+Serif+Kannada.

@felipesanches
Copy link
Collaborator

can we create a separate python package containing those textproto files? Just like we did for glyphsets.

gftools is too big to be added as a fontbakery dependency

@felipesanches felipesanches added New check proposal We expect new check proposals to include a detailed rationale description and a suggested check-id P2 Important labels Feb 10, 2022
@felipesanches felipesanches added this to the 0.8.8 milestone Feb 10, 2022
@m4rc1e
Copy link
Collaborator Author

m4rc1e commented Feb 11, 2022

Agreed but we'll need to coordinate this first with the internal team.

I know this is a hack but for gftools, I use a script which pulls the data I need from google/fonts.

@felipesanches
Copy link
Collaborator

OK. I will start prototyping this by:

  • creating a separate python package for the language data on my own github account (not yet to be hosted on the googlefonts organization).
  • Then I'll publish it on pypi and use it in fontbakery to implement this new check.
  • Then I'll send a PR to gftools to also use this new package. So that you can review and give some feedback.
  • Finally I'll ask @davelab6 to accept migrating the repo from felipesanches/languages to googlefonts/languages.

All of this will happen without touching the language files currently hosted at google/fonts repo, so it should not affect internal Google Fonts team usage of those files. In the future we can keep track of those files and keep the separate module in sync, or the internal Google team may actually start using the module, removing the files from the google/fonts repo. In both scenarios, all is good, I think.

felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
This python module provides an API with data about languages/regions/scripts for use in the language-support categorization of the font families in the Google Fonts collection.

You can also directly access the raw **textproto** files on the `Lib/languages/data` directory:
* [`languages`](https://github.com/felipesanches/languages/tree/main/Lib/languages/data/languages)
* [`regions`](https://github.com/felipesanches/languages/tree/main/Lib/languages/data/regions)
* [`scripts`](https://github.com/felipesanches/languages/tree/main/Lib/languages/data/scripts)

Most of the code in this project was copied from the `gftools` repository (https://github.com/googlefonts/gftools/) so that language/region/script data can be easily available to all our tools without having to also get the large dependency tree of `gftools`. The most immediate user of this module is `Font Bakery`, which needs to validate language support on font binaries being checked. (see fonttools/fontbakery#3605)

The second obvious user of this `languages` module will be `gftools` itself. I'll be sending a pull request soon.

Language/region/script definitions are still being gradualy updated on the `google/fonts` repo, on its **lang/** directory (https://github.com/google/fonts/tree/main/lang) and this `languages` module will try to be kept in sync.

Ideally at some point this module would become the main place to update these definitions, avoiding data duplication and guaranteeing uniformity across tools. But that will require coordination with the Google Fonts team, so I hope this module can serve, for now, as a prototype for such proposed integration.
@felipesanches
Copy link
Collaborator

The repo with my initial implementation of the module is here:
https://github.com/felipesanches/languages

Sample usage:

from languages import lang_support
langs = lang_support.LoadLanguages()
numerals = langs["yi_Hebr"].exemplar_chars.numerals
assert numerals == '- ‑ , . % ‰ + 0 1 2 3 4 5 6 7 8 9'

scripts = lang_support.LoadScripts()
assert scripts["Tagb"].name == 'Tagbanwa'

regions = lang_support.LoadRegions()
br = regions["BR"]
assert br.name == 'Brazil'
assert br.region_group == ['Americas']

font = TEST_FILE('nunito/Nunito-Regular.ttf')
supported = lang_support.SupportedLanguages(font)
langs = [supported[i].name for i, _ in enumerate(supported)]
assert len(langs) == 225
assert 'Lithuanian' in langs

@felipesanches
Copy link
Collaborator

A python module called "languages" already exists: https://pypi.org/project/Languages/

So I'll have to give it a different name. I may use something like GFLanguages. Any other suggestions?

@felipesanches felipesanches self-assigned this Feb 16, 2022
felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
as there was already a `languages` python module on PyPI
(fonttools/fontbakery#3605 (comment))
felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
as there was already a `languages` python module on PyPI
(fonttools/fontbakery#3605 (comment))
felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
as there was already a `languages` python module on PyPI
(fonttools/fontbakery#3605 (comment))
felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
as there was already a `languages` python module on PyPI
(fonttools/fontbakery#3605 (comment))
felipesanches added a commit to googlefonts/lang that referenced this issue Feb 16, 2022
as there was already a `languages` python module on PyPI
(fonttools/fontbakery#3605 (comment))
@felipesanches
Copy link
Collaborator

@felipesanches
Copy link
Collaborator

Screenshot from 2022-02-18 04-24-55

Screenshot from 2022-02-18 04-25-22

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Feb 18, 2022
com.google.fonts/check/metadata/can_render_samples

Check that the fonts can render the sample texts for all languages specified on METADATA.pb, by using the new `gflanguages` module.
(issue fonttools#3605)
felipesanches added a commit to felipesanches/fontbakery that referenced this issue Feb 18, 2022
com.google.fonts/check/metadata/can_render_samples

Check that the fonts can render the sample texts for all languages specified on METADATA.pb, by using the new `gflanguages` module.
(issue fonttools#3605)
felipesanches added a commit to felipesanches/fontbakery that referenced this issue Feb 18, 2022
com.google.fonts/check/metadata/can_render_samples

Check that the fonts can render the sample texts for all languages specified on METADATA.pb, by using the new `gflanguages` module.
(issue fonttools#3605)
felipesanches added a commit that referenced this issue Feb 18, 2022
com.google.fonts/check/metadata/can_render_samples

Check that the fonts can render the sample texts for all languages specified on METADATA.pb, by using the new `gflanguages` module.
(issue #3605)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New check proposal We expect new check proposals to include a detailed rationale description and a suggested check-id P2 Important
Projects
None yet
Development

No branches or pull requests

2 participants