WIP: Fix font selection

[tacaswell]

This pulls the commits from #8607 and #8551 together and starts to add starts to add code to standardize extracting the weight and style from the 4 places it could be stashed

Eventually this the weight / style selection code will look something like

fc-list : file > /tmp/font_list.txt

from matplotlib import ft2font
from matplotlib import font_manager
from pathlib import PurePath
with open('/tmp/font_list.txt', 'r') as fin:
    font_paths = [_.strip().strip(':') for _ in fin.readlines()]

fonts = []

for fn in font_paths:
    fn = PurePath(fn)
    if fn.suffix not in ('.otf', '.ttf'):
        continue
    try:
        ft = ft2font.FT2Font(str(fn))
        reasonable_effort_style = [v.lower().split() for v in get_all_style(ft.get_sfnt())]
        reasonable_effort_style += [ft.style_name.lower().split()]
        hit_weight = set(k for v in reasonable_effort_style for k in font_manager.weight_dict
                         if k in v)
        print(ft.family_name, reasonable_effort_style, hit_weight, sep=' | ')
    except RuntimeError:
        print('failed')

attn @cy18 @Rufflewind

[anntzer]

looks like you chmod+x'd the file accidentally.

[anntzer]

I guess it was supposed to be called "_ttf_header_parse.py" (not tff)

[tacaswell]

Yes, that would explain why imports were failing on me...

[jhelie]

is this always meant to be parsed with Python 3?

[tacaswell]

It should be both py2 and py3 (but I work in 3 day-to-day). Did I accidentally use a py3 only feature?

[jhelie]

the lgtm PR check is flagging some encoding issue. Python 2 uses ascii by default and Python 3 utf-8 so that would explain why that's not an issue for you, but in a 2 environment I think this would be a problem

[tacaswell]

In things that are fun, I think I found a bug in the metadata in the ubuntu fonts

Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:09:58) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from matplotlib._ttf_header_data import decode_name_table

In [2]: from matplotlib import ft2font

In [3]: ft2 = ft2font.FT2Font('/usr/share/fonts/truetype/ubuntu-font-family/Ubuntu-M.ttf')

In [4]: list(decode_name_table(ft2.get_sfnt()))
Out[4]: 
[(('copyright', 'English'),
  'Copyright 2011 Canonical Ltd.  Licensed under the Ubuntu Font Licence 1.0'),
 (('family_name', 'English'), 'Ubuntu'),
 (('sub_family', 'English'), 'Medium'),
 (('ufi', 'English'), 'DaltonMaagLtd: Ubuntu Medium 0.83'),
 (('full_font_name', 'English'), 'Ubuntu Medium'),
 (('version', 'English'), 'Version 0.83'),
 (('ps_name', 'English'), 'Ubuntu-Medium'),
 (('trademark', 'English'),
  'Ubuntu and Canonical are registered trademarks of Canonical Ltd.'),
 (('manufacturer', 'English'), 'Dalton Maag Ltd'),
 (('designer', 'English'), 'Dalton Maag Ltd'),
 (('description', 'English'),
  "The Ubuntu Font Family are libre fonts funded by Canonical Ltd on behalf of the Ubuntu project. The font design work and technical implementation is being undertaken by Dalton Maag. The typeface is sans-serif, uses OpenType features and is manually hinted for clarity on desktop and mobile computing screens.\r\n\r\nThe scope of the Ubuntu Font Family includes all the languages used by the various Ubuntu users around the world in tune with Ubuntu's philosophy which states that every user should be able to use their software in the language of their choice. The project is ongoing, and we expect the family will be extended to cover many written languages in the coming years."),
 (('url_vendor', 'English'), 'http://www.daltonmaag.com/'),
 (('url_designer', 'English'), 'http://www.daltonmaag.com'),
 (('typographic_family_name', 'English'), 'Ubuntu'),
 (('typographic_sub_family', 'English'), 'Medium'),
 (('compat_ful_font_name', 'English'), 'Ubuntu Medium'),
 (('copyright', 'English (United States)'),
  'Copyright 2011 Canonical Ltd.  Licensed under the Ubuntu Font Licence 1.0'),
 (('family_name', 'English (United States)'), 'Ubuntu Light'),
 (('sub_family', 'English (United States)'), 'Bold'),
 (('ufi', 'English (United States)'), 'DaltonMaagLtd: Ubuntu Medium 0.83'),
 (('full_font_name', 'English (United States)'), 'Ubuntu Medium'),
 (('version', 'English (United States)'), '0.83'),
 (('ps_name', 'English (United States)'), 'Ubuntu-Medium'),
 (('trademark', 'English (United States)'),
  'Ubuntu and Canonical are registered trademarks of Canonical Ltd.'),
 (('manufacturer', 'English (United States)'), 'Dalton Maag Ltd'),
 (('designer', 'English (United States)'), 'Dalton Maag Ltd'),
 (('description', 'English (United States)'),
  "The Ubuntu Font Family are libre fonts funded by Canonical Ltd on behalf of the Ubuntu project. The font design work and technical implementation is being undertaken by Dalton Maag. The typeface is sans-serif, uses OpenType features and is manually hinted for clarity on desktop and mobile computing screens.\r\n\r\nThe scope of the Ubuntu Font Family includes all the languages used by the various Ubuntu users around the world in tune with Ubuntu's philosophy which states that every user should be able to use their software in the language of their choice. The project is ongoing, and we expect the family will be extended to cover many written languages in the coming years."),
 (('url_vendor', 'English (United States)'), 'http://www.daltonmaag.com/'),
 (('url_designer', 'English (United States)'), 'http://www.daltonmaag.com'),
 (('typographic_family_name', 'English (United States)'), 'Ubuntu'),
 (('typographic_sub_family', 'English (United States)'), 'Medium')]

In [5]: 

not that the family_name / sub family is inconsistent. One place it is ('Ubuntu', 'Medium') and another it is ('Ubuntu Light', 'Bold'), hence the extra cut-out. Have to sort out where upstream is to report a bug....

[anntzer]

FWIW, another case that would be nice to cover is the bold face in the computer modern unicode font (https://ctan.org/pkg/cm-unicode?lang=en, also available as ttf elsewhere online) which has a family name of "CMU Serif Bold Extended Roman"); the line

    weight = next((w for w in weight_dict if sfnt4.find(w) >= 0), None)

in font_manager.py will currently match the font as either weight = 500 (roman) or weight = 700 (bold) depending on the order of iteration on the weight_dict dictionary :-) (on Py3.6+, this means roman (which is incorrect), as it comes first in the dict).

[anntzer]

See also https://github.com/anntzer/mplfccache which relies on fontconfig to generate (once) the matplotlib font cache.

[anntzer]

Also https://github.com/anntzer/freetypybind/blob/master/lib/freetypybind/query_face.py#L499 for what is I think a mostly complete reimplementation of what fontconfig does, plus support for some common language tags (python ships a bunch of encodings without actually documenting them anywhere afaict (https://github.com/python/cpython/tree/master/Lib/encodings) :-))

[tacaswell]

Closing as very old and @anntzer has the font config algorithm in python.