Font-weight range seems wrong

[takis]

When using the font-weight property values in Matplotlib I got the impression the weights were all extremely bold. I looked up the font-weight property in the W3C CSS spec and it appears the legal range is 100-900, and 400 is supposed to look "Regular", 700 "Bold".

When using a plot with a Text object, everything above 100 seems to be bold though, as can be seen in the attached image.

http://www.w3.org/wiki/CSS/Properties/font-weight

[takis]

The expected appearance can be seen here:
https://www.google.com/fonts/specimen/Roboto

[mdboom]

Can you provide the code you used to produce the plot in the original post?

Also, what version of matplotlib are you using, which platform, and what version of freetype?

I can't seem to reproduce the problem, but it may just be the particular method I'm using.

[takis]

Sure, I am using Python 3.4.1 with matplotlib 1.4.3 on OSX 10.10.5 (in a virtualenv). Although I generated the test files in IPython Notebook, I originally encountered the problem in a separate Python program.

Here's the example code:

%pylab inline

for ff in ['Roboto', 'courier']:
    f = plt.figure(figsize=(1, 1), dpi=200)
    ax = f.add_subplot(111)
    ax.plot(range(200))
    for fs in range(6, 32, 2):
        for fw in range(100, 1000, 100):
            ax.text(
                (6 * fs) * (fs - 6),
                fw,
                "s %d w %d" % (fs, fw),
                family=ff,
                fontsize=fs,
                weight=fw,
            )
    f.savefig('font_weight_test-%s.svg' % ff)
    f.savefig('font_weight_test-%s.png' % ff)

[takis]

Oh, and the version of freetype according to freetype-config:

(notebook3)ic@MacBookPro grofac (master) $ freetype-config --version
17.4.11
(notebook3)ic@MacBookPro grofac (master) $ freetype-config --ftversion
2.5.5
(notebook3)ic@MacBookPro grofac (master) $ 

[mdboom]

It seems to work fine for me. Have you tried deleting your matplotlib font cache (in ~/.matplotlib). Matplotlib won't pick up newly installed fonts otherwise.

[takis]

There was only one file in that directory (fontList.py3k.cache) which I deleted, but the results appear the same.

As I was getting worried that there might be a problem with the installed fonts I opened the /Library/Fonts map using Finder and they same to be available:

Also FontConfig seems to have access to these fonts:
fc-list |grep -i roboto
/Library/Fonts/Roboto-Black.ttf: Roboto,Roboto Black:style=Black,Regular
/Library/Fonts/Roboto-ThinItalic.ttf: Roboto,Roboto Thin:style=Thin Italic,Italic
/Library/Fonts/Roboto-Thin.ttf: Roboto,Roboto Thin:style=Thin,Regular
/Library/Fonts/Roboto-MediumItalic.ttf: Roboto,Roboto Medium:style=Medium Italic,Italic
/Library/Fonts/Roboto-LightItalic.ttf: Roboto,Roboto Light:style=Light Italic,Italic
/Library/Fonts/Roboto-Bold.ttf: Roboto:style=Bold
/Library/Fonts/Roboto-BoldItalic.ttf: Roboto:style=Bold Italic
/Library/Fonts/Roboto-Medium.ttf: Roboto,Roboto Medium:style=Medium,Regular
/Library/Fonts/Roboto-BlackItalic.ttf: Roboto,Roboto Black:style=Black Italic,Italic
/Library/Fonts/Roboto-Italic.ttf: Roboto:style=Italic
/Library/Fonts/Roboto-Regular.ttf: Roboto:style=Regular
/Library/Fonts/Roboto-Light.ttf: Roboto,Roboto Light:style=Light,Regular

[mdboom]

Hmm... Can you put verbose.level : debug in ~/.matplotlib/matplotlibrc and run your script again, sending the output here? There is definitely something environmental about your system that's preventing it from seeing/using all the fonts.

[takis]

I tried, but for some reason setting it in matplotlibrc didn't give any output (and I did set the fileo to filename, file object, and sys.stdout/sys.stderr). I tried with:

mpl.rcParams['verbose.level'] = 'debug-annoying'
mpl.rcParams['verbose.fileo'] = '/tmp/mpl.txt'

And with:

from matplotlib import verbose
verbose.set_fileo('/Users/ic/mpl-log.txt')
verbose.set_level('debug-annoying')

And got just once a file with some output, but not very useful output. It was huge (due to debug-annoying I guess, but level 'debug' gave me no output at all), so I used filter the data a bit:

ic@MacBookPro ~ $ sort -u mpl-log.txt 
FigureCanvasAgg.draw
RendererAgg.__init__
RendererAgg.__init__ _RendererAgg done
RendererAgg.__init__ done
RendererAgg.__init__ width=1386.555, height=301.5
RendererAgg.__init__ width=1417.93, height=298.5
RendererAgg.__init__ width=72.0, height=72.0
RendererAgg._get_agg_font
RendererAgg.draw_text
RendererAgg.points_to_pixels
backend_agg.new_figure_manager
ic@MacBookPro ~ $

So, in the end, I gave up on set_level and fileo, and tried the Linux strace equivalent:

sudo dtruss -f -t open python ic-20151017-matplotlib_font_weight.py 2> mpl-log.txt
grep ttf mpl-log.txt |sort -u
20907/0xc3cdc:  open("/Library/Fonts/Courier New.ttf\0", 0x0, 0x1B6)         = 28 0
20907/0xc3cdc:  open("/Library/Fonts/Roboto-Black.ttf\0", 0x0, 0x1B6)        = 16 0
20907/0xc3cdc:  open("/Library/Fonts/Roboto-Bold.ttf\0", 0x0, 0x1B6)         = 16 0
20907/0xc3cdc:  open("/Library/Fonts/Roboto-Medium.ttf\0", 0x0, 0x1B6)       = 16 0
20907/0xc3cdc:  open("/Library/Fonts/Verdana.ttf\0", 0x0, 0x1B6)         = 15 0

But these are the fonts available:

ic@MacBookPro ~ $ ls /Library/Fonts/Robo*
/Library/Fonts/Roboto-Black.ttf
/Library/Fonts/Roboto-BoldItalic.ttf
/Library/Fonts/Roboto-LightItalic.ttf
/Library/Fonts/Roboto-Regular.ttf
/Library/Fonts/Roboto-BlackItalic.ttf
/Library/Fonts/Roboto-Italic.ttf
/Library/Fonts/Roboto-Medium.ttf
/Library/Fonts/Roboto-Thin.ttf
/Library/Fonts/Roboto-Bold.ttf  
/Library/Fonts/Roboto-Light.ttf 
/Library/Fonts/Roboto-MediumItalic.ttf
/Library/Fonts/Roboto-ThinItalic.ttf

So, I noticed that struss also showed these directories to be accessed, and I figured it might be a stale cache issue.

/usr/local/etc/fonts/fonts.conf
/usr/local/etc/fonts/conf.d/10-scale-bitmap-fonts.conf
...
/usr/local/etc/fonts/conf.d/50-user.conf
/usr/local/etc/fonts/conf.d/51-local.conf
/System/Library/Fonts
...
/usr/local/var/cache/fontconfig/b0a...
/Users/ic/.cache/fontconfig/b0a...
/Library/Fonts

And indeed the dates were really old:

ic@MacBookPro ~ $ ls -l ~/.fontconfig/
total 2728
-rw-r--r--  1 ic  staff  266128 Nov  5  2014 0f0db7876307790c19e1f91eb9095080-x86.cache-3
-rw-r--r--  1 ic  staff  265432 Nov  5  2014 1487dd4aecf3164c4a11193169052443-x86.cache-3
-rw-r--r--  1 ic  staff      80 Nov  5  2014 49aa604a5ac92994756d3008e408245c-x86.cache-3
-rw-r--r--  1 ic  staff      72 Jun 26  2014 530698deb7878507edc87921bdb300f1-x86.cache-3
-rw-r--r--  1 ic  staff     144 Nov 12  2014 558352270fb122ca08359d23b5a778d4-x86.cache-3
-rw-r--r--  1 ic  staff      96 Nov  5  2014 5dc9fcf026e07a49c5f91c19054b9930-x86.cache-3
-rw-r--r--  1 ic  staff   64616 Nov  5  2014 79652363633577d7d713baab7f54ad8c-x86.cache-3
-rw-r--r--  1 ic  staff  703568 Nov  5  2014 84c0f976e30e948e99073af70f4ae876-x86.cache-3
-rw-r--r--  1 ic  staff     200 Jan 23  2014 CACHEDIR.TAG
-rw-r--r--  1 ic  staff   19560 Nov  5  2014 a1a78d9c18cd095d3829c724810e6ffb-x86.cache-3
-rw-r--r--  1 ic  staff   18760 Nov  5  2014 ba022efc551c75e21c690774bbcf5304-x86.cache-3
-rw-r--r--  1 ic  staff   18576 Nov  5  2014 bc06c1eea3e636f72101cafc3fb39508-x86.cache-3
-rw-r--r--  1 ic  staff     456 Jun 26  2014 c5f5d66d15c24edc3e863c27139db87e-x86.cache-3
-rw-r--r--  1 ic  staff      80 Nov  5  2014 f0f76015ef54ea3f0895465d59f141ef-x86.cache-3
-rw-r--r--  1 ic  staff      88 Nov  5  2014 f22309b238134d3cca63435f528976cd-x86.cache-3

So, I tried clearing the cache with font-config:

fc-cache -f

That didn't change anything.

fc-cache -r  

This didn't change anything either (the dates in the directories shown above remained the same).

So, then "rm" did get rid of the cached files, after which I used fc-cache -f again to regenerate the caches.

ic@MacBookPro ~ $ rm /Users/ic/.cache/fontconfig/*
ic@MacBookPro ~ $ sudo rm /usr/local/var/cache/fontconfig/*
ic@MacBookPro ~ $ fc-cache -f
ic@MacBookPro ~ $ ls -l /Users/ic/.cache/fontconfig/
total 3208
-rw-r--r--  1 ic  staff     1600 Oct 17 22:31 530698deb7878507edc87921bdb300f1-le64.cache-4
-rw-r--r--  1 ic  staff  1228856 Oct 17 22:31 84c0f976e30e948e99073af70f4ae876-le64.cache-4
-rw-r--r--  1 ic  staff   403704 Oct 17 22:30 b0a71e6bf6a8a1a908413a823d76e21f-le64.cache-4
sudo dtruss -f -t open python ic-20151017-matplotlib_font_weight.py 2> mpl-log3.txt
ic@MacBookPro ~ $ grep ttf mpl-log3.txt |sort -u
25403/0xc6dfb:  open("/Library/Fonts/Courier New.ttf\0", 0x0, 0x1B6)         = 28 0
25403/0xc6dfb:  open("/Library/Fonts/Roboto-Black.ttf\0", 0x0, 0x1B6)        = 16 0
25403/0xc6dfb:  open("/Library/Fonts/Roboto-Bold.ttf\0", 0x0, 0x1B6)         = 16 0
25403/0xc6dfb:  open("/Library/Fonts/Roboto-Bold.ttf\0", 0x0, 0x1B6)         = 19 0
25403/0xc6dfb:  open("/Library/Fonts/Roboto-Medium.ttf\0", 0x0, 0x1B6)       = 16 0
25403/0xc6dfb:  open("/Library/Fonts/Roboto-Medium.ttf\0", 0x0, 0x1B6)       = 17 0
25403/0xc6dfb:  open("/Library/Fonts/Verdana.ttf\0", 0x0, 0x1B6)         = 15 0
25403/0xc6dfb:  open("/Library/Fonts/Verdana.ttf\0", 0x0, 0x1B6)         = 16 0

Unfortunately, the fonts still look the same. And grepping to the new struss output, still shows only the same files being accessed, and so the roboto-thin and roboto-light still being ignored/not found.

I also had a look at the configuration files of font-config

ic@MacBookPro ~ $ grep -ir roboto /usr/local/etc/fonts/conf.d/
ic@MacBookPro ~ $

But, I isn't mentioned in these files. I am not sure if this might be the problem, so I am looking at this now.

[mdboom]

After you cleared the fontconfig cache, did you also clear matplotlib's cache again?

I'd still like to see the long debug-annoying output, even though it is long. It does have lines in there about how the font lookup is performed (which strace/dtrace obviously won't give you).