New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeated warning about fc-list #5836

Closed
astrofrog opened this Issue Jan 12, 2016 · 66 comments

Comments

Projects
None yet
@astrofrog
Contributor

astrofrog commented Jan 12, 2016

Since Matplotlib 1.5.1, I see a warning (twice) each time I instantiate a FontManager():

In [2]: from matplotlib.font_manager import FontManager

In [3]: FontManager()
/Users/tom/miniconda3/envs/production/lib/python3.4/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
/Users/tom/miniconda3/envs/production/lib/python3.4/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Out[3]: <matplotlib.font_manager.FontManager at 0x1057b6630>

In [4]: FontManager()
/Users/tom/miniconda3/envs/production/lib/python3.4/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
/Users/tom/miniconda3/envs/production/lib/python3.4/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Out[4]: <matplotlib.font_manager.FontManager at 0x1064d4ba8>

This did not happen with 1.5.0. There's two issues here - the warning is repeated twice at each call, which indicates that the command is being run twice (inefficient), and the warning is being shown at all - why does fc-list need to be run every time FontManager is run? Even if there is a good reason for it to run, I'm not sure if a warning is necessary?

@jenshnielsen

This comment has been minimized.

Member

jenshnielsen commented Jan 12, 2016

The only change in 1.5.1 is the addition of the warning in #5640 so there should be no performance difference. It's called twice to find both ttf and afm fonts. Not sure there is a good way to do that with just one call.

Internally matplotlib pickles the fontmanager and only ever runs this code if there is no font cache installed so this should not be seen on other than the first import of pyplot. The warning was added because that call may be really slow on a system with many fonts installed see #5640

In the long term there are plans to significantly rewrite the font handling see notes in #5414

@larrybradley

This comment has been minimized.

Contributor

larrybradley commented Jan 12, 2016

This warning is triggering travis-ci failures for Sphinx builds for packages that use matplotlib in documentation (e.g. https://travis-ci.org/astropy/photutils/jobs/101686421#L521).

@mdboom

This comment has been minimized.

Member

mdboom commented Jan 12, 2016

This warning is triggering travis-ci failures for Sphinx builds for packages that use matplotlib in documentation (e.g. https://travis-ci.org/astropy/photutils/jobs/101686421#L521).

Hmmm... That's an interesting outcome of all this. We could suppress the warning if sphinx is in sys.modules. (Yuck). I am starting to wonder if this warning was worth the trouble. I'd love to get #5414 finished at some point and address this once and for all (by eliminating matplotlib's own font cache altogether).

@tacaswell

This comment has been minimized.

Member

tacaswell commented Jan 12, 2016

I was always ambivalent to it and would be fine with removing it again. The strongest argument in favor of it is users having short attention spans which I do not really find persuasive.

warnings.warn seems like the wrong mechanism for providing user feed back here, there is nothing that can be done about this and there is no change to the user code that can suppress this warning. We probably should be using logging of some sort (but adding std logging is a huge task which is even more thankless than most).

@tacaswell tacaswell added this to the Critical bug fix release (1.5.2) milestone Jan 12, 2016

@jenshnielsen

This comment has been minimized.

Member

jenshnielsen commented Jan 13, 2016

What about just changing it to a print? The main point is to provide interactive feedback to users when importing pyplot that would be lost if we used logging.

@fmaussion

This comment has been minimized.

Contributor

fmaussion commented Jan 15, 2016

On our travis tests we get the message at each build. On which kind of architecture will the cache take so long that it requires a message?

@WeatherGod

This comment has been minimized.

Member

WeatherGod commented Jan 25, 2016

The problem with "print" is that that can't be filtered at all. A warning
can at least be filtered if an application developer wants it to be
filtered.

As for what kinds of systems this can take so long, basically any system
with Adobe Illustrator installed. It can take several minutes to create the
initial cache.

On Fri, Jan 15, 2016 at 6:00 AM, Fabien Maussion notifications@github.com
wrote:

On our travis tests we get the message at each build. On which kind of
architecture will the cache take so long that it requires a message?


Reply to this email directly or view it on GitHub
#5836 (comment)
.

@astrofrog

This comment has been minimized.

Contributor

astrofrog commented Jan 25, 2016

A somewhat crazy idea, but what about emitting the warning only if the process takes more than a certain time? For example:

import time
import warnings
import subprocess
import multiprocessing as mp

def delayed_warning():
    time.sleep(5)
    warnings.warn('Matplotlib is building the font cache using fc-list. '
                  'This may take a moment.')

proc = mp.Process(target=delayed_warning)
proc.start()
subprocess.call('fc-list')
proc.terminate()
@mdboom

This comment has been minimized.

Member

mdboom commented Jan 26, 2016

Not a crazy idea, if it's robust enough.

Note that all of this will be wasted effort following #5414, where we just call the font-config API directly and don't have to copy all of its contents into our own cache.

@dereknheiley

This comment has been minimized.

dereknheiley commented Feb 4, 2016

not the prettiest work around, but i silenced the warnings for the time being (although the small startup delay is still present) with the following code, maybe a better filter could be used to allow other warnings

import warnings;
with warnings.catch_warnings():
    warnings.simplefilter("ignore"); 
    import matplotlib.pyplot as plt
@tacaswell

This comment has been minimized.

Member

tacaswell commented Feb 4, 2016

@derekneil try removing everything the mpl cache, you should only have seen this once on the first import....

@dereknheiley

This comment has been minimized.

dereknheiley commented Feb 8, 2016

@tacaswell i did try removing everything in the cache, but it happens every time on my system for some reason

@catubc

This comment has been minimized.

catubc commented Feb 14, 2016

I ran the python code w. sudo and it cured it...my guess was that there wasn't permission to write that table... good luck!

@tacaswell

This comment has been minimized.

Member

tacaswell commented Feb 15, 2016

@catubc You should sort out what file you do not have permission to write as a user and fix the permissions on that file. Running python with sudo is almost always a bad idea.

@fmder

This comment has been minimized.

fmder commented Feb 23, 2016

I got this issue after upgrading matplotlib and clearing the cache solved my issue!

@mpenkov

This comment has been minimized.

mpenkov commented Feb 24, 2016

@derekneil @fmder +1 "rm -rf ~/.matplotlib" solved the problem for me too

@sebhahn

This comment has been minimized.

sebhahn commented Feb 25, 2016

For me it was rm -rf ~/.cache/matplotlib

@eakbas

This comment has been minimized.

eakbas commented Mar 9, 2016

rm -rf ~/.cache/matplotlib solved the problem for me.

@matthewhoffman

This comment has been minimized.

matthewhoffman commented Mar 16, 2016

rm -rf ~/.matplotlib/fontList.cache worked for me.

@grantjenks

This comment has been minimized.

grantjenks commented Mar 23, 2016

rm -rf ~/.matplotlib worked for me.

@anntzer

This comment has been minimized.

Contributor

anntzer commented Aug 30, 2016

I think @astrofrog is seeing #4993 (comment), which is a somewhat different issue.

fangohr added a commit to joommf/discretisedfield that referenced this issue Sep 25, 2016

josephcslater added a commit to vibrationtoolbox/vibration_toolbox that referenced this issue Oct 30, 2016

@rmjarvis

This comment has been minimized.

Contributor

rmjarvis commented Nov 28, 2016

I do still believe that this should probably not be a warning but probably should be more at the logging.INFO level (since it's not something actionable on the part of the user). But if there's otherwise agreement this should be a warning, feel free to close this!

I strongly agree with @astrofrog about this. This is broken behavior, as far as I'm concerned. Warnings should be reserved for things that are either likely to be user error or where the code's behavior is probably not what the user is expecting. And in the latter case, there should be an easy way to suppress the warning if it really is what the user meant.

e.g. The warning emitted by matplotlib.use('Agg') is a good exemplar. It warns if the use statement happens too late to be effective, tells you how to make it effective, and also gives an option (warn=False) to suppress the warning.

The warning about fc-list, OTOH, is not actionable. And in many cases (e.g. imports by dependencies) it is not suppressible. And with the rising popularity of Travis CI (and others), we get a clean system each time, so people are seeing it a lot. It adds quite a lot of noise to our test output, especially with the warning being emitted multiple times, not just once as it should.

For my case, none of the above suggestions for Travis have worked. My solution has been to use matplotlib==1.5.0, which I now consider the latest working version of matplotlib. Please reopen this issue and fix this bug.

FWIW, I thought the suggestion by @astrofrog to only emit the warning if building the cache takes more than 5 seconds or so was a good one. This would solve most of the cases where this behavior is seen as a bug and preserve the cases where people find it useful. And figuring out why it is emitting the warning multiple times and fixing that would be nice as well.

@efiring efiring modified the milestones: 2.0 (style change major release), v1.5.2 Nov 28, 2016

@efiring efiring reopened this Nov 28, 2016

@efiring

This comment has been minimized.

Member

efiring commented Nov 28, 2016

I agree; we need to figure out a better way to handle this.

Also, as @astrofrog noted originally, the warning is emitted twice, which in itself indicates there is a real bug in the code.

@efiring

This comment has been minimized.

Member

efiring commented Nov 28, 2016

Not a bug, but an inefficiency:

  • The warning is in get_fontconfig_fonts(), right before it runs fc-list.
  • get_fontconfig_fonts() is called once by getSystemFonts().
  • But getSystemFonts() is called twice by FontManager.__init__(), first looking for ttf, then for afm.
@tacaswell

This comment has been minimized.

Member

tacaswell commented Nov 29, 2016

Being emitted twice is correct behavior as we are looking for two types of fonts, maybe that should be included in the warning.

For my case, none of the above suggestions for Travis have worked. My solution has been to use matplotlib==1.5.0, which I now consider the latest working version of matplotlib. Please reopen this issue and fix this bug.

@rmjarvis I find it difficult to take this sort of over-the-top rhetoric seriously.

Almost all of the CI services let you keep a persistent cache between runs, push ~/.matplotlib in that cache and you will only see the warning occasionally. Failing that, run a step in your CI that just imports matplotlib with the warning suppressed before you run your main tests.

mpl sits at a weird edge between a library and a user interface. Providing some sort of feedback to users who hit surprisingly long import times (my understanding is that if you have all the adobe fonts installed this can take minutes) is important.

The problem with moving to a logging based solution is that it brings with it a tremendous amount of overhead (which mpl is currently using none of) and if it is not very noisy by default it will miss the target audience for this warning.

If there is a non-thread based way to get a 5s delay I am 👍 on that, but bringing threads in this seems like complexity overkill.

I think this should be closed again.

@efiring

This comment has been minimized.

Member

efiring commented Nov 29, 2016

@astrofrog

This comment has been minimized.

Contributor

astrofrog commented Nov 29, 2016

mtmccrea pushed a commit to mtmccrea/migrate-atk-sc-src that referenced this issue Nov 30, 2016

ATK: Update Intro-to-the-ATK & imaging figures
More verbose Intro-to-the-ATK. Describe in more detail how to read
transform figures with description of sound field quality measure, rE.

Transform plots now illustrate rE rather than rV, which is more closely
tied to perception.

NOTE: Fonts and scaling have changed as a result of an issue with
anaconda/matplotlib. Some relevant issues can be found here:

fonnesbeck/ScipySuperpack#39
matplotlib/matplotlib#5836
matplotlib/matplotlib#5640
matplotlib/matplotlib#6721
matplotlib/matplotlib#3590
@NelleV

This comment has been minimized.

Member

NelleV commented Dec 1, 2016

This is annoying, but not release critical. Bumping this to 2.0.1, as we almost have a patch ready for that one (so it'll probably go in for 2.0).

@NelleV NelleV modified the milestones: 2.0.1 (next bug fix release), 2.0 (style change major release) Dec 1, 2016

@anntzer

This comment has been minimized.

Contributor

anntzer commented Dec 12, 2016

Closed by #7596.

@anntzer anntzer closed this Dec 12, 2016

@QuLogic QuLogic modified the milestones: 2.0 (style change major release), 2.0.1 (next bug fix release) Dec 12, 2016

@adaifotis

This comment has been minimized.

adaifotis commented Jan 19, 2017

For those arriving after experiencing this problem with docker specifically, I added the below to my Dockerfile:

RUN python -c "import matplotlib.pyplot"

That way it will create the cache when you first build the image, but then should not warn anymore on subsequent runs.

@empty16

This comment has been minimized.

empty16 commented Nov 6, 2017

rm -rf ~/.cache/matplotlib
Remove the cache works for me, too. Thanks, you guys.

@jklymak

This comment has been minimized.

Contributor

jklymak commented Nov 6, 2017

It also seems this could be moved from warnings.warn to logging.info when #9313 gets merged...

grlee77 added a commit to grlee77/scikit-image that referenced this issue Nov 9, 2017

@grlee77 grlee77 referenced this issue Nov 9, 2017

Merged

ENH: add cycle spinning routine #2647

4 of 4 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment