Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Font cache thread safety #5226

Closed
timj opened this issue Oct 12, 2015 · 2 comments
Closed

Font cache thread safety #5226

timj opened this issue Oct 12, 2015 · 2 comments
Assignees

Comments

@timj
Copy link

timj commented Oct 12, 2015

The tests for the LSST software stack are run in parallel. We were getting failures roughly 10% of the time because of font cache directory creation issues. Looking at the code in font_manager.py we realised that setting $TRAVIS would fix our problem, which it did. This feels wrong though. It seems to me that this TRAVIS-specific bandaid is masking a more general problem with the cache. At the very least the environment variable being used to work around the problem should not be TRAVIS as the problem occurs any time two processes are trying to run matplotlib at the same time. Trying to explain why our test harness is forcing export TRAVIS=1 is challenging 😄

This is the trace without TRAVIS being set:

   import matplotlib.colorbar
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/colorbar.py", line 34, in <module>
    import matplotlib.collections as collections
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/collections.py", line 27, in <module>
    import matplotlib.backend_bases as backend_bases
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 56, in <module>
    import matplotlib.textpath as textpath
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/textpath.py", line 19, in <module>
    import matplotlib.font_manager as font_manager
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1415, in <module>
    _rebuild()
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1402, in _rebuild
    pickle_dump(fontManager, _fmcache)
  File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 952, in pickle_dump
    with open(filename, 'wb') as fh:
IOError: [Errno 2] No such file or directory: u'/tmp/matplotlib-vagrant/fontList.cache'
@mdboom mdboom self-assigned this Oct 12, 2015
@mdboom
Copy link
Member

mdboom commented Oct 12, 2015

Another workaround this is to persist the font cache directory (since it shouldn't be changing between test runs anyway). But really what we need to do is create a lockfile while the cache is being created.

@timj
Copy link
Author

timj commented Oct 16, 2015

Yes. I'm really saying that the TRAVIS=1 fix should be regarded as a temporary hack and that a lock file is required. It's relatively easy to trigger this problem in a multi-process environment and treating it as a one-off Travis problem is not the right answer (and, of course, hides the problem from your CI).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants