Max recursion errors for mathtext on Python 3.6 / OSX #7799

Open
matthew-brett opened this Issue Jan 11, 2017 · 8 comments

Projects

None yet

3 participants

@matthew-brett
Contributor

See: MacPython/matplotlib-wheels#3

Errors occur testing 1.5.3 and current v2.x branch on Python 3.6, OSX.

======================================================================
ERROR: matplotlib.tests.test_mathtext.test_mathtext_cm_53.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/testing/decorators.py", line 207, in do_test
    figure.savefig(actual_fname, **self._savefig_kwarg)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/figure.py", line 1563, in savefig

[snip]

 File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/pyparsing.py", line 1510, in _parseCache
    value = cache.get(lookup)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/pyparsing.py", line 1459, in get
    return cache.get(key, not_in_cache)
RecursionError: maximum recursion depth exceeded

https://s3.amazonaws.com/archive.travis-ci.org/jobs/187592465/log.txt

Further investigation reveals:

  • error does not occur when running the individual mathtext tests, e.g. with python tests.py matplotlib.tests.test_mathtext, but only when running these tests as part of a full test suite python tests.py;
  • when dropping into the debugger on the test failure - python tests.py --pdb - this sequence of commands also cause the recursion error:
from matplotlib.font_manager import FontProperties
from matplotlib import mathtext as mt

my_p = mt.Parser()
backend = mt.MathtextBackendPath()
font_output = mt.BakomaFonts(FontProperties(), backend)
my_str = '$\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+x}}}}}}}$'
my_p._state_stack = [my_p.State(font_output, 'default', 'rm', 100, 72)]
my_p._em_width_cache = {}
result = my_p._expression.parseString(my_str)
print(result)

The same sequence of commands run on its own in Python 3.6, passes.

This string also generates the recursion errors, when dropped into the debugger:

r'$\cos{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1+\cos{1+x}}}}}}}$'

but not:

r'$\cos{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \cos{1+\cos{1+x}}}}}}}$'
r'$\cos{1 + \cos{1 + \cos{1 + \cos{1 + \cos{1 + \cos{1+\cos{1+x}}}}}}}$'
r'$\sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1+\sqrt{1+x}}}}}}$'

It looks like there is some horrible global state in the MathText module or pyparsing that is being triggered only under certain circumstances.

@mdboom - can you reproduce? Any insights into the problem?

@tomkooij tomkooij referenced this issue in MacPython/astropy-wheels Jan 11, 2017
Merged

Build astropy v1.3 #4

@matthew-brett
Contributor

@mdboom - do you have any time to look into this one? Or any suggestions as to how to investigate further?

@tacaswell
Member

Can the recursion limit on travis be pushed up? The test runner adds a whole bunch of layers, I wonder if we were always sitting on the limit?

@matthew-brett
Contributor

Had a shot at that - but now tests all failing - https://travis-ci.org/MacPython/matplotlib-wheels/jobs/192055053#L1757 - something to do with the results images change?

@jkseppan
Member

That Travis log looks like it's installing matplotlib from a wheel, and as of recently the wheels don't include test images.

@matthew-brett matthew-brett added a commit to matthew-brett/matplotlib that referenced this issue Jan 16, 2017
@matthew-brett matthew-brett MAINT: add ability to specify recursionlimit
We're getting a test failure on OSX and Python 3.6 where parsing reaches the
recursion limit: matplotlib#7799

Upping the recursionlimit resolves the error.

Add command line argument to tests to up the recursion limit for the
test run.
c013ed7
@matthew-brett matthew-brett added a commit to matthew-brett/matplotlib that referenced this issue Jan 16, 2017
@matthew-brett matthew-brett MAINT: add ability to specify recursionlimit
We're getting a test failure on OSX and Python 3.6 where parsing reaches the
recursion limit: matplotlib#7799

Upping the recursionlimit resolves the error.

Add command line argument to tests to up the recursion limit for the
test run.
3434eb5
@matthew-brett
Contributor

OK - I have solved the test images problem - see discussion over at #7757 (comment) .

Upping the recursion limit does make the tests pass on Python 3.6 - see https://travis-ci.org/MacPython/matplotlib-wheels/jobs/192072933

I've submitted a couple of PRs to add the ability to specify the recursion limit to the test function, see #7843, #7849.

Once those are merged, I think wheel building will be OK for master and for the 2.x branch.

@tacaswell
Member

I can now reproduce this on linux as will, but only if I run the test suite not parallel.

@tacaswell
Member

Also, I wonder if that very deep recursion a performance bottle neck that we should be looking into.

@matthew-brett
Contributor

It's certainly crazy slow to do the parsing - at one point I ran 1000 parses of that multiply nested sqrt string, and it took something like 30 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment