assert mods.pop(0) == 'tests' errors for multiprocess tests on OSX

[matthew-brett]

I'm testing OSX builds using Matt Terry's build grid. All the matplotlib builds are giving multiple errors like this:

======================================================================
FAIL: Failure: AssertionError ()
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 284, in generate
    for test in g():
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 152, in test
    baseline_dir, result_dir = _image_directories(self._func)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 303, in _image_directories
    assert mods.pop(0) == 'tests'
AssertionError

See: https://s3.amazonaws.com/archive.travis-ci.org/jobs/30794968/log.txt

It's not clear where the failures are coming from, but this is an excerpt from the stdout of the tests.

matplotlib.tests.test_triangulation.test_tri_smooth_contouring.test ... ok
Failure: AssertionError () ... FAIL
matplotlib.tests.test_triangulation.test_triinterpcubic_C1_continuity ... ok
matplotlib.tests.test_triangulation.test_tri_smooth_gradient.test ... ok
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
matplotlib.tests.test_transforms.test_pre_transform_plotting.test ... ok

I can replicate the failures on my laptop in a clean virtual machine, but only when running the tests via the multiprocessing plugin:

python ../matplotlib/tests.py -sv --processes=8 --process-timeout=300

Running in single process mode gives no failures. Does anyone have any idea how I could investigate further?

[WeatherGod]

I ran into this recently, too, but couldn't reproduce it after doing a
clean build.
On Jul 26, 2014 10:31 PM, "Matthew Brett" notifications@github.com wrote:

I'm testing OSX builds using Matt Terry's build grid. All the matplotlib
builds are giving multiple errors like this:

FAIL: Failure: AssertionError ()

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/loader.py", line 284, in generate
for test in g():
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 152, in test
baseline_dir, result_dir = _image_directories(self._func)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 303, in _image_directories
assert mods.pop(0) == 'tests'
AssertionError

See: https://s3.amazonaws.com/archive.travis-ci.org/jobs/30794968/log.txt

It's not clear where the failures are coming from, but this is an excerpt
from the stdout of the tests.

matplotlib.tests.test_triangulation.test_tri_smooth_contouring.test ... ok
Failure: AssertionError () ... FAIL
matplotlib.tests.test_triangulation.test_triinterpcubic_C1_continuity ... ok
matplotlib.tests.test_triangulation.test_tri_smooth_gradient.test ... ok
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
Failure: AssertionError () ... FAIL
matplotlib.tests.test_transforms.test_pre_transform_plotting.test ... ok

I can replicate the failures on my laptop in a clean virtual machine, but
only when running the tests via the multiprocessing plugin:

python ../matplotlib/tests.py -sv --processes=8 --process-timeout=300

Running in single process mode gives no failures. Does anyone have any
idea how I could investigate further?

—
Reply to this email directly or view it on GitHub
#3314.

[matthew-brett]

The travis builds are clean, though. I suppose it could be something to do with timings, making it somewhat random. But it happens for all the travis builds.

[mdboom]

This is puzzling -- I can't reproduce locally on my Mac box. It might be helpful if we knew what mods.pop(0) was on that line. Can we add it to the output somehow?

[jenshnielsen]

This happens on Travis if I replace python setup.py install with pip install . as part of trying to get #4678 to work

[jenshnielsen]

https://travis-ci.org/jenshnielsen/matplotlib/builds/88694941 fails on Travis due to this with the only change being the install via pip build from https://github.com/jenshnielsen/matplotlib/tree/pipinstalltravis

[jenshnielsen]

Some more debugging shows that this happens for tests within mpl_toolkits

For regular Matplotlib tests mods is something like ['tests', 'test_cycles'] where
it returns ['test_mplot3d'] or ['test_axes_grid1'] and so on within mpl_toolkits. Not at all sure why this is different with multiprocessing or why it's different with a pip install

Edit:

This is after popping the first element. mods is normally something like ['matplotlib', 'tests', 'test_cycles'] whereas in the mpl_toolkits test mods is ['tests', 'test_mplot3d'] ...

[jenshnielsen]

The same happens without multiprocessing if you do

python tests.py mpl_toolkits

It looks like yet another namespace oddity

[mdboom]

@jenshnielsen: Thanks for getting to the bottom of this. I'm now able to reproduce...