Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest 4.6.0 breaks matplotlib test gathering with `TypeError: cannot unpack non-iterable NoneType object` #5358

Closed
jkseppan opened this issue Jun 2, 2019 · 7 comments

Comments

Projects
None yet
6 participants
@jkseppan
Copy link

commented Jun 2, 2019

This build: https://travis-ci.org/matplotlib/matplotlib/jobs/540086010 passed fine, with pytest 4.5.0. The next build from the same code, initiated via cron and using pytest 4.6.0: https://travis-ci.org/matplotlib/matplotlib/jobs/540212001 failed with

Calling pytest with the following arguments: -raR --maxfail=50 --timeout=300 --durations=25 --cov-report= --cov=lib -n 2 --log-level=DEBUG
============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-4.6.0, py-1.7.0, pluggy-0.12.0
rootdir: /home/travis/build/matplotlib/matplotlib, inifile: pytest.ini, testpaths: lib
plugins: forked-1.0.2, cov-2.7.1, rerunfailures-7.0, xdist-1.28.0, timeout-1.3.3, faulthandler-1.6.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I
gw0 [5784] / gw1 [5784]
...
_______ ERROR collecting lib/matplotlib/tests/test_constrainedlayout.py ________
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:963: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:902: in _find_spec
    ???
<frozen importlib._bootstrap>:876: in _find_spec_legacy
    ???
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:169: in find_module
    source_stat, co = _rewrite_test(self.config, fn_pypath)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:404: in _rewrite_test
    rewrite_asserts(tree, fn, config)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:453: in rewrite_asserts
    AssertionRewriter(module_path, config).run(mod)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:701: in run
    new.extend(self.visit(child))
/opt/python/3.7.1/lib/python3.7/ast.py:262: in visit
    return visitor(node)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:828: in visit_Assert
    top_condition, explanation = self.visit(assert_.test)
E   TypeError: cannot unpack non-iterable NoneType object
_______ ERROR collecting lib/matplotlib/tests/test_constrainedlayout.py ________
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
<frozen importlib._bootstrap>:983: in _find_and_load
    ???
<frozen importlib._bootstrap>:963: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:902: in _find_spec
    ???
<frozen importlib._bootstrap>:876: in _find_spec_legacy
    ???
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:169: in find_module
    source_stat, co = _rewrite_test(self.config, fn_pypath)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:404: in _rewrite_test
    rewrite_asserts(tree, fn, config)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:453: in rewrite_asserts
    AssertionRewriter(module_path, config).run(mod)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:701: in run
    new.extend(self.visit(child))
/opt/python/3.7.1/lib/python3.7/ast.py:262: in visit
    return visitor(node)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/_pytest/assertion/rewrite.py:828: in visit_Assert
    top_condition, explanation = self.visit(assert_.test)
E   TypeError: cannot unpack non-iterable NoneType object
____________ ERROR collecting lib/matplotlib/tests/test_mathtext.py ____________
In test_mathtext_rendering: 1 parameter sets specified, with different number of ids: 83

The operating system is Ubuntu 16.04.6 LTS (xenial).

I don't know how to unpack the line numbers for the breaking tests from the above, but test_constrainedlayout.py has the following asserts:

    assert all(ax.get_position().extents == ax2.get_position().extents)
    assert all(ax.get_position().extents == ax2.get_position().extents)
    np.testing.assert_allclose(extents0, extents1)
    np.testing.assert_allclose(extents0, extents1)

In addition, there are multiple tests using matplotlib's @image_comparison decorator, but there are so many of those all over the codebase that I think then we'd see much more failures.

  • a detailed description of the bug or suggestion
  • output of pip list from the virtual environment you are using
  • pytest and operating system versions
  • minimal example if possible

jkseppan added a commit to jkseppan/matplotlib that referenced this issue Jun 2, 2019

Don't install pytest 4.6.0 on Travis
It seems to be causing test failures, see pytest-dev/pytest#5358
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

i believe this is a flaw in the "any unrolling" see #5103

@slarse

This comment has been minimized.

Copy link

commented Jun 2, 2019

@RonnyPfannschmidt I think that's correct, I battled this problem yesterday and managed to reproduce it with the test case below (or any test case including all)

def test_all():
    assert all([])

Works fine in 4.5.0, breaks in 4.6.0

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

@slarse thanks for providing a minimal reproducer

the triggering code in matplotlib is https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/tests/test_constrainedlayout.py#L300

rm-hull added a commit to rm-hull/luma.core that referenced this issue Jun 2, 2019

Juanlu001 added a commit to Juanlu001/poliastro that referenced this issue Jun 2, 2019

Juanlu001 added a commit to Juanlu001/poliastro that referenced this issue Jun 2, 2019

@unintended unintended referenced this issue Jun 2, 2019

Merged

update exception matcher and fix tests #134

2 of 2 tasks complete
@nicoddemus

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 2, 2019

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 2, 2019

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 2, 2019

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 2, 2019

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 2, 2019

@asottile

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

We'll be making a 4.6.1 release today containing this fix -- thanks again for the report!

@asottile

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

This has been released as part of 4.6.1 -- thanks again for the report!

jkseppan added a commit to jkseppan/matplotlib that referenced this issue Jun 2, 2019

@jkseppan jkseppan referenced this issue Jun 2, 2019

Merged

MAINT: Upgrade pytest again #14420

0 of 6 tasks complete
@Tadaboody

This comment has been minimized.

Copy link
Contributor

commented Jun 4, 2019

So sorry about this 🙏 looks like my PR wasn't fully formed

jvesely added a commit to jvesely/PsyNeuLink that referenced this issue Jun 6, 2019

requirements: Enable pytest >= 4.6
The issue with test enumeration was fixed in 4.6.1:
pytest-dev/pytest#5358

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

jvesely added a commit to PrincetonUniversity/PsyNeuLink that referenced this issue Jun 6, 2019

requirements: Enable pytest >= 4.6
The issue with test enumeration was fixed in 4.6.1:
pytest-dev/pytest#5358

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.