Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest silently stops importing plugins on ImportError #1479

Closed
The-Compiler opened this issue Mar 24, 2016 · 13 comments
Closed

pytest silently stops importing plugins on ImportError #1479

The-Compiler opened this issue Mar 24, 2016 · 13 comments
Labels
status: critical grave problem or usability issue that affects lots of users type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog type: bug problem that needs to be addressed

Comments

@The-Compiler
Copy link
Member

Yesterday @craftyguy was trying to contribute something to qutebrowser, and we spent the time debugging an interesting pytest/pluggy issue instead 😆

When he tried to run pytest (tox -e py35-cov with this tox.ini), he got errors about missing arguments (added in my pytest.ini):

py35-cov runtests: commands[1] | /home/clayton/src/qutebrowser/.tox/py35-cov/bin/python -m py.test --cov --cov-report xml --cov-report=html --cov-report= tests/unit/utils/test_standarddir.py
usage: test.py [options] [file_or_dir] [file_or_dir] [...]                                                                                                                                                                         test.py: error: unrecognized arguments: --instafail --cov --cov-report --cov-report=html --cov-report= tests/unit/utils/test_standarddir.py
  inifile: /home/clayton/src/qutebrowser/pytest.ini                                                                                                                                                                                  rootdir: /home/clayton/src/qutebrowser

Looking at --version output, only a subset of the plugins was loaded:

This is pytest version 2.9.1, imported from /home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest.py
setuptools registered plugins:
  pytest-faulthandler-1.3.0 at /home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_faulthandler.py
  pytest-catchlog-1.2.2 at /home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_catchlog.py
  pytest-travis-fold-1.2.0 at /home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_travis_fold.py

However discovering them wasn't the issue:

$ ./.tox/py35-cov/bin/python -c 'import pkg_resources, pprint; pprint.pprint(list(pkg_resources.iter_entry_points("pytest11")))'
[EntryPoint.parse('pytest_faulthandler = pytest_faulthandler'),
 EntryPoint.parse('pytest_catchlog = pytest_catchlog'),
 EntryPoint.parse('travis-fold = pytest_travis_fold'),
 EntryPoint.parse('pytest-qt = pytestqt.plugin'),                                                                                                                                                                                   EntryPoint.parse('rerunfailures = pytest_rerunfailures'),
 EntryPoint.parse('pytest_mock = pytest_mock'),                                                                                                                                                                                     EntryPoint.parse('xvfb = pytest_xvfb'),                                                                                                                                                                                            EntryPoint.parse('html = pytest_html.plugin'),                                                                                                                                                                                     EntryPoint.parse('hypothesispytest = hypothesis.extra.pytestplugin'),                                                                                                                                                              EntryPoint.parse('pytest-bdd = pytest_bdd.plugin'),
 EntryPoint.parse('pytest_cov = pytest_cov.plugin'),
 EntryPoint.parse('repeat = pytest_repeat'),                                                                                                                                                                                        EntryPoint.parse('instafail = pytest_instafail')]

We then patched pytest's pluggy.py like so:

    def load_setuptools_entrypoints(self, entrypoint_name):
        """ Load modules from querying the specified setuptools entrypoint name.
        Return the number of loaded plugins. """
        from pkg_resources import iter_entry_points, DistributionNotFound
        print(list(iter_entry_points(entrypoint_name)))
        for ep in list(iter_entry_points(entrypoint_name)):
            print(ep.name)
            # is the plugin registered or blocked?
            if self.get_plugin(ep.name) or self.is_blocked(ep.name):
                continue
            try:
                plugin = ep.load()
            except DistributionNotFound:
                continue
            except:
                import traceback
                traceback.print_exc()
            self.register(plugin, name=ep.name)
            self._plugin_distinfo.append((plugin, ep.dist))
        return len(self._plugin_distinfo

(printing the discovered entrypoints, printing the entrypoint name inside the for-loop, and showing all exceptions)

This finally showed us what was going wrong: pytest-qt failed to import due to some C++ ABI mismatch, but some outer layer swallowed the exception and didn't report anything at all!

./.tox/py35-cov/bin/python -m py.test --version
[EntryPoint.parse('pytest_faulthandler = pytest_faulthandler'), EntryPoint.parse('pytest_catchlog = pytest_catchlog'), EntryPoint.parse('travis-fold = pytest_travis_fold'), EntryPoint.parse('pytest-qt = pytestqt.plugin'), EntryPoint.parse('rerunfailures = pytest_rerunfailures'), EntryPoint.parse('pytest_mock = pytest_mock'), EntryPoint.parse('xvfb = pytest_xvfb'), EntryPoint.parse('html = pytest_html.plugin'), EntryPoint.parse('hypothesispytest = hypothesis.extra.pytestplugin'), EntryPoint.parse('pytest-bdd = pytest_bdd.plugin'), EntryPoint.parse('pytest_cov = pytest_cov.plugin'), EntryPoint.parse('repeat = pytest_repeat'), EntryPoint.parse('instafail = pytest_instafail')]
pytest_faulthandler
pytest_catchlog
travis-fold
pytest-qt
Traceback (most recent call last):
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 503, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2202, in load
    return self.resolve()
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2208, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytestqt/plugin.py", line 5, in <module>
    from pytestqt.logging import QtLoggingPlugin, _QtMessageCapture, Record
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytestqt/logging.py", line 7, in <module>
    from pytestqt.qt_compat import qInstallMsgHandler, qInstallMessageHandler, \
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytestqt/qt_compat.py", line 80, in <module>
    QtTest = _import_module('QtTest')
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytestqt/qt_compat.py", line 55, in _import_module
    m = __import__(_root_module, globals(), locals(), [module_name], 0)
ImportError: /home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/PyQt5/QtTest.so: undefined symbol: _Z19qt_handleMouseEventP7QWindowRK7QPointFS3_6QFlagsIN2Qt11MouseButtonEES4_INS5_16KeyboardModifierEE
Traceback (most recent call last):
  File "/usr/lib64/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/py/test.py", line 4, in <module>
    sys.exit(pytest.main())
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 39, in main
    config = _prepareconfig(args, plugins)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 118, in _prepareconfig
    pluginmanager=pluginmanager, args=args)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 729, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 600, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
    wrap_controller.send(call_outcome)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/helpconfig.py", line 28, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
    self.result = func()
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 601, in execute
    res = hook_impl.function(*args)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 861, in pytest_cmdline_parse
    self.parse(args)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 966, in parse
    self._preparse(args, addopts=addopts)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 927, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 509, in load_setuptools_entrypoints
    self.register(plugin, name=ep.name)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/config.py", line 208, in register
    ret = super(PytestPluginManager, self).register(plugin, name)
  File "/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 350, in register
    (plugin_name, plugin, self._name2plugin))
ValueError: Plugin already registered: pytest-qt=<module 'pytest_travis_fold' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_travis_fold.py'>
{'pastebin': <module '_pytest.pastebin' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/pastebin.py'>, 'doctest': <module '_pytest.doctest' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/doctest.py'>, '140035651859008': <_pytest.config.PytestPluginManager object at 0x7f5c9748f240>, 'monkeypatch': <module '_pytest.monkeypatch' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/monkeypatch.py'>, 'junitxml': <module '_pytest.junitxml' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/junitxml.py'>, 'pdb': <module '_pytest.pdb' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/pdb.py'>, 'assertion': <module '_pytest.assertion' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/assertion/__init__.py'>, 'helpconfig': <module '_pytest.helpconfig' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/helpconfig.py'>, 'terminal': <module '_pytest.terminal' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/terminal.py'>, 'main': <module '_pytest.main' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/main.py'>, 'nose': <module '_pytest.nose' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/nose.py'>, 'mark': <module '_pytest.mark' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/mark.py'>, 'cacheprovider': <module '_pytest.cacheprovider' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/cacheprovider.py'>, 'travis-fold': <module 'pytest_travis_fold' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_travis_fold.py'>, 'python': <module '_pytest.python' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/python.py'>, 'pytest_catchlog': <module 'pytest_catchlog' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_catchlog.py'>, 'tmpdir': <module '_pytest.tmpdir' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/tmpdir.py'>, 'genscript': <module '_pytest.genscript' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/genscript.py'>, 'pytestconfig': <_pytest.config.Config object at 0x7f5c96bc7710>, 'skipping': <module '_pytest.skipping' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/skipping.py'>, 'runner': <module '_pytest.runner' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/runner.py'>, 'resultlog': <module '_pytest.resultlog' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/resultlog.py'>, 'recwarn': <module '_pytest.recwarn' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/recwarn.py'>, 'unittest': <module '_pytest.unittest' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/unittest.py'>, 'capture': <module '_pytest.capture' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/_pytest/capture.py'>, 'pytest_faulthandler': <module 'pytest_faulthandler' from '/home/clayton/src/qutebrowser/.tox/py35-cov/lib/python3.5/site-packages/pytest_faulthandler.py'>}

Also, I noticed this part which is very confusing: ValueError: Plugin already registered: pytest-qt=<module 'pytest_travis_fold' from ...> - but maybe this was just a consequence of ignoring the exception and continuing to run the loop.

@RonnyPfannschmidt
Copy link
Member

Your Workaround has a bug, it Registers Plugins even on exceptions, thus registering the last value under New name

@RonnyPfannschmidt
Copy link
Member

Moving the Register part into the else part of the previous try should fix it

@The-Compiler
Copy link
Member Author

That's what I meant with "but maybe this was just a consequence of ignoring the exception and continuing to run the loop" - I was debugging this via IRC with @craftyguy as I couldn't reproduce it on my local machine first, so it was cumbersome enough already 😉

That explains the second traceback, but not why the first was silently ignored.

@RonnyPfannschmidt RonnyPfannschmidt added type: bug problem that needs to be addressed type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog status: critical grave problem or usability issue that affects lots of users vendored-pluggy labels Mar 24, 2016
@nicoddemus
Copy link
Member

@The-Compiler did you manage to find a reproducible example?

@The-Compiler
Copy link
Member Author

So it seems to do what I expected with a local plugin loaded via pytest_plugins = ['plugin']:

Traceback (most recent call last):
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 283, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/florian/tmp/issue1479')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 314, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/florian/tmp/issue1479/conftest.py')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 385, in import_plugin
    __import__(importspec)
  File "/home/florian/tmp/issue1479/plugin.py", line 1, in <module>
    import blah
ImportError: No module named 'blah'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../.venv/bin/py.test", line 11, in <module>
    sys.exit(main())
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 39, in main
    config = _prepareconfig(args, plugins)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 118, in _prepareconfig
    pluginmanager=pluginmanager, args=args)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
    wrap_controller.send(call_outcome)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/helpconfig.py", line 28, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
    self.result = func()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
    res = hook_impl.function(*args)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 861, in pytest_cmdline_parse
    self.parse(args)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 966, in parse
    self._preparse(args, addopts=addopts)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 937, in _preparse
    args=args, parser=self._parser)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 253, in _wrapped_call
    return call_outcome.get_result()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
    self.result = func()
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
    res = hook_impl.function(*args)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 907, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(early_config.known_args_namespace)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 269, in _set_initial_conftests
    self._try_load_conftest(current)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 272, in _try_load_conftest
    self._getconftestmodules(anchor)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 297, in _getconftestmodules
    mod = self._importconftest(conftestpath)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 333, in _importconftest
    self.consider_conftest(mod)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 357, in consider_conftest
    self.consider_module(conftestmodule)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 363, in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", None))
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 370, in _import_plugin_specs
    self.import_plugin(import_spec)
  File "/home/florian/tmp/.venv/lib/python3.5/site-packages/_pytest/config.py", line 392, in import_plugin
    raise new_exc
ImportError: Error importing plugin "plugin": No module named 'blah'

However when doing the same thing in a setuptools entry point, I only get a pytest warning:

============================================= test session starts ==============================================
platform linux -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/florian/tmp/issue1479, inifile: 
collected 0 items 

============================================ pytest-warning summary ============================================
WI2 None could not load setuptools entry import: No module named 'blah'
====================================== 1 pytest-warnings in 0.01 seconds =======================================

This should really be a hard error, IMHO - it's really easy to miss, especially when you're trying to figure out why the hell your commandline arguments aren't accepted and pytest doesn't even run 😉

setup.py of the plugin I installed:

from setuptools import setup

setup(
    name="pytest-issue1479",
    packages = ['plugin'],

    # the following makes a plugin available to pytest
    entry_points = {
        'pytest11': [
            'issue1479 = plugin.plugin',
        ]
    },
)

plugin/plugin.py:

import blah

@alenickwork
Copy link

Hi!
What's the prognosys? My work stopped due to this issue :)
Any workaround?
Thnx in advance

@alenickwork
Copy link

BTW,
I met +1 strange issue:

C:\Python27\lib\site-packages\allure\pytest_plugin.py:7: in <module>
    from _pytest.junitxml import mangle_testnames
E   ImportError: cannot import name mangle_testnames

@The-Compiler
Copy link
Member Author

What's the prognosys? My work stopped due to this issue :)

If you're talking about commercial work, have you considered getting a support contract with Merlinux? 😉

Any workaround?

I don't think so, other than fixing the real underlying issue. The issue here is simply that there's no proper error displayed when a plugin fails to import something.

C:\Python27\lib\site-packages\allure\pytest_plugin.py:7: in <module>
    from _pytest.junitxml import mangle_testnames
E   ImportError: cannot import name mangle_testnames

Blame the allure package for importing pytest-internal stuff. _pytest.junitxml.mangle_testnames is not public pytest API and got renamed in 2853083.

Seems allure-framework/allure-python#92 is open already.

@The-Compiler
Copy link
Member Author

Proposed fix in #1564.

@utkjad
Copy link

utkjad commented Nov 27, 2017

seeing this issue again on 3.3.0
If I revert pytest back to 3.2.5 the test suite runs fine.

@The-Compiler
Copy link
Member Author

@utkjad I can't reproduce:

Traceback (most recent call last):
  File "./.venv/bin/pytest", line 11, in <module>
    sys.exit(main())
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/config.py", line 50, in main
    config = _prepareconfig(args, plugins)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/config.py", line 160, in _prepareconfig
    pluginmanager=pluginmanager, args=args)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 196, in _multicall
    gen.send(outcome)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/helpconfig.py", line 68, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/config.py", line 942, in pytest_cmdline_parse
    self.parse(args)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/config.py", line 1107, in parse
    self._preparse(args, addopts=addopts)
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/_pytest/config.py", line 1070, in _preparse
    self.pluginmanager.load_setuptools_entrypoints('pytest11')
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pluggy/__init__.py", line 397, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2291, in load
    return self.resolve()
  File "/home/florian/tmp/issue1479/.venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2297, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/florian/tmp/issue1479/plugin/plugin.py", line 1, in <module>
    import blah
ModuleNotFoundError: No module named 'blah'

with:

attrs==17.3.0
pluggy==0.6.0
py==1.5.2
pytest==3.3.0
pytest-issue1479==0.0.0
six==1.11.0

Do you have a reproducer? Can you show the full output you get?

@The-Compiler
Copy link
Member Author

Wait - you say "If I revert pytest back to 3.2.5 the test suite runs fine." but this issue is about pytest hiding a traceback. It sounds like you're seeing an entirely different issue.

@matthijskooijman
Copy link

I'm also seeing this problem again on a Windows system where a dependency is missing. However, it is not pytest that's eating the exception here, but pluggy (which eats DistributionNotFound exceptions). For reference, here is the traceback that's being eaten (by adding a traceback.print_exc() inside pluggy):

Traceback (most recent call last):
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pluggy\manager.py", line 254, in load_setuptools_entrypoints
    plugin = ep.load()
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 2320, in load
    self.require(*args, **kwargs)
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 2343, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "c:\users\brenda\.virtualenvs\django-machina-_lipc3zt\lib\site-packages\pkg_resources\__init__.py", line 777, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'colorama; sys_platform == "win32"' distribution was not found and is required by pytest

Since this is an issue in pluggy, I've created a separate issue to further track this: pytest-dev/pluggy#174

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: critical grave problem or usability issue that affects lots of users type: backward compatibility might present some backward compatibility issues which should be carefully noted in the changelog type: bug problem that needs to be addressed
Projects
None yet
Development

No branches or pull requests

6 participants