[UX?] Facilitate weird DeprecationWarning implementations

[webknjaz]

Yesterday cryptography published an update that injects a DeprecationWarning right into their package __init__. Their DeprecationWarning is custom and is declared in their own project, in cryptography.utils submodule.

So adding

filterwarnings =
    error
    ignore:Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.:cryptography.utils.CryptographyDeprecationWarning

makes pytest explode:

$ pytest --testmon-off --collect-only
Test session starts (platform: linux2, Python 2.7.15, pytest 4.6.11, pytest-sugar 0.9.4)
cachedir: .tox/py27/.pytest_cache
rootdir: ~/src/github/cherrypy/cheroot, inifile: pytest.ini, testpaths: cheroot/test/
plugins: forked-1.2.0, xdist-1.33.0, mock-2.0.0, cov-2.7.1, sugar-0.9.4, testmon-0.9.19
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 206, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 249, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/callers.py", line 81, in get_result
INTERNALERROR>     _reraise(*ex)  # noqa
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/pluggy/callers.py", line 182, in _multicall
INTERNALERROR>     next(gen)  # first yield
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/_pytest/warnings.py", line 151, in pytest_collection
INTERNALERROR>     config=config, ihook=config.hook, when="collect", item=None
INTERNALERROR>   File "~/.pyenv/versions/2.7.15/lib/python2.7/contextlib.py", line 17, in __enter__
INTERNALERROR>     return self.gen.next()
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/_pytest/warnings.py", line 88, in catch_warnings_for_item
INTERNALERROR>     _setoption(warnings, arg)
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/_pytest/warnings.py", line 27, in _setoption
INTERNALERROR>     category = wmod._getcategory(category)
INTERNALERROR>   File ".tox/py27/lib/python2.7/warnings.py", line 184, in _getcategory
INTERNALERROR>     m = __import__(module, None, None, [klass])
INTERNALERROR>   File ".tox/py27/lib/python2.7/site-packages/cryptography/__init__.py", line 39, in <module>
INTERNALERROR>     CryptographyDeprecationWarning,
INTERNALERROR> CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.


Results (0.03s):

To be fair, it's the behavior of the interpreter:

$ python27 -W ignore::cryptography.utils.CryptographyDeprecationWarning
Invalid -W option ignored: invalid module name: 'cryptography.utils'
Python 2.7.15 (default, Sep 25 2018, 19:19:39) 
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cryptography
.tox/py27/lib/python2.7/site-packages/cryptography/__init__.py:39: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  CryptographyDeprecationWarning,

Does pytest try to import the ignored category on its own on startup? Is there any way it could work around such cases?

[RonnyPfannschmidt]

this is a structural issue

the setoption in python is ignored because cryptography is not yet importable
in pytest the setoption blows up because cryptography is importable

[RonnyPfannschmidt]

i believe you need the ignore line twice, first to trigger the import before the error spec then after the error spec to ensure it works

its unfortunate the warning triggers directly on import

[webknjaz]

i believe you need the ignore line twice, first to trigger the import before the error spec then after the error spec to ensure it works

If I understand you correctly, you're suggesting that I do this:

filterwarnings =
    ignore:Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.:cryptography.utils.CryptographyDeprecationWarning:cryptography
    error
    ignore:Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.:cryptography.utils.CryptographyDeprecationWarning:cryptography

It does work (tried under pytest 4.6.11 because this is related to Python 2). But then, it also works if I don't have that second ignore, as in:

filterwarnings =
    ignore:Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.:cryptography.utils.CryptographyDeprecationWarning:cryptography
    error

I assume that is because the imported modules are cached and the following imports don't emit warnings because the module is not interpreted for the second time.

[RonnyPfannschmidt]

Exactly

[webknjaz]

Looks like there's no way to improve this programmatically.
The action items here may be:

1. Document this quirk
2. Try detecting the chicken-egg cases and print out something explanatory