pytest 6: How to have a pytest plugin depend on a version of pytest?

[okken]

• a detailed description of the bug or suggestion

Is there a way to specify a pytest version dependency, so new versions of a plugin support pytest 6, while if pytest 5 is around, a previous version of the plugin is used?

I know this is more of a Python dependency management thing. But since pytest relies on and welcomes the plugin community, this seems like it should be documented in the "writing plugins" section.

I'm checking to see if pytest 6.0.0rc1 is compatible with a plugin I support, pytest-check.

Well, it's not, because:

• The internal structure of how xfail works has changed.
  • pytest.skipping.evalxfail_key changed to xfailed_key
  • item._store[evalxfail_key], now item._store[xfailed_key] no longer has wasvalid() or istrue() methods.
  • .getexplanation() changed to .reason
  • taking my que from pytest_runtest_call from skipping.py to figure out how to interact with xfail data structure.
• For pytester, assert_outcomes() change from "error" to "errors" as keywords.

That's fair enough, kinda, it's a new major version, and breaking changes are allowed.
I've already worked through the necessary changes, but it definitely won't work now with older versions of pytest.

So, what is the recommendation for plugin authors to have a plugin depend on a min version of pytest.

Does normal min/max dependency stuff in packaging work?

• output of pip list from the virtual environment you are using

$ pip list
Package        Version
-------------- --------
attrs          19.3.0
iniconfig      1.0.0
more-itertools 8.4.0
packaging      20.4
pip            20.1.1
pluggy         0.13.1
py             1.9.0
pyparsing      2.4.7
pytest         6.0.0rc1
pytest-check   0.3.9
setuptools     46.4.0
six            1.15.0
toml           0.10.1
wcwidth        0.2.5
wheel          0.34.2

• pytest and operating system versions

pytest 6.0.0rc1, macOS 10.15.5 (19F101), Darwin 19.5.0

• minimal example if possible

$ python -m venv venv
$ source venv/bin/activate
(venv) $ git clone https://github.com/okken/pytest-check
(venv) $ cd pytest-check
(venv) $ pip install pytest
...
Successfully installed ... pytest-5.4.3 ...
(venv) $ pip install ./
(venv) $ pytest
...
====== 35 passed in 0.73s =====
(venv) $ pip install pytest==6.0.0rc1
...
Successfully installed ... pytest-6.0.0rc1 ...
(venv) $ pytest --tb=no
================================ test session starts =================================
platform darwin -- Python 3.8.3, pytest-6.0.0rc1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/okken/projects/pytest-check/foo/pytest-check, configfile: tox.ini
plugins: pytest_check-0.3.9
collected 35 items                                                                   

tests/test_check.py .................FF....                                    [ 65%]
tests/test_check_context_manager.py ......                                     [ 82%]
tests/test_check_errors.py FF                                                  [ 88%]
tests/test_check_fixture.py .                                                  [ 91%]
tests/test_check_func_decorator.py ...                                         [100%]

============================== short test summary info ===============================
FAILED tests/test_check.py::test_check_xfail - AssertionError: assert {'errors': 0,...
FAILED tests/test_check.py::test_check_xfail_strict - AssertionError: assert {'erro...
FAILED tests/test_check_errors.py::test_setup_failure - TypeError: assert_outcomes(...
FAILED tests/test_check_errors.py::test_teardown_failure - TypeError: assert_outcom...
============================ 4 failed, 31 passed in 0.69s ============================

[okken]

Also, is there a way to check the version of pytest at runtime to use as a behavior switch?

[Zac-HD]

Does normal min/max dependency stuff in packaging work?

Yep, install_requires=["pytest>=6.0.0", ...] in your plugin's setup.py (or whatever packaging tool you prefer) will do the usual thing.

Personally I also add a runtime check on pytest.__version__ though, because pip is still quite capable of installing package versions that don't work togther - for example by installing your plugin and then downgrading pytest afterwards. The ongoing resolver project should help with that, but something like this is IMO still worth having.

[The-Compiler]

Closing this as I think @Zac-HD's answer is spot on!

[okken]

Thanks @Zac-HD @The-Compiler