Option --last-failed with all prior failures filtered runs no tests

[k-husmann]

When using --last-failed and there are no last failures, per default and as documented, all tests should run. That works fine unless combined with markers, if those filter all prior failures -> then it runs no tests. It would be more convenient if that could be detected, i.e. if all prior failures are filtered --lf should behave as if there had been NO last failures => rerun all.

pseudo - example:

$ pytest
... 1 test fails, which is marked 'bad'
... make some changes that will also fail another test 'later_bad' 
pytest --lf -m 'not bad'
... no tests are run, later_bad is not detected
... wanted behaviour: realize that all last failed are filtered => rerun all not-filtered tests, i.e. as without --lf

As a workaround I use:

pytest --lf -m 'not bad'
if [ "$?" == 5 ]; then pytest  -m 'not bad'

I am running pytest under Cygwin, and that is my version:

$ pytest --version
This is pytest version 3.8.2, imported from /usr/lib/python3.6/site-packages/pytest.py
setuptools registered plugins:
  pytest-cov-2.6.0 at /usr/lib/python3.6/site-packages/pytest_cov/plugin.py
PARSEER <class '_pytest.config.argparsing.Parser'>

Btw, thanks for the good work, pytest is great! If you are interested in my upgrade-feature request, I happily provide a full test case with py-code and shell script.

[blueyed]

@RonnyPfannschmidt
I do not think it is a regression, is it?

This is pytest version 3.8.2

Please try 3.10.1, just in case.

I actually like the current behavior more: with -m you limit the test collection already, and it should not cause tests to run that are not matching this.

[k-husmann]

I actually like the current behavior more: with -m you limit the test collection already, and it should not cause tests to run that are not matching this.

It is true, that tests filtered via -m switch should not (ever) be run. And that is not my request. Only that --lf, should run all-not-otherwise-filtered tests if it finds no last failed ones, after having applied the -m switch.

I'll build a test case for this and test it with 3.10.1 in my private environment as I cannot provide this during working hours/corporate environment. Expect an update within the next few days.

[blueyed]

JFI: 1f5a61e improves the reporting/feedback, i.e. it should be easier to see what is going on (not released yet IIRC, but in the features branch).