Last-failed option missing tests when failed tests include a mix stable and random node IDs

[goodspark]

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using

  attrs (22.1.0)
  Faker (14.2.0)
  importlib-metadata (4.12.0)
  iniconfig (1.1.1)
  packaging (21.3)
  pip (9.0.1)
  pkg-resources (0.0.0)
  pluggy (1.0.0)
  py (1.11.0)
  pyparsing (3.0.9)
  pytest (7.1.2)
  pytest-faker (2.0.0)
  python-dateutil (2.8.2)
  setuptools (39.0.1)
  six (1.16.0)
  tomli (2.0.1)
  typing-extensions (4.3.0)
  zipp (3.8.1)
  

• pytest and operating system versions
  • Python 3.7.5
  • pytest 7.1.2
  • Ubuntu 18.04.6
• minimal example if possible

1. Start with the given test code.

   import pytest
   from faker import Faker
   fake = Faker()
   
   @pytest.mark.parametrize('a', (fake.pystr(),))
   def test_asd(a):
       assert False
   
   def test_zxc():
       assert False

2. Run pytest once

   pytest --tb=no -q
   FF                                                                                             [100%]
   ====================================== short test summary info =======================================
   FAILED test_asd.py::test_asd[fPLeEKOcteuKAoJyUmpz] - assert False
   FAILED test_asd.py::test_zxc - assert False
   2 failed in 0.05s
   

3. Run pytest again with the --last-failed option

   pytest --tb=no -q --lf
   F                                                                                              [100%]
   ====================================== short test summary info =======================================
   FAILED test_asd.py::test_zxc - assert False
   1 failed in 0.05s
   

4. Expect both tests to be rerun - only test_zxc is rerun

Initially we thought this was because test_asd is using a randomized parameterized fixture. Since the node IDs are essentially random it would make sense pytest doesn't reselect them.

But if you remove test_zxc in the example above and run steps 2 and 3, pytest actually works and reruns test_asd and the node IDs are different each time.

pytest --tb=no -q; pytest --tb=no -q --lf
F                                                                                              [100%]
====================================== short test summary info =======================================
FAILED test_asd.py::test_asd[IBWBNcpQQilmccBLPmIB] - assert False
1 failed in 0.05s
F                                                                                              [100%]
====================================== short test summary info =======================================
FAILED test_asd.py::test_asd[AsUfmfLBeTdfakmHXPmm] - assert False
1 failed in 0.05s

[Zad2]

Hello @goodspark I'm having a very similar issue, did you get to the bottom of this issue?