When both a directory and a file are passed on the CLI, only the file is run

[JelleZijlstra]

When the command-line args include both a directory with test files and a specific test file in that directory, pytest runs only the file and not the other files in that directory.

As an example:

$ pytest --collect-only pycroscope/ pycroscope/test_analysis_lib.py 
================================================================================================= test session starts =================================================================================================
platform darwin -- Python 3.12.2, pytest-8.4.0, pluggy-1.5.0
rootdir: /Users/jelle/py/pycroscope
configfile: pyproject.toml
collected 3 items                                                                                                                                                                                                     

<Dir pycroscope>
  <Package pycroscope>
    <Module test_analysis_lib.py>
      <Function test_get_indentation>
      <Function test_get_line_range_for_node>
      <Function test_object_from_string>

============================================================================================= 3 tests collected in 0.10s ==============================================================================================
$ pytest --collect-only pycroscope/
================================================================================================= test session starts =================================================================================================
platform darwin -- Python 3.12.2, pytest-8.4.0, pluggy-1.5.0
rootdir: /Users/jelle/py/pycroscope
configfile: pyproject.toml
collected 1026 items / 3 skipped                                                                                                                                                                                      

This is dangerous because it can lead us to silently skip running some tests.

[lubenicapp]

Hello, first of all this is the first time I dive into this code

I've tracked the bug down to this line: : https://github.com/pytest-dev/pytest/blob/main/src/_pytest/main.py#L793

It populates Session._collection_cache, which causes genitems to skip cached nodes here: https://github.com/pytest-dev/pytest/blob/main/src/_pytest/main.py#L975

I have tried the most atomic fix I can think of for this main...lubenicapp:pytest:13494-skipped-files

but due to the high coupling in the code and opacity, it breaks 137 other tests

Kinda feel hopeless for this

[bluetech]

This is a duplicate of #12083. It needs some thought and experimentation to get at a good result...