Pytest Assertion Error

[azihna]

Describe the bug

In today's version of sklearn, trying to run the tests has the result of collecting the test than failing by throwing the assertion shown below. Trying different versions of pytest and pluggy didn't solve the issue.

The environment was created following the steps shown in the following link.

Steps/Code to Reproduce

Example:
pytest sklearn

Expected Results

Tests should run.

Actual Results

=========================================================================================================== test session starts ============================================================================================================
platform win32 -- Python 3.8.5, pytest-6.1.2, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\AZihna\homl\skdev\scikit-learn, configfile: setup.cfg
collected 19705 items

sklearn\_config.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\main.py", line 257, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\main.py", line 313, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\main.py", line 338, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\runner.py", line 110, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\runner.py", line 121, in runtestprotocol
INTERNALERROR>     rep = call_and_report(item, "setup", log)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\runner.py", line 218, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)  # type: TestReport
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\pluggy\callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "c:\users\azihna\miniconda3\envs\sklearndev\lib\site-packages\_pytest\skipping.py", line 306, in pytest_runtest_makereport
INTERNALERROR>     assert line is not None
INTERNALERROR> AssertionError

Versions

sklearn.show_versions()

System:
    python: 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)]
   executable: %HOME%\Miniconda3\envs\sklearndev\python.exe
   machine: Windows-10-10.0.19041-SP0

Python dependencies:
          pip: 20.3.3
         setuptools: 51.3.3.post20210118
        sklearn: 1.0.dev0
        numpy: 1.19.2
        scipy: 1.5.2
       Cython: 0.29.21
       pandas: 1.2.1
       matplotlib: 3.3.2
       joblib: 1.0.0
       threadpoolctl: 2.1.0

Built with OpenMP: True

[mjkanji]

I ran into the same issue when setting up my environment for yesterday's Data Umbrella sprint, and brought it up with @amueller. Switching to a GitHub codespaces VM running Linux solved the problem for me at the same, so perhaps it's a Windows-specific issue?

[glemaitre]

@adrinjalali was suggesting that it could come from having packages from conda-forge, conda default and pip.
I have to check the tutorial of @amueller to check how the install was intended.

[azihna]

The tests work after downgrading to pytest 5.0.1 (the lowest acceptable version). I have found some other issues in pytest that people have this issue on windows or on Linux after some hotfixes (example1, example2). I think it might be better to raise it up with the pytest team.