Please describe the installation error better

[celestinoxp]

Description

I made a pull-request in pycaret to support scikit-learn 1.4. When I send the pull-request, github fails the installation and doesn't even get to the tests. The problem is that I can't understand what the problem is! It should show a more specific message and show if there is a package that is in conflict and which one!

ERROR: Exception:
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_internal\commands\install.py", line 377, in run
    requirement_set = resolver.resolve(
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 457, in resolve
    raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000
Error: Process completed with exit code 2.

Expected behavior

.

pip version

23.3.2

Python version

3.9, 3.10, 3.11

OS

Windows / Linux

How to Reproduce

https://github.com/pycaret/pycaret/actions/runs/7666101173/job/20893249869?pr=3857

Output

...
Downloading fastapi-0.108.0-py3-none-any.whl.metadata (24 kB)
Collecting starlette<0.33.0,>=0.29.0 (from fastapi>=0.75.0->pycaret==3.3.0)
Downloading starlette-0.32.0.post1-py3-none-any.whl.metadata (5.8 kB)
INFO: pip is still looking at multiple versions of mlflow to determine which version is compatible with other requirements. This could take a while.
Collecting fugue[dask]>=0.8.0 (from pycaret==3.3.0)
Downloading fugue-0.8.6-py3-none-any.whl.metadata (18 kB)
Collecting dask<2023.7.1,>=2022.9.0 (from dask[dataframe,distributed]<2023.7.1,>=2022.9.0; python_version >= "3.8" and extra == "dask"->fugue[dask]>=0.8.0; extra == "full"->pycaret==3.3.0)
Downloading dask-2023.7.0-py3-none-any.whl.metadata (3.6 kB)
Collecting pyarrow>=0.15.1 (from fugue[dask]>=0.8.0; extra == "full"->pycaret==3.3.0)
Downloading pyarrow-6.0.1-cp310-cp310-win_amd64.whl (15.5 MB)
--------------------------------------- 15.5/15.5 MB 73.0 MB/s eta 0:00:00
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting fugue[dask]>=0.8.0 (from pycaret==3.3.0)
Downloading fugue-0.8.5-py3-none-any.whl.metadata (18 kB)
Downloading fugue-0.8.4-py3-none-any.whl.metadata (17 kB)
Downloading fugue-0.8.3-py3-none-any.whl (372 kB)
------------------------------------- 372.4/372.4 kB 22.6 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of fugue[dask] to determine which version is compatible with other requirements. This could take a while.
Downloading fugue-0.8.2-py3-none-any.whl (370 kB)
------------------------------------- 370.2/370.2 kB 11.2 MB/s eta 0:00:00
Downloading fugue-0.8.1-py3-none-any.whl (364 kB)
------------------------------------- 364.4/364.4 kB 22.1 MB/s eta 0:00:00
Downloading fugue-0.8.0-py3-none-any.whl (356 kB)
------------------------------------- 356.1/356.1 kB 23.1 MB/s eta 0:00:00
Collecting fastapi>=0.75.0 (from pycaret==3.3.0)
Downloading fastapi-0.107.0-py3-none-any.whl.metadata (24 kB)
Collecting anyio<4.0.0,>=3.7.1 (from fastapi>=0.75.0->pycaret==3.3.0)
Downloading anyio-3.7.1-py3-none-any.whl.metadata (4.7 kB)
Collecting starlette<0.29.0,>=0.28.0 (from fastapi>=0.75.0->pycaret==3.3.0)
Downloading starlette-0.28.0-py3-none-any.whl.metadata (5.8 kB)
ERROR: Exception:
Traceback (most recent call last):
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_internal\cli\base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_internal\cli\req_command.py", line 245, in wrapper
return func(self, options, args)
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_internal\commands\install.py", line 377, in run
requirement_set = resolver.resolve(
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_internal\resolution\resolvelib\resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\pip_vendor\resolvelib\resolvers.py", line 457, in resolve
raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000
Error: Process completed with exit code 2.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

"Resolution too deep" means that pip was unable to determine if there was a package in conflict before it hit the limit on how many attempts to resolve it made. So we can't say what packages (if any) were in conflict. But I agree the error could be improved to explain what's going on.

PRs would be welcome (from any interested community member) to improve the error in this case, otherwise it's something that will have to wait until one of the maintainers has the time to look into this. It should be a relatively simple fix, if anyone is interested in taking a look.

[notatallshaw]

Let's take a step back, I beleive OPs motiviation is in solving their problem (i.e. their failing PR because Pip spends hours never reaching a resolution), and that OP beleives with a better error message when this happens may be able to do this.

I have good news about solving your problem, and bad news about a better error message.

The bad news first, having been working on Pip resolution problems now for ~4 years, I don't think there is any easy path forward to producing a better error messsage in this scenario, where some arbitaray limit is hit and the resolution gives up, as Pip does not use a resolver which produces a detailed explaination of the path it took or why it got stuck. I do beleive Pip could do better logging, such that if you take the time to understand the high level approach to how Pip resolves and you take the time to pick apart the logging you could figure out the likely cause. I may open a PR on this soon.

The good news, a better, and turns out easier, solution to your issue is to just have Pip be better at resolving conflicts. And in fact, I have an open PR that does just that: #12459. I checked out your PR, was able to produce the long backtracking, and then checked out my PR to see if would help, it did, and very quickly it produced this message:

ERROR: Cannot install None, pycaret and pycaret[full]==3.3.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    pycaret 3.3.0 depends on scikit-learn<1.5.0 and >=1.4.0
    pycaret[full] 3.3.0 depends on scikit-learn<1.5.0 and >=1.4.0
    fairlearn 0.7.0 depends on scikit-learn>=0.22.1
    sktime 0.25.1 depends on scikit-learn<1.4.0 and >=0.24

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Resolution impossible errors are the ones that tend to take the most amount of time to resolve, and therefore are good candidates to see how intelligent (or not intelligent) your resolution algorithm is. As we can see here there is a conflict between sktime requirement on scikit-learn and your own requirement on scikit-learn.

You wrote in your PR:

sktime>=0.25.1 # 0.25.1 support for scikit-learn 1.4

But I can see, unfortunately this isn't the case: https://github.com/sktime/sktime/blob/v0.25.1/pyproject.toml#L60

Do let me know if you have any more questions.

[celestinoxp]

I asked sktime for information sktime/sktime#5831
​it seems that my PR is dependent on sktime

[pradyunsg]

Consolidating into #11480