Don't fail on packages during dependency resolution

[bra-fsn]

Description

If you give pip a large enough requirements.txt with no direct version constraints, it sooner or later will go berserk and try to download/parse all versions.
This means it will run into all kinds of issues where the setup.py fails to execute, either because language incompatibilty or other stuff, like deprecated setuptools features (for eg. https://setuptools.pypa.io/en/latest/history.html#v58-0-0).
So instead of dying and breaking the whole process, wouldn't it better to just skip those packages, whose setup.py couldn't run?
We won't be able to install them anyways, right?

Expected behavior

I would expect that if there is a compatible mix of packages, pip should be able to actually find out a working version list and not fail on multi-year packages, which aren't able to run anyways.

pip version

22.1.2

Python version

3.8.10

OS

Ubuntu 20.04.4 LTS

How to Reproduce

Give pip this list:

aiobotocore[boto3]
aiohttp
aioitertools
aiosignal
alembic
asn1crypto
async-timeout
attrs
autohash
autopage
azure-core
azure-storage-blob
bcrypt
berkeleydb
blis
boto
boto3
botocore
cached-property
cachetools
catalogue
catboost
certifi
cffi
chardet
charset-normalizer
click-help-colors
click
cliff
cloudperf
cloudpickle
cmaes
cmd2
coloredlogs
colorful
colorlog
ConfigIt
cramjam
cryptography
cssselect
curlify
cycler
cymem
Cython
dask
deansi
decorator
Deprecated
detoxify
df-io
dill
distlib
distributed
docopt
domonic
elementpath
facebook-business
fastai
fastcore
fastdownload
fastjsonschema
fastparquet
fastprogress
filelock
flake8
Flask
flatten-dict
fonttools
frozenlist
fsspec
future
geoip2
gitdb
GitPython
google-auth-oauthlib
google-auth
googleads
graphviz
greenlet
grpcio-tools
grpcio
gspread-dataframe
gspread
hashable-df
HeapDict
html5lib
httpagentparser
httplib2
huggingface-hub
humanfriendly
idna
imbalanced-learn
importlib-metadata
importlib-resources
iniconfig
iptools
isodate
isort
itsdangerous
Jinja2
jmespath
joblib
jobspy
jsonschema
kiwisolver
langcodes
lightgbm
locket
lxml
makefun
Mako
MarkupSafe
matplotlib
maxminddb
mccabe
more-itertools
msgpack
msrest
multidict
murmurhash
networkx
newrelic
nltk
numpy
oauth2client
oauthlib
optional-import
optuna
oscrypto
packaging
pandag
pandas
parallel-write
paramiko
partd
pathy
pbr
pid-controller
Pillow
pipdeptree
pkgmod
platformdirs
plotly
pluggy
preshed
prettytable
protobuf
psutil
psycopg2
py
pyarrow
pyasn1-modules
pyasn1
pycodestyle
pycountry
pycparser
pycryptodomex
pydantic
pyflakes
PyJWT
PyNaCl
pynamodb
pyOpenSSL
pyparsing
pyperclip
pyrsistent
pytest
python-dateutil
pytimeparse
pytz
PyYAML
redis
regex
requests-file
requests-oauthlib
requests-toolbelt
requests
requirements-parser
retry
retrying
rsa
s1crets
s3fs
s3transfer
scikit-learn
scipy
sentence-transformers
sentencepiece
sentry-sdk
simplejson
six
slack-sdk
smart-open
smmap
snowflake-connector-python[pandas]
snowflake-sqlalchemy
sortedcontainers
spacy-legacy
spacy-loggers
spacy
SQLAlchemy
srsly
stevedore
tabulate
tblib
tenacity
thinc
threadpoolctl
thrift
tokenizers
toml
tomli
toolz
tornado
tox-battery
tox-pip-version
tox
tqdm
transformers
typeguard
typer
types-setuptools
typing_extensions
uritemplate
urllib3
virtualenv
wasabi
wcwidth
webencodings
Werkzeug
wrapt
xgboost
XlsxWriter
xmltodict
xxhash
yarl
zeep
zict
zipp
zstandard
autogluon.text
elasticsearch==7.10.1
marshmallow==2.21.0
torch
torchvision

It will eventually die with

Collecting decorator
  Downloading decorator-3.4.2.tar.gz (6.1 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in decorator setup command: use_2to3 is invalid.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

or this:

INFO: pip is looking at multiple versions of optuna to determine which version is compatible with other requirements. This could take a while.
  Downloading optional_import-1.0.tar.gz (2.1 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-ghtslwyr/optional-import_e4ed4101b15a499e8d90c81d64f6fc47/setup.py", line 1, in <module>
          from ez_setup import use_setuptools
      ModuleNotFoundError: No module named 'ez_setup'
      [end of output]

Or anything like those...

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

This behavior is intentional, you can see the discussion and reasoning here: #10655 and the relevant pull requests #10722

But in summary:

• Pip treats a user requirements file as the packages that Pip must install, so if one can not be installed it's up to the user to fix the requirements file
• Pip has no way to know ahead of time if a package build is going to fail
• The alternative behavior is to keep downloading older packages until finding a backtracking solution that doesn't fail
• The build failure may not be related to the version of the package but rather the user environment, e.g. when I tried to install your requirements berkeleydb fails because I don't have "a local Oracle Berkeley DB installation", no amount of downloading other versions of berkeleydb is going to fix that.
• Therefore the onus is put on the user to improve their requirements and constraints files, by putting lower bounds on the requirements or constraints or fixing their environment, rather than letting Pip potentially run forever and download 1000s of files

If you read in to the discussions I was definitely one of the voices who was skeptical of this change thinking it would cause a large user outcry. I'm happy to say that, as far as I know, few people seem to have complained.

[Midnighter]

I have a similar issue with a shorter list of packages

    darglint
    flake8
    flake8-bandit
    flake8-black
    flake8-bugbear
    flake8-builtins
    flake8-comprehensions
    flake8-docstrings
    flake8-eradicate
    flake8-isort
    flake8-pytest-style

this worked perfectly fine until the new backtracking was implemented. My only way to resolve this is to use pip-tools to compile the dependencies, then set those versions as constraints. Not sure how much longer this will work. From a user perspective, this new dependency resolution definitely feels like a step backwards. It's much slower in general, too.

[uranusjr]

Using constraints is the recommended way to supply the correct user intent and will continue to work for the foreseeable future.

I’m going to close this since failing on a package is an intentional design decision.

If you are interested in history, the resolver actually used to do what is requested in this issue (skip failed packages) when it was first implemented, but it generated a pretty significant number of users wanting this changed, so we changed it. If I remember correctly, there are less than a handful of reports wanting the “old” behaviour, so IMO the current design is likely the correct choice.

I also want to mention that if a certain version of a package can’t correctly build on your working environment, it is likely you don’t want that version in the first place, and you should express that in your requirements with version constraints such as >= and !=. It is better for both the tools and human maintainers, including yourself in the future.