Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`--ignore-requires-python` has no effect #6371

Closed
asottile opened this issue Mar 31, 2019 · 22 comments

Comments

@asottile
Copy link
Contributor

commented Mar 31, 2019

Environment

  • pip version:
$ ./venv/bin/pip --version
pip 19.0.3 from /tmp/t/venv/site-packages/pip (python 3.5)
  • Python version:
$ ./venv/bin/python --version
Python 3.5.3 (fdd60ed87e94, Apr 24 2018, 06:10:04)
[PyPy 6.0.0 with GCC 6.2.0 20160901]
  • OS:
$ uname -a
Linux asottile-VirtualBox 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.2 LTS
Release:	18.04
Codename:	bionic

Description / Expected behaviour / How to Reproduce

I expect this to succeed:

$ ./venv/bin/pip install --ignore-requires-python black -v

black requires python>=3.6, though I am running pypy3.5 and explicitly asking pip to --ignore-requires-python

Output

$ ./venv/bin/pip install --ignore-requires-python black -v
Created temporary directory: /tmp/pip-ephem-wheel-cache-7_gvlskb
Created temporary directory: /tmp/pip-req-tracker-t4ef_ow9
Created requirements tracker '/tmp/pip-req-tracker-t4ef_ow9'
Created temporary directory: /tmp/pip-install-3x0i9w7v
Collecting black
  1 location(s) to search for versions of black:
  * https://pypi.org/simple/black/
  Getting page https://pypi.org/simple/black/
  Looking up "https://pypi.org/simple/black/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/black/ HTTP/1.1" 304 0
  Analyzing links from page https://pypi.org/simple/black/
    The package https://files.pythonhosted.org/packages/22/5a/ebe5881cbf07435ea1ad76119ee142f42fb7a333d562c202db015b959ae7/black-18.3a0-py3-none-any.whl#sha256=7183263650ba3071034e90b40a1ea74abccbd32cf525cef6d7914479dbe7f2fb (from https://pypi.org/simple/black/) (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6
    The package https://files.pythonhosted.org/packages/83/a2/fae3ddb516a52a0f37e6e2ef3bd789b6d093d2c7015a41a41bff30f39897/black-18.3a0.tar.gz#sha256=0461c7a52b5beb378936bf642753dec7a45305c96c6129d540b9c53227121a5a (from https://pypi.org/simple/black/) (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6
    The package https://files.pythonhosted.org/packages/61/b4/3a39dcf59342f53680c59a5e44c0b172ea5d6470a5aaba8ae3cdbfccc1e7/black-18.3a1-py3-none-any.whl#sha256=04edc87b51b34a7aa093f39bf207e2ebc3612b26f2c79fdf8ff13b2d60622c9a (from https://pypi.org/simple/black/) (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6

[ many repeated lines removed ]


    The package https://files.pythonhosted.org/packages/e8/5f/0f79fcd943ba465cbd4bf303c9794970c13a95e5456630de9f72e7f37ad4/black-18.9b0.tar.gz#sha256=e030a9a28f542debc08acceb273f228ac422798e5215ba2a791a6ddeaaca22a5 (from https://pypi.org/simple/black/) (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6
    Skipping link https://files.pythonhosted.org/packages/30/62/cf549544a5fe990bbaeca21e9c419501b2de7a701ab0afb377bc81676600/black-19.3b0-py36-none-any.whl#sha256=09a9dcb7c46ed496a9850b76e4e825d6049ecd38b611f1224857a79bd985a8cf (from https://pypi.org/simple/black/) (requires-python:>=3.6); it is not compatible with this Python
    The package https://files.pythonhosted.org/packages/89/07/aebb10fb8f2ffbac672dfbebffa724643bc84cf012a57737a622d1dabddb/black-19.3b0.tar.gz#sha256=68950ffd4d9169716bcb8719a56c07a2f4485354fec061cdd5910aa07369731c (from https://pypi.org/simple/black/) (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6
  Could not find a version that satisfies the requirement black (from versions: )
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-t4ef_ow9'
No matching distribution found for black
Exception information:
Traceback (most recent call last):
  File "/tmp/t/venv/site-packages/pip/_internal/cli/base_command.py", line 179, in main
    status = self.run(options, args)
  File "/tmp/t/venv/site-packages/pip/_internal/commands/install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "/tmp/t/venv/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/tmp/t/venv/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/t/venv/site-packages/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
    self.require_hashes
  File "/tmp/t/venv/site-packages/pip/_internal/operations/prepare.py", line 269, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/tmp/t/venv/site-packages/pip/_internal/req/req_install.py", line 196, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/tmp/t/venv/site-packages/pip/_internal/index.py", line 688, in find_requirement
    'No matching distribution found for %s' % req
pip._internal.exceptions.DistributionNotFound: No matching distribution found for black
@asottile

This comment has been minimized.

Copy link
Contributor Author

commented Mar 31, 2019

I'm ok with this being closed as "works as intended" though the option should probably be deprecated and/or removed if it has no effect

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

Are you able to get the install to work by passing something for --python-version? My understanding is that --ignore-requires-python has a slightly different purpose (introduced in response to issue #3846).

@cjerdonek cjerdonek added the triage label Apr 1, 2019

@asottile

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2019

nope:

$ pip install --ignore-requires-python --python-version 36 black
ERROR: When restricting platform and interpreter constraints using --python-version, --platform, --abi, or --implementation, either --no-deps must be set, or --only-binary=:all: must be set and --no-binary must not be set (or must be set to :none:).
@asottile

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2019

I even reverted back to when it was introduced -- it didn't work then either:

$ pip install git+https://github.com/pypa/pip@8df742e56db99c48f7ad85085501fe23456b419c
Collecting git+https://github.com/pypa/pip@8df742e56db99c48f7ad85085501fe23456b419c
  Cloning https://github.com/pypa/pip (to revision 8df742e56db99c48f7ad85085501fe23456b419c) to ./pip-req-build-mdzrw938
Building wheels for collected packages: pip
  Building wheel for pip (setup.py) ... done
  Stored in directory: /tmp/pip-ephem-wheel-cache-3htnnj3r/wheels/f4/06/5d/1652f21a0ecf6bed88b63647ec859e0384f0f07fcae05c57f3
Successfully built pip
Installing collected packages: pip
  Found existing installation: pip 19.0.3
    Uninstalling pip-19.0.3:
      Successfully uninstalled pip-19.0.3
Successfully installed pip-8.2.0.dev0
$ pip install --ignore-requires-python black
Collecting black
  Cache entry deserialization failed, entry ignored
  Could not find a version that satisfies the requirement black (from versions: )
No matching distribution found for black
You are using pip version 8.2.0.dev0, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

The ignore-requires option wasn’t meant to be used for the purpose you’re trying to use it. I was wondering if another option could help though.

@asottile

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2019

my reading of #3846 especially these comments seems to indicate it is intended for this:

#3846 (comment)

#3846 (comment)

2df0ea9

The --python-version / abi / etc. parameters did not even exist at the time, my reading is that this argument was intended as an escape hatch for pip refusing to install incompatible requires_python (what else could it be for?)

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

ERROR: When restricting platform and interpreter constraints using --python-version, --platform, --abi, or --implementation, either --no-deps must be set, or --only-binary=:all: must be set and --no-binary must not be set (or must be set to :none:).

When you're trying --python-version, can you also try adding --only-binary=:all: like the error message says?

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

(what else could it be for?)

When installing a package, Python performs a check of the Requires-Python metadata prior to allowing it:

try:
check_dist_requires_python(dist)
except UnsupportedPythonVersion as err:
if self.ignore_requires_python:
logger.warning(err.args[0])
else:
raise

The --ignore-requires-python option is an escape hatch to suppress that check.

In your case, it's rejecting the package before even getting to that point (before it's even downloaded), so the command option isn't coming into play. The --python-version and related options are for controlling whether a package is eligible for download.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

Okay, it looks like there is a second application of check_requires_python() that isn't taking ignore_requires_python into account:

try:
support_this_python = check_requires_python(link.requires_python)
except specifiers.InvalidSpecifier:
logger.debug("Package %s has an invalid Requires-Python entry: %s",
link.filename, link.requires_python)
support_this_python = True
if not support_this_python:
logger.debug("The package %s is incompatible with the python "
"version in use. Acceptable python versions are: %s",
link, link.requires_python)
return None

@cjerdonek cjerdonek added type: bug C: finder and removed triage labels Apr 1, 2019

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

PS - the most important part of the log messages in your OP are getting hidden by default because GitHub doesn't wrap the long lines when using the fixed-width formatting:

... (requires-python:>=3.6) is incompatible with the python version in use. Acceptable python versions are: >=3.6

I didn't see that until now.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

(Note that the wheels will still get rejected without using —python-version.)

@asottile

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2019

😆 I'm glad you finally took the time to look instead of assuming I'm wrong

I've been following closely the --python-version / platform / abi / etc. options, they are a non-solution until the --target requirement is lifted: #5453 (comment)

@uranusjr

This comment has been minimized.

Copy link
Member

commented Apr 2, 2019

Out of curiosity, what does it mean to pass --ignore-requires-python without --python-version? That the Python… does not have a version? That seems a bit pointless in this case. I would expect these two to be mutually exclusive: if --python-version is passed, the value is used to check against Requires-Python (both in metadata and on index), and if --ignore-requires-python is passed, checks against Requires-Python are disabled altogether (so all links are collected on the index).

@xavfernandez

This comment has been minimized.

Copy link
Contributor

commented Apr 24, 2019

FWIW, --ignore-requires-python was added (and still has the expected effect) for escape hatching this kind of case:

$ docker run -ti --rm python:3.4 bash
root@b30693b8f33e:/# pip install https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Collecting Django==2.2 from https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl
  Downloading https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl (7.4MB)
    100% |████████████████████████████████| 7.5MB 3.6MB/s 
Django requires Python '>=3.5' but the running Python is 3.4.10
You are using pip version 19.0.3, however version 19.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@b30693b8f33e:/# echo $?
1
root@b30693b8f33e:/# pip install https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl --ignore-requires-python
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Collecting Django==2.2 from https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl
  Using cached https://files.pythonhosted.org/packages/54/85/0bef63668fb170888c1a2970ec897d4528d6072f32dee27653381a332642/Django-2.2-py3-none-any.whl
Django requires Python '>=3.5' but the running Python is 3.4.10
Collecting sqlparse (from Django==2.2)
  Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Collecting pytz (from Django==2.2)
  Downloading https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl (510kB)
    100% |████████████████████████████████| 512kB 17.0MB/s 
Installing collected packages: sqlparse, pytz, Django
Successfully installed Django-2.2 pytz-2019.1 sqlparse-0.3.0
You are using pip version 19.0.3, however version 19.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@b30693b8f33e:/# echo $?
0

It might still be interesting to also use it in check_requires_python() though.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented May 21, 2019

I posted PR #6518 to address this.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented May 21, 2019

Out of curiosity, what does it mean to pass --ignore-requires-python without --python-version? That the Python… does not have a version? That seems a bit pointless in this case. I would expect these two to be mutually exclusive: if --python-version is passed, the value is used to check against Requires-Python (both in metadata and on index)

One issue that comes up when considering this is that pip's --python-version only permits a major-minor version (e.g. 3 or 34) for checking wheels, whereas the Requires-Python check uses major-minor-micro (e.g. (3, 6, 5)). How would pip's options work to support specifying both of these?

@cjerdonek

This comment has been minimized.

Copy link
Member

commented May 22, 2019

How would pip's options work to support specifying both of these?

Incidentally, @xavfernandez, my comment / question above relates to the review comment you made less than a week ago on one of my PR's as to whether _py_version and _py_version_info should be provided separately or as a single argument.

@xavfernandez

This comment has been minimized.

Copy link
Contributor

commented May 22, 2019

@cjerdonek Maybe we should migrate --python-version from 2, 27, 3, 38, ... values to dotted ones 2, 2.7, 3, 3.8, 3.6.4, ... allowing for the extra patch version.
During the transition phase, 2-digits numbers mn would be interpreted as m.n.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented May 22, 2019

@xavfernandez Something like that sounds good. However, the other issue is that because --python-version currently requires either "major" or "major-minor", it seems like we might still need a way to support saying whether, say, 3 or 36 should be used for the wheel-compatibility check if, say, 3.6.4 is given (for the Requires-Python check). Alternatively, maybe the Requires-Python code can be enhanced to support matching on a range of Python versions instead of just an exact major-minor-micro. It's not clear to me yet what's needed.

@xavfernandez

This comment has been minimized.

Copy link
Contributor

commented May 22, 2019

it seems like we might still need a way to support saying whether, say, 3 or 36 should be used for the wheel-compatibility check if, say, 3.6.4 is given

Wheels selection is based on the most precise/matching first, so py36 wheel should be preferred over the py3 one (all other tags being equal).
If the user wants it the other way, providing --python-version 3.0.0 might work, and if it doesn't I'd argue it seems like a very special case that we might not want to cover.

cjerdonek added a commit that referenced this issue May 23, 2019

Merge pull request #6518 from cjerdonek/issue-6371-ignore-requires-py…
…thon

Fix #6371: make pip install respect --ignore-requires-python
@cjerdonek

This comment has been minimized.

Copy link
Member

commented May 25, 2019

@cjerdonek Maybe we should migrate --python-version from 2, 27, 3, 38, ... values to dotted ones 2, 2.7, 3, 3.8, 3.6.4, ... allowing for the extra patch version.
During the transition phase, 2-digits numbers mn would be interpreted as m.n.

I just posted PR #6539 to work towards this.

@cjerdonek

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

I would expect these two to be mutually exclusive: if --python-version is passed, the value is used to check against Requires-Python (both in metadata and on index),

FYI, I just filed PR #6577, which does the above,

@lock lock bot added the S: auto-locked label Jul 6, 2019

@lock lock bot locked as resolved and limited conversation to collaborators Jul 6, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.