Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pex does not download foreign abi3 wheels correctly #823

Closed
jsirois opened this issue Dec 10, 2019 · 1 comment · Fixed by #835
Closed

Pex does not download foreign abi3 wheels correctly #823

jsirois opened this issue Dec 10, 2019 · 1 comment · Fixed by #835
Assignees
@jsirois
Labels
bug resolver

Comments

@jsirois
Copy link
Member

jsirois commented Dec 10, 2019

A similar looking issue was encountered in #539 and fixed in #692, although that was for the Pex runtime resolve case; i.e.: resolving wheels stored inside the PEX at runtime. With the switch from the pex resolver to pip, we have a similar issue for downloads of foreign platforms. In particular, passing pip an explicit foreign platform, say: --platform macosx_10_11_x86_64 --implementation cp --python-version 36 --abi cp36m, we get the following calculation of supported tags:

$ PYTHONPATH=$PWD/pex/vendor/_vendored/pip python -mpip -vvv --disable-pip-version-check --isolated --exists-action i --no-cache-dir download --dest /tmp/tmp7991z44v/resolved_dists/macosx_10_11_x86_64-cp-36-cp36m --no-index --find-links /home/jsirois/dev/pantsbuild/pants/dist/deploy/wheels/pantsbuild.pants/879df20df1df1b9cf16e29cb4a2836f4831f23d9/1.24.0.dev1+git879df20d --find-links /home/jsirois/dev/pantsbuild/pants/dist/deploy/wheels/3rdparty/879df20df1df1b9cf16e29cb4a2836f4831f23d9/1.24.0.dev1+git879df20d --platform macosx_10_11_x86_64 --implementation cp --python-version 36 --abi cp36m --only-binary :all: cryptography>=1.9
>>> calculating supported for versions=36, noarch=False, platform=macosx_10_11_x86_64, impl=cp, abi=cp36m:...

	('cp36', 'cp36m', 'macosx_10_11_x86_64')
	('cp36', 'cp36m', 'macosx_10_11_intel')
	('cp36', 'cp36m', 'macosx_10_11_fat64')
	('cp36', 'cp36m', 'macosx_10_11_fat32')
	('cp36', 'cp36m', 'macosx_10_11_universal')
	('cp36', 'cp36m', 'macosx_10_10_x86_64')
	('cp36', 'cp36m', 'macosx_10_10_intel')
	('cp36', 'cp36m', 'macosx_10_10_fat64')
	('cp36', 'cp36m', 'macosx_10_10_fat32')
	('cp36', 'cp36m', 'macosx_10_10_universal')
	('cp36', 'cp36m', 'macosx_10_9_x86_64')
	('cp36', 'cp36m', 'macosx_10_9_intel')
	('cp36', 'cp36m', 'macosx_10_9_fat64')
	('cp36', 'cp36m', 'macosx_10_9_fat32')
	('cp36', 'cp36m', 'macosx_10_9_universal')
	('cp36', 'cp36m', 'macosx_10_8_x86_64')
	('cp36', 'cp36m', 'macosx_10_8_intel')
	('cp36', 'cp36m', 'macosx_10_8_fat64')
	('cp36', 'cp36m', 'macosx_10_8_fat32')
	('cp36', 'cp36m', 'macosx_10_8_universal')
	('cp36', 'cp36m', 'macosx_10_7_x86_64')
	('cp36', 'cp36m', 'macosx_10_7_intel')
	('cp36', 'cp36m', 'macosx_10_7_fat64')
	('cp36', 'cp36m', 'macosx_10_7_fat32')
	('cp36', 'cp36m', 'macosx_10_7_universal')
	('cp36', 'cp36m', 'macosx_10_6_x86_64')
	('cp36', 'cp36m', 'macosx_10_6_intel')
	('cp36', 'cp36m', 'macosx_10_6_fat64')
	('cp36', 'cp36m', 'macosx_10_6_fat32')
	('cp36', 'cp36m', 'macosx_10_6_universal')
	('cp36', 'cp36m', 'macosx_10_5_x86_64')
	('cp36', 'cp36m', 'macosx_10_5_intel')
	('cp36', 'cp36m', 'macosx_10_5_fat64')
	('cp36', 'cp36m', 'macosx_10_5_fat32')
	('cp36', 'cp36m', 'macosx_10_5_universal')
	('cp36', 'cp36m', 'macosx_10_4_intel')
	('cp36', 'cp36m', 'macosx_10_4_fat32')
	('cp36', 'cp36m', 'macosx_10_4_universal')
	('cp36', 'cp36m', 'macosx_10_3_fat32')
	('cp36', 'cp36m', 'macosx_10_3_universal')
	('cp36', 'cp36m', 'macosx_10_2_fat32')
	('cp36', 'cp36m', 'macosx_10_2_universal')
	('cp36', 'cp36m', 'macosx_10_1_fat32')
	('cp36', 'cp36m', 'macosx_10_1_universal')
	('cp36', 'cp36m', 'macosx_10_0_fat32')
	('cp36', 'cp36m', 'macosx_10_0_universal')
	('cp36', 'abi3', 'macosx_10_11_x86_64')
	('cp36', 'abi3', 'macosx_10_11_intel')
	('cp36', 'abi3', 'macosx_10_11_fat64')
	('cp36', 'abi3', 'macosx_10_11_fat32')
	('cp36', 'abi3', 'macosx_10_11_universal')
	('cp36', 'abi3', 'macosx_10_10_x86_64')
	('cp36', 'abi3', 'macosx_10_10_intel')
	('cp36', 'abi3', 'macosx_10_10_fat64')
	('cp36', 'abi3', 'macosx_10_10_fat32')
	('cp36', 'abi3', 'macosx_10_10_universal')
	('cp36', 'abi3', 'macosx_10_9_x86_64')
	('cp36', 'abi3', 'macosx_10_9_intel')
	('cp36', 'abi3', 'macosx_10_9_fat64')
	('cp36', 'abi3', 'macosx_10_9_fat32')
	('cp36', 'abi3', 'macosx_10_9_universal')
	('cp36', 'abi3', 'macosx_10_8_x86_64')
	('cp36', 'abi3', 'macosx_10_8_intel')
	('cp36', 'abi3', 'macosx_10_8_fat64')
	('cp36', 'abi3', 'macosx_10_8_fat32')
	('cp36', 'abi3', 'macosx_10_8_universal')
	('cp36', 'abi3', 'macosx_10_7_x86_64')
	('cp36', 'abi3', 'macosx_10_7_intel')
	('cp36', 'abi3', 'macosx_10_7_fat64')
	('cp36', 'abi3', 'macosx_10_7_fat32')
	('cp36', 'abi3', 'macosx_10_7_universal')
	('cp36', 'abi3', 'macosx_10_6_x86_64')
	('cp36', 'abi3', 'macosx_10_6_intel')
	('cp36', 'abi3', 'macosx_10_6_fat64')
	('cp36', 'abi3', 'macosx_10_6_fat32')
	('cp36', 'abi3', 'macosx_10_6_universal')
	('cp36', 'abi3', 'macosx_10_5_x86_64')
	('cp36', 'abi3', 'macosx_10_5_intel')
	('cp36', 'abi3', 'macosx_10_5_fat64')
	('cp36', 'abi3', 'macosx_10_5_fat32')
	('cp36', 'abi3', 'macosx_10_5_universal')
	('cp36', 'abi3', 'macosx_10_4_intel')
	('cp36', 'abi3', 'macosx_10_4_fat32')
	('cp36', 'abi3', 'macosx_10_4_universal')
	('cp36', 'abi3', 'macosx_10_3_fat32')
	('cp36', 'abi3', 'macosx_10_3_universal')
	('cp36', 'abi3', 'macosx_10_2_fat32')
	('cp36', 'abi3', 'macosx_10_2_universal')
	('cp36', 'abi3', 'macosx_10_1_fat32')
	('cp36', 'abi3', 'macosx_10_1_universal')
	('cp36', 'abi3', 'macosx_10_0_fat32')
	('cp36', 'abi3', 'macosx_10_0_universal')
	('cp36', 'none', 'macosx_10_11_x86_64')
	('cp36', 'none', 'macosx_10_11_intel')
	('cp36', 'none', 'macosx_10_11_fat64')
	('cp36', 'none', 'macosx_10_11_fat32')
	('cp36', 'none', 'macosx_10_11_universal')
	('cp36', 'none', 'macosx_10_10_x86_64')
	('cp36', 'none', 'macosx_10_10_intel')
	('cp36', 'none', 'macosx_10_10_fat64')
	('cp36', 'none', 'macosx_10_10_fat32')
	('cp36', 'none', 'macosx_10_10_universal')
	('cp36', 'none', 'macosx_10_9_x86_64')
	('cp36', 'none', 'macosx_10_9_intel')
	('cp36', 'none', 'macosx_10_9_fat64')
	('cp36', 'none', 'macosx_10_9_fat32')
	('cp36', 'none', 'macosx_10_9_universal')
	('cp36', 'none', 'macosx_10_8_x86_64')
	('cp36', 'none', 'macosx_10_8_intel')
	('cp36', 'none', 'macosx_10_8_fat64')
	('cp36', 'none', 'macosx_10_8_fat32')
	('cp36', 'none', 'macosx_10_8_universal')
	('cp36', 'none', 'macosx_10_7_x86_64')
	('cp36', 'none', 'macosx_10_7_intel')
	('cp36', 'none', 'macosx_10_7_fat64')
	('cp36', 'none', 'macosx_10_7_fat32')
	('cp36', 'none', 'macosx_10_7_universal')
	('cp36', 'none', 'macosx_10_6_x86_64')
	('cp36', 'none', 'macosx_10_6_intel')
	('cp36', 'none', 'macosx_10_6_fat64')
	('cp36', 'none', 'macosx_10_6_fat32')
	('cp36', 'none', 'macosx_10_6_universal')
	('cp36', 'none', 'macosx_10_5_x86_64')
	('cp36', 'none', 'macosx_10_5_intel')
	('cp36', 'none', 'macosx_10_5_fat64')
	('cp36', 'none', 'macosx_10_5_fat32')
	('cp36', 'none', 'macosx_10_5_universal')
	('cp36', 'none', 'macosx_10_4_intel')
	('cp36', 'none', 'macosx_10_4_fat32')
	('cp36', 'none', 'macosx_10_4_universal')
	('cp36', 'none', 'macosx_10_3_fat32')
	('cp36', 'none', 'macosx_10_3_universal')
	('cp36', 'none', 'macosx_10_2_fat32')
	('cp36', 'none', 'macosx_10_2_universal')
	('cp36', 'none', 'macosx_10_1_fat32')
	('cp36', 'none', 'macosx_10_1_universal')
	('cp36', 'none', 'macosx_10_0_fat32')
	('cp36', 'none', 'macosx_10_0_universal')
	('py3', 'none', 'macosx_10_11_x86_64')
	('py3', 'none', 'macosx_10_11_intel')
	('py3', 'none', 'macosx_10_11_fat64')
	('py3', 'none', 'macosx_10_11_fat32')
	('py3', 'none', 'macosx_10_11_universal')
	('py3', 'none', 'macosx_10_10_x86_64')
	('py3', 'none', 'macosx_10_10_intel')
	('py3', 'none', 'macosx_10_10_fat64')
	('py3', 'none', 'macosx_10_10_fat32')
	('py3', 'none', 'macosx_10_10_universal')
	('py3', 'none', 'macosx_10_9_x86_64')
	('py3', 'none', 'macosx_10_9_intel')
	('py3', 'none', 'macosx_10_9_fat64')
	('py3', 'none', 'macosx_10_9_fat32')
	('py3', 'none', 'macosx_10_9_universal')
	('py3', 'none', 'macosx_10_8_x86_64')
	('py3', 'none', 'macosx_10_8_intel')
	('py3', 'none', 'macosx_10_8_fat64')
	('py3', 'none', 'macosx_10_8_fat32')
	('py3', 'none', 'macosx_10_8_universal')
	('py3', 'none', 'macosx_10_7_x86_64')
	('py3', 'none', 'macosx_10_7_intel')
	('py3', 'none', 'macosx_10_7_fat64')
	('py3', 'none', 'macosx_10_7_fat32')
	('py3', 'none', 'macosx_10_7_universal')
	('py3', 'none', 'macosx_10_6_x86_64')
	('py3', 'none', 'macosx_10_6_intel')
	('py3', 'none', 'macosx_10_6_fat64')
	('py3', 'none', 'macosx_10_6_fat32')
	('py3', 'none', 'macosx_10_6_universal')
	('py3', 'none', 'macosx_10_5_x86_64')
	('py3', 'none', 'macosx_10_5_intel')
	('py3', 'none', 'macosx_10_5_fat64')
	('py3', 'none', 'macosx_10_5_fat32')
	('py3', 'none', 'macosx_10_5_universal')
	('py3', 'none', 'macosx_10_4_intel')
	('py3', 'none', 'macosx_10_4_fat32')
	('py3', 'none', 'macosx_10_4_universal')
	('py3', 'none', 'macosx_10_3_fat32')
	('py3', 'none', 'macosx_10_3_universal')
	('py3', 'none', 'macosx_10_2_fat32')
	('py3', 'none', 'macosx_10_2_universal')
	('py3', 'none', 'macosx_10_1_fat32')
	('py3', 'none', 'macosx_10_1_universal')
	('py3', 'none', 'macosx_10_0_fat32')
	('py3', 'none', 'macosx_10_0_universal')
	('cp36', 'none', 'any')
	('cp3', 'none', 'any')
	('py36', 'none', 'any')
	('py3', 'none', 'any')
ERROR: Could not find a version that satisfies the requirement cryptography (from versions: none)
ERROR: No matching distribution found for cryptography
@jsirois jsirois mentioned this issue Dec 10, 2019
Closed
@jsirois jsirois self-assigned this Dec 10, 2019
@jsirois
Copy link
Member Author

jsirois commented Dec 21, 2019

Noting that this will be fixed in pip once it re-works its pep425tags.py to use pypa/packaging tags: pypa/packaging#231

jsirois added a commit to jsirois/pex that referenced this issue Dec 26, 2019
@jsirois
Use pypa/packaging.
e13cb9c 
Vendoring pypa/packaging allows us to eliminate our copies of
`pep425tags.py` and `glibc.py` from pip as well as eliminate custom
environment marker setup code. We accomplish the latter by re-working
the responsibility split between `PythonIdentity` and `Platform`.

`Platform` now just handles parsing extended platform specifiers and
all capability testing is moved into `PythonIdentity` which consults
`packaging` at runtime to determine the compatible tag set and
appropriate environment markers.

Work towards pex-tool#823
Fixes pex-tool#696
@jsirois jsirois mentioned this issue Dec 26, 2019
Merged
jsirois added a commit to jsirois/pex that referenced this issue Dec 26, 2019
@jsirois
Support VCS urls for vendoring.
03c2d21 
This will support using a patched version of pip pending acceptance
of patches upstream.

Work towards pex-tool#823.
@jsirois jsirois mentioned this issue Dec 26, 2019
Merged
jsirois added a commit to jsirois/pex that referenced this issue Dec 26, 2019
@jsirois
Support VCS urls for vendoring.
626093a 
This will support using a patched version of pip pending acceptance
of patches upstream.

Work towards pex-tool#823.
This was referenced Dec 27, 2019
Merged
Merged
Merged
jsirois added a commit to jsirois/pex that referenced this issue Dec 27, 2019
@jsirois
Fix pex resolving for foreign platforms.
d1ac2be 
This change restores manylinux handling to the pre pex 2.0.0 status quo
while properly supporting manylinux-next via the new `--manylinux`
option that takes a manylinux platform upper bound to assume whenever
the user specified platform is linux. This upper bound now defaults to
manylinux2014.

The change also brings in a pip fix for abi3 handling for foreign
platforms and a test is added to track this continues to work.

Fixes pex-tool#823
@jsirois jsirois mentioned this issue Dec 27, 2019
Merged
jsirois added a commit that referenced this issue Dec 27, 2019
@jsirois
Support VCS urls for vendoring. (#834)
89efbfb 
This will support using a patched version of pip pending acceptance
of patches upstream.

Work towards #823.
jsirois added a commit to jsirois/pex that referenced this issue Dec 27, 2019
@jsirois
Use pypa/packaging.
150f9a9 
Vendoring pypa/packaging allows us to eliminate our copies of
`pep425tags.py` and `glibc.py` from pip as well as eliminate custom
environment marker setup code. We accomplish the latter by re-working
the responsibility split between `PythonIdentity` and `Platform`.

`Platform` now just handles parsing extended platform specifiers and
all capability testing is moved into `PythonIdentity` which consults
`packaging` at runtime to determine the compatible tag set and
appropriate environment markers.

Work towards pex-tool#823
Fixes pex-tool#696
jsirois added a commit that referenced this issue Dec 27, 2019
@jsirois
Use pypa/packaging. (#831)
2a8b950 
Vendoring pypa/packaging allows us to eliminate our copies of
`pep425tags.py` and `glibc.py` from pip as well as eliminate custom
environment marker setup code. We accomplish the latter by re-working
the responsibility split between `PythonIdentity` and `Platform`.

`Platform` now just handles parsing extended platform specifiers and
all capability testing is moved into `PythonIdentity` which consults
`packaging` at runtime to determine the compatible tag set and
appropriate environment markers.

Work towards #823
Fixes #696
jsirois added a commit to jsirois/pex that referenced this issue Dec 28, 2019
@jsirois
Fix pex resolving for foreign platforms.
52b1a37 
This change restores manylinux handling to the pre pex 2.0.0 status quo
while properly supporting manylinux-next via the new `--manylinux`
option that takes a manylinux platform upper bound to assume whenever
the user specified platform is linux. This upper bound now defaults to
manylinux2014.

The change also brings in a pip fix for abi3 handling for foreign
platforms and a test is added to track this continues to work.

Fixes pex-tool#823
jsirois added a commit that referenced this issue Dec 28, 2019
@jsirois
Fix pex resolving for foreign platforms. (#835)
e078b88 
This change restores manylinux handling to the pre pex 2.0.0 status quo
while properly supporting manylinux-next via the new `--manylinux`
option that takes a manylinux platform upper bound to assume whenever
the user specified foreign platform is linux.
This upper bound now defaults to manylinux2014.

The change also brings in a pip fix for abi3 handling for foreign
platforms and a test is added to track this continues to work.

Fixes #823
@jsirois jsirois mentioned this issue Dec 28, 2019
Closed
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsirois jsirois

Labels
bug resolver
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix pex resolving for foreign platforms. jsirois/pex
1 participant
@jsirois