pex does not resolve abi3 wheels correctly

[zsimic]

pex doesn't seem to resolve abi3 wheels correctly. I have a project that depends on cryptography, which seems to have started publishing abi3 wheels on March 20 2018 (v2.2)

At first I thought this was missing wheels from cryptography (see pyca/cryptography#4404), but actually the python 3.4 abi3 wheel should cover python 3.6 for example (which I'm using).

I pex package my project like so:

pip --cache-dir dist wheel --wheel-dir dist .
pex --no-pypi --cache-dir dist --repo dist -cmyproject -omyproject.pex myproject

If I pin cryptography to 2.1.4 (last version that published non-abi3 wheels), pex packaging above works.

If I try latest cryptography version, pip wheel correctly finds cryptography-2.3.1-cp34-abi3-macosx_10_6_intel.whl, however pex complains with Could not satisfy all requirements for cryptography

My environment:

• OSX 10.13.6
• python 3.6.6
• pex 1.4.5
• pip 18.0

[clintharrison]

From what I can tell, when pex is doing dependency resolution, pep425tags.get_supported() is always called with a version argument. In this case, that'd be '36'; and the code does not reach the branch where previous minor versions are appended.

It's also somewhat unfortunate in that the abi3 tag will not be added if pex is running under py2 resolving for py3 distributions, since this code assumes it can use imp.get_suffixes() from the active interpreter.

@kwlzn et al., are there any concerns with further diverging from the pep425tags from upstream pip? I think modifying get_supported() to always expand back the version range would be the simplest solution.

[jsjeannotte]

Any update on the issue?

[piger]

Indeed by inverting the if here https://github.com/pantsbuild/pex/blob/fe405f9965dd1698095dde8241c07f8ef2a41517/pex/pep425tags.py#L262 (i.e. if version is not None) the versions list gets populated with all 3.x versions (['37', '36', '35', '34', '33', '32', '31', '30']), which allows pex to find the correct wheel for cryptography 2.5.

pex and pip end up downloading the same wheels:

pex:     Fetching file:///Users/user/.pex/build/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl: 9.0ms
pex:       Fetching file:///Users/user/.pex/build/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl: 8.1ms
pex:     Translating /private/var/folders/1v/cdy4c5112fn_vc18qvzblhxc0000gp/T/tmpfw7w1r38/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl into distribution: 8.0ms
pex:     Fetching file:///Users/user/.pex/build/asn1crypto-0.24.0-py2.py3-none-any.whl: 8.2ms
pex:       Fetching file:///Users/user/.pex/build/asn1crypto-0.24.0-py2.py3-none-any.whl: 1.3ms
pex:     Translating /private/var/folders/1v/cdy4c5112fn_vc18qvzblhxc0000gp/T/tmp4btiyiqd/asn1crypto-0.24.0-py2.py3-none-any.whl into distribution: 1.8ms
pex:     Fetching file:///Users/user/.pex/build/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl: 1.7ms
pex:       Fetching file:///Users/user/.pex/build/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl: 0.9ms
pex:     Translating /private/var/folders/1v/cdy4c5112fn_vc18qvzblhxc0000gp/T/tmpd17j7mhy/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl into distribution: 1.6ms
pex:     Fetching file:///Users/user/.pex/build/six-1.12.0-py2.py3-none-any.whl: 1.9ms
pex:       Fetching file:///Users/user/.pex/build/six-1.12.0-py2.py3-none-any.whl: 0.9ms
pex:     Translating /private/var/folders/1v/cdy4c5112fn_vc18qvzblhxc0000gp/T/tmp4ha57vdq/six-1.12.0-py2.py3-none-any.whl into distribution: 2.6ms

$ pip install -U cryptography
Collecting cryptography
  Using cached https://files.pythonhosted.org/packages/d7/9e/12bb10fd009b0146935c169cc0e1e86221eacf8dc207990d54b783c47a7d/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl
Collecting asn1crypto>=0.21.0 (from cryptography)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography)
  Using cached https://files.pythonhosted.org/packages/0b/ba/32835c9965d8a0090723e1d0b47373365525c4bd08c807b5efdc9fecbc99/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting six>=1.4.1 (from cryptography)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl

Tested on:

• OS X: 10.13.6
• pex: 1.6.1 (master branch)
• pip 19.0.1
• Python 3.7.1

[jsirois]

Thanks for debugging a bit @piger. Although your fix works - it clearly goes against what the conditional is trying to do. I think the answer is to dig and see who / why passes version. I'll dig in here since I just finished dealing with pex 1.6.x errors building cryptography in #661 / #666 so this is fresh on my mind.

[jsirois]

Note to self: making sure #492 continues to be fixed is important here.

[piger]

@jsirois I know, it was mostly to get a sense of what was going on and try to resume the conversation :) Thanks a lot for working on the other issue!

[piger]

I'm digging a bit more into how pex and pip behavior differs when it comes to install a abi3 wheel like "cryptography", in particular in those code sections that are shared between the 2 projects. I'm doing my tests on Python 3.7.1.

The wheel file that I'm trying to install is cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl (note the cp34 tag).

It seems that pip does not specify any python_version when it tries to lookup (resolve?) the wheel for a package, meaning that the "supported tags" generated by pip include older versions of python, e.g.

• ('cp34', 'abi3', 'macosx_10_11_x86_64')
• ('cp37', 'abi3', 'macosx_10_1_universal')
• ('py3', 'none', 'any')
• ('cp32', 'abi3', 'macosx_10_8_universal')

while the "supported tags" generated by pex only includes the current interpreter version, e.g. ('cp37', 'abi3', 'macosx_10_1_universal') and similar. pex gets the supported tags for a given requirement by instantiating a Platform instance.

To have pex behave more like pip in this scenario, it could resolve dependencies limiting them to a python version only when a --python-version command line option is specified.

Does this make any sense? Apologies in advance if it doesn't, but this bug is blocking a py3 porting that I'm doing, enough to trigger my will to try to help :)