"No matching distribution" due to wheel tags not matching in pip 20.0

[IRDonch]

Environment

• pip version: 20.0.1
• Python version: 3.6.9
• OS: Ubuntu 18.04

Description
Pip can seemingly no longer install any version of mxnet newer than 0.9.5.

Expected behavior
It should be able to. :-) This worked before pip 20.

How to Reproduce
Try to install mxnet==1.3.1 in a virtual environment.

Output

$ virtualenv -ppython3 /tmp/venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /tmp/venv/bin/python3
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
$ /tmp/venv/bin/pip install mxnet==1.3.1
ERROR: Could not find a version that satisfies the requirement mxnet==1.3.1 (from versions: 0.9.5)
ERROR: No matching distribution found for mxnet==1.3.1

Running pip install with --verbose produces a huge log, in which this seems relevant:

  Skipping link: none of the wheel's tags match: py2-none-manylinux1_x86_64, py3-none-manylinux1_x86_64: https://files.pythonhosted.org/packages/f0/2e/b26eb7273aed1945f59993b3b306442eb41684f931b5380821c39cf50a31/mxnet-1.3.1-py2.py3-none-manylinux1_x86_64.whl#sha256=939575fddd45e8ba39177dd3d53ccce64dea312bc08f493392b1ecace9e1b117 (from https://pypi.org/simple/mxnet/)

[otrejoso]

We are also having this error using version 20.0.1 with our in-house wheel

(venv) C:\depot\bitbucket\mytests\tests_pti>pip -vvv install C:\Users\otrejoso\Downloads\pti-2.0.510-py3-none-win_amd64.whl
Non-user install because user site-packages disabled
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-ephem-wheel-cache-wquw3si6
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Initialized build tracking at C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Entered build tracker: C:\Users\otrejoso\AppData\Local\Temp\pip-req-tracker-ik56de2r
Created temporary directory: C:\Users\otrejoso\AppData\Local\Temp\pip-install-vb0u5yy4
Cleaning up...
Removed build tracker: 'C:\\Users\\otrejoso\\AppData\\Local\\Temp\\pip-req-tracker-ik56de2r'
ERROR: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.
Exception information:
....
pip._internal.exceptions.InstallationError: pti-2.0.510-py3-none-win_amd64.whl is not a supported wheel on this platform.

Using pip install pip==19.3.1 works fine.

[jeroendecroos]

Same here with an in-house wheel.

Doesn't work:
pip install -U pip==20.0.1; pip install <wheel>
ERROR: is not a supported wheel on this platform.

Works:
pip install -U pip==19.3.1; pip install <wheel>

Seems that the platform tags are the problem here: tags 'any' are working, but this specify wheel has 'linux_x86_64'.

Note that I have:

uname -a
Linux <propretiery> 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 python -c "import wheel.pep425tags as w; print(w.get_supported())"
[('cp27', 'cp27mu', 'linux_x86_64'), ('cp27', 'none', 'linux_x86_64'), ('cp27', 'none', 'any'), ('cp2', 'none', 'any'), ('cp26', 'none', 'any'), ('cp25', 'none', 'any'), ('cp24', 'none', 'any'), ('cp23', 'none', 'any'), ('cp22', 'none', 'any'), ('cp21', 'none', 'any'), ('cp20', 'none', 'any'), ('py2', 'none', 'linux_x86_64'), ('py27', 'none', 'any'), ('py2', 'none', 'any'), ('py26', 'none', 'any'), ('py25', 'none', 'any'), ('py24', 'none', 'any'), ('py23', 'none', 'any'), ('py22', 'none', 'any'), ('py21', 'none', 'any'), ('py20', 'none', 'any')]

[chrisoro]

Same here.

19.3.1 works but 20.0.1 gives:
pip._internal.exceptions.InstallationError: pyenchant-2.0.0-py2.py3.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp33.pp35-none-win32.whl is not a supported wheel on this platform.

Tags for my pc: [('cp37', 'cp37m', 'win32'), ('cp37', 'none', 'win32'), ('cp37', 'none', 'any'), ('cp3', 'none', 'any'), ('cp36', 'none', 'any'), ('cp35', 'none', 'any'), ('cp34', 'none', 'any'), ('cp33', 'none', 'any'), ('cp32', 'none', 'any'), ('cp31', 'none', 'any'), ('cp30', 'none', 'any'), ('py3', 'none', 'win32'), ('py37', 'none', 'any'), ('py3', 'none', 'any'), ('py36', 'none', 'any'), ('py35', 'none', 'any'), ('py34', 'none', 'any'), ('py33', 'none', 'any'), ('py32', 'none', 'any'), ('py31', 'none', 'any'), ('py30', 'none', 'any')]

Tags for file can be seen in filename.

[xavfernandez]

Could you print the differences between pip debug -v in pip 20.0.1 & pip 19.3.1 ?

[IRDonch]

--- /tmp/old.txt	2020-01-21 17:22:10.221211433 +0300
+++ /tmp/new.txt	2020-01-21 17:22:30.725552363 +0300
@@ -1,4 +1,4 @@
-pip version: pip 19.3.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
+pip version: pip 20.0.1 from /tmp/venv/lib/python3.6/site-packages/pip (python 3.6)
 sys.version: 3.6.9 (default, Nov  7 2019, 10:44:02) 
 [GCC 8.3.0]
 sys.executable: /tmp/venv/bin/python3
@@ -8,7 +8,11 @@
 sys.platform: linux
 sys.implementation:
   name: cpython
-Compatible tags: 42
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): /tmp/venv/lib/python3.6/site-packages/pip/_vendor/certifi/cacert.pem
+Compatible tags: 41
   cp36-cp36m-manylinux2014_x86_64
   cp36-cp36m-manylinux2010_x86_64
   cp36-cp36m-manylinux1_x86_64
@@ -37,12 +41,11 @@
   cp32-abi3-manylinux2010_x86_64
   cp32-abi3-manylinux1_x86_64
   cp32-abi3-linux_x86_64
-  py3-none-manylinux2014_x86_64
-  py3-none-manylinux2010_x86_64
-  py3-none-manylinux1_x86_64
-  py3-none-linux_x86_64
+  py36-none-manylinux2014_x86_64
+  py36-none-manylinux2010_x86_64
+  py36-none-manylinux1_x86_64
+  py36-none-linux_x86_64
   cp36-none-any
-  cp3-none-any
   py36-none-any
   py3-none-any
   py35-none-any

[chrisoro]

-pip version: pip 19.3.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
+pip version: pip 20.0.1 from c:\sdks\python37-32\lib\site-packages\pip (python 3.7)
 sys.version: 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 18 2019, 23:46:00) [MSC v.1916 32 bit (Intel)]
 sys.executable: c:\sdks\python37-32\python.exe
 sys.getdefaultencoding: utf-8
@@ -8,14 +8,21 @@ locale.getpreferredencoding: cp1252
 sys.platform: win32
 sys.implementation:
   name: cpython
-Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
-Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
-Compatible tags: 14
+'cert' config value: global
+REQUESTS_CA_BUNDLE: None
+CURL_CA_BUNDLE: None
+pip._vendor.certifi.where(): c:\sdks\python37-32\lib\site-packages\pip\_vendor\certifi\cacert.pem
+Compatible tags: 19
   cp37-cp37m-win32
+  cp37-abi3-win32
   cp37-none-win32
-  py3-none-win32
+  cp36-abi3-win32
+  cp35-abi3-win32
+  cp34-abi3-win32
+  cp33-abi3-win32
+  cp32-abi3-win32
+  py37-none-win32
   cp37-none-any
-  cp3-none-any
   py37-none-any
   py3-none-any
   py36-none-any

[pfmoore]

Similar on Windows - the tag section of the output:

--- ".\\pip19.txt"      2020-01-21 14:30:16 +0000
+++ ".\\pip20.txt"      2020-01-21 14:26:54 +0000
@@ -1,9 +1,15 @@
-Compatible tags: 15
+Compatible tags: 21
   cp38-cp38-win_amd64
+  cp38-abi3-win_amd64
   cp38-none-win_amd64
-  py3-none-win_amd64
+  cp37-abi3-win_amd64
+  cp36-abi3-win_amd64
+  cp35-abi3-win_amd64
+  cp34-abi3-win_amd64
+  cp33-abi3-win_amd64
+  cp32-abi3-win_amd64
+  py38-none-win_amd64
   cp38-none-any
-  cp3-none-any
   py38-none-any
   py3-none-any
   py37-none-any

Looks like packaging.tags has different values than the version pip used internally in pip 19. The main difference is the lack of {py3,cp3}-none-win_amd64. Which isn't a standard tag generated by bdist_wheel AFAIK, so at least the impact will be limited to people who set custom tags.

The specs don't say much about what custom tags like this are valid, so this is arguably in the area of "undefined behaviour". While that doesn't help the people affected by this, it does suggest that being more specific in the standards would be good.

BTW, I'm not entirely sure what mxnet-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl even means - the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not? Numpy's manylinux builds have an ABI, so it doesn't appear to be a general issue with the manylinux toolchain. The tags for pyenchant also look a bit bizarre...

[IRDonch]

the MacOS releases of mxnet have a specific ABI set, why does the manylinux build not?

I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses ctypes for interop with native code, so not having an ABI makes sense.

[CeadeS]

Have the same issue installing icc-rt (from intel-numpy) (2020.0.133) using pip==20.0.1

[pfmoore]

I briefly checked the Linux package, and it appears that none of the native libraries there refer to Python symbols. It looks like MXNet uses ctypes for interop with native code, so not having an ABI makes sense.

OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

At a minimum, this sounds like it should be raised as an issue against the packaging library. Regardless of what pip does, if these are valid tags they should be supported in packaging.tags, and a general discussion of what tags should be supported is probably better had there than here.

[IRDonch]

OK. Why does it need a "manylinux" tag in that case, if it's using ctypes for everything? Actually, don't spend any time on that question, I'm not a Linux expert so I probably wouldn't follow the answer anyway.

I will answer anyway: the wheel has native Linux libraries in it, so the manylinux1 tag makes sense.

[jamadden]

In #7620 (comment) @tomasaschan reported what I think is this same issue for xgboost, which ships as xgboost-0.90-py2.py3-none-manylinux1_x86_64.whl. It also appears to contain native libs, perhaps for the JVM.

[pfmoore]

@IRDonch Thanks. I did actually follow that explanation 🙂 Makes sense.

[pfmoore]

@jamadden Agreed, that looks like the same issue.