Tag macosx_10_9_universal2 not accepted when running native arm64 (on macOS 11) #379

rluce · 2020-12-30T09:33:56Z

Environment

MacBook Pro with M1 chip
Released Python 3.9.1 universal2 installer
Latest pip, setuptools and wheel

Description

Running the arm64 portion of Python 3.9.1 to install a universal2 binary wheel ("macosx_10_9_universal2" is the default tag) fails. On contrary, using the x86_64 portion of the very same binary the installation succeeds. For me the expected behaviour is that such wheels can be installed with both the arm64 and x86_64 portions on macOS 11.

The offending portion that pip rejects is the 10_9 part. If I manually rename this portion of the wheel file to 11_0, the installation succeeds. Again, for me the expected behaviour is that such wheels can be installed on macOS 11

What did you want to do?

Install a universal2 binary wheel on macOS 11 using Python 3.9.1. Simplified setup using a dummy wheel:

% touch asdf-1.0.0-cp39-cp39-macosx_10_9_universal2.whl 
% python -m pip install asdf-1.0.0-cp39-cp39-macosx_10_9_universal2.whl

Output

ERROR: asdf-1.0.0-cp39-cp39-macosx_10_9_universal2.whl is not a supported wheel on this platform.

Additional information

Using the x86_64 portion, the wheel tag is accepted (and we get the expected error from using a dummy wheel):

% arch -x86_64 python -m pip install asdf-1.0.0-cp39-cp39-macosx_10_9_universal2.whl
Processing ./asdf-1.0.0-cp39-cp39-macosx_10_9_universal2.whl
[ ... ]
zipfile.BadZipFile: File is not a zip file

Version of the interpreter, showing that the system platform matches the wheel tag:

(Python 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:44:01) 
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig; sysconfig.get_platform()
'macosx-10.9-universal2'

Version of installed packages:

pip        21.0.dev0
setuptools 51.1.1
wheel      0.36.2

The text was updated successfully, but these errors were encountered:

rluce · 2020-12-30T09:42:36Z

I think PR pypa/wheel#390 is related, but I'm not sure to which extend. Maybe @ronaldoussoren knows off the top of his head what is going on here?

uranusjr · 2020-12-30T13:29:49Z

pypa/wheel#390 is for creating wheels, so this is not related.

Can you try what this prints for you?

from pip._vendor.packaging.tags import cpython_tags
for t in cpython_tags(): print(str(t))

You may want to report this to https://github.com/pypa/packaging instead (which pip uses underneath, and this error is emitted if pip detects a tag not recognisable by the library); you’d more easily get attention from people with relevant knowledge there.

pradyunsg · 2020-12-30T13:35:58Z

Could you also provide the output of pip debug --verbose on your system?

rluce · 2020-12-30T13:48:02Z

Can you try what this prints for you?

from pip._vendor.packaging.tags import cpython_tags
for t in cpython_tags(): print(str(t))

>>> from pip._vendor.packaging.tags import cpython_tags
>>> for t in cpython_tags(): print(str(t))
... 
cp39-cp39-macosx_11_0_arm64
cp39-cp39-macosx_11_0_universal2
cp39-abi3-macosx_11_0_arm64
cp39-abi3-macosx_11_0_universal2
cp39-none-macosx_11_0_arm64
cp39-none-macosx_11_0_universal2
cp38-abi3-macosx_11_0_arm64
cp38-abi3-macosx_11_0_universal2
cp37-abi3-macosx_11_0_arm64
cp37-abi3-macosx_11_0_universal2
cp36-abi3-macosx_11_0_arm64
cp36-abi3-macosx_11_0_universal2
cp35-abi3-macosx_11_0_arm64
cp35-abi3-macosx_11_0_universal2
cp34-abi3-macosx_11_0_arm64
cp34-abi3-macosx_11_0_universal2
cp33-abi3-macosx_11_0_arm64
cp33-abi3-macosx_11_0_universal2
cp32-abi3-macosx_11_0_arm64
cp32-abi3-macosx_11_0_universal2

Could you also provide the output of pip debug --verbose on your system?

% /usr/local/bin/pip3.9 debug --verbose
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 20.3.3 from /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pip (python 3.9)
sys.version: 3.9.1 (v3.9.1:1e5d33e9b9, Dec  7 2020, 12:44:01) 
[Clang 12.0.0 (clang-1200.0.32.27)]
sys.executable: /usr/local/bin/python3.9
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: UTF-8
sys.platform: darwin
sys.implementation:
  name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pip/_vendor/certifi/cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
  appdirs==1.4.4
  CacheControl==0.12.6
  colorama==0.4.4
  contextlib2==0.6.0.post1 (Unable to locate actual module version, using vendor.txt specified version)
  distlib==0.3.1
  distro==1.5.0 (Unable to locate actual module version, using vendor.txt specified version)
  html5lib==1.1
  ipaddress==1.0.23
  msgpack==1.0.0 (Unable to locate actual module version, using vendor.txt specified version)
  packaging==20.8
  pep517==0.9.1
  progress==1.5
  pyparsing==2.4.7
  requests==2.25.0
  certifi==2020.11.08
  chardet==3.0.4
  idna==2.10
  urllib3==1.26.2
  resolvelib==0.5.3
  retrying==1.3.3 (Unable to locate actual module version, using vendor.txt specified version)
  setuptools==44.0.0 (Unable to locate actual module version, using vendor.txt specified version)
  six==1.15.0
  toml==0.10.2
  webencodings==0.5.1 (Unable to locate actual module version, using vendor.txt specified version)
Compatible tags: 54
  cp39-cp39-macosx_11_0_arm64
  cp39-cp39-macosx_11_0_universal2
  cp39-abi3-macosx_11_0_arm64
  cp39-abi3-macosx_11_0_universal2
  cp39-none-macosx_11_0_arm64
  cp39-none-macosx_11_0_universal2
  cp38-abi3-macosx_11_0_arm64
  cp38-abi3-macosx_11_0_universal2
  cp37-abi3-macosx_11_0_arm64
  cp37-abi3-macosx_11_0_universal2
  cp36-abi3-macosx_11_0_arm64
  cp36-abi3-macosx_11_0_universal2
  cp35-abi3-macosx_11_0_arm64
  cp35-abi3-macosx_11_0_universal2
  cp34-abi3-macosx_11_0_arm64
  cp34-abi3-macosx_11_0_universal2
  cp33-abi3-macosx_11_0_arm64
  cp33-abi3-macosx_11_0_universal2
  cp32-abi3-macosx_11_0_arm64
  cp32-abi3-macosx_11_0_universal2
  py39-none-macosx_11_0_arm64
  py39-none-macosx_11_0_universal2
  py3-none-macosx_11_0_arm64
  py3-none-macosx_11_0_universal2
  py38-none-macosx_11_0_arm64
  py38-none-macosx_11_0_universal2
  py37-none-macosx_11_0_arm64
  py37-none-macosx_11_0_universal2
  py36-none-macosx_11_0_arm64
  py36-none-macosx_11_0_universal2
  py35-none-macosx_11_0_arm64
  py35-none-macosx_11_0_universal2
  py34-none-macosx_11_0_arm64
  py34-none-macosx_11_0_universal2
  py33-none-macosx_11_0_arm64
  py33-none-macosx_11_0_universal2
  py32-none-macosx_11_0_arm64
  py32-none-macosx_11_0_universal2
  py31-none-macosx_11_0_arm64
  py31-none-macosx_11_0_universal2
  py30-none-macosx_11_0_arm64
  py30-none-macosx_11_0_universal2
  cp39-none-any
  py39-none-any
  py3-none-any
  py38-none-any
  py37-none-any
  py36-none-any
  py35-none-any
  py34-none-any
  py33-none-any
  py32-none-any
  py31-none-any
  py30-none-any

I'll ping pypa/packaging then, thanks!

rluce · 2020-12-30T13:52:35Z

Actually if anyone with write permissions on pypa/packaging would transfer this issue, we could skip the duplicate. I'll wait an hour or so now before opening another issue over there.

pradyunsg · 2020-12-30T13:56:46Z

How are you getting the 10_9 tag for the universal2 wheel?

rluce · 2020-12-30T14:02:02Z

For the actual wheel I was looking at: I was building it through the standard toolchain python setup.py bdist_wheel, using the exact same Python interpreter on the exact same macOS 11 machine. It would take me a while to construct a minimal source example though.

rluce · 2020-12-30T14:03:58Z

That said, getting the 10_9 platform tag was my expectation -- I wanted a binary wheel whose x86_64 portion would be compatible down to macOS 10.9 and whose arm64 portion would run on all supported versions too (which currently only is 11.X...)

ronaldoussoren · 2020-12-30T14:14:32Z

How are you getting the 10_9 tag for the universal2 wheel?

Using a recentish version of wheel and the universal2 build of CPython. I don't have a link to the relevant PR at hand, but the basic idea is that a "macosx_10_9_universal2" wheel will work on all mac systems with an x86_64 or arm64 CPU running macOS 10.9 or later.

pradyunsg · 2020-12-30T14:34:37Z

Ah.

No one has updated packaging.tags for this change in CPython yet, so I'd request someone familiar with the situation to go ahead and file a PR for this. :)

ronaldoussoren · 2020-12-30T16:28:13Z

No one has updated packaging.tags for this change in CPython yet, so I'd request someone familiar with the situation to go ahead and file a PR for this. :)

I would have expected that #319 would have done the trick, but apparently not. I just received my M1 system and am currently geeking on on configuring it.

What I don't quite understand is that pip install pyobjc-core tries to build from source, while pip install --only-binary :all: pyobjc-core does use the universal2 binary wheel, but only the "macosx_11_0_universal2" wheel for PyObjC 7.0.1 and not the "macosx_10_9_universal2" one for PyObjC 7.1.

I guess its time to clone the repo and try to understand what's going on here.

pradyunsg · 2020-12-30T16:44:59Z

We're basically using the tags from this package as-is. Running pip debug -v will print the list of tags that pip tries to use with wheels.

ronaldoussoren · 2020-12-30T16:48:40Z

pip debug -v shows that pip only looks at 11.0 wheels (as was to be expected given this issue).

ronaldoussoren · 2020-12-31T11:00:09Z

The attached PR should fix the issue. When I replace the copy of packaging in pip._vendor by the patched version in the PR "pip debug -v" shows the expected output and "pip install pyobjc-core" uses the universal2 wheel to install.

henryiii · 2021-01-01T02:05:41Z

Independently discovered by @joerick in pypa/cibuildwheel#484 as well.

pradyunsg transferred this issue from pypa/pip Dec 30, 2020

pradyunsg added the packaging.tags label Dec 30, 2020

ronaldoussoren mentioned this issue Dec 31, 2020

Support "macosx_10_9_universal2" wheels #380

Merged

ronaldoussoren mentioned this issue Jan 1, 2021

Universal2 wheels on macOS pypa/cibuildwheel#484

Merged

7 tasks

brettcannon closed this as completed in #380 Jan 4, 2021

henryiii mentioned this issue Jan 24, 2021

Pip cannot download macosx_10_9_universal2 from Apple Silicon pypa/pip#9506

Closed

pradyunsg mentioned this issue Jan 13, 2022

packaging does not know that macOS 10.16 and 11.0 are the same #497

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tag macosx_10_9_universal2 not accepted when running native arm64 (on macOS 11) #379

Tag macosx_10_9_universal2 not accepted when running native arm64 (on macOS 11) #379

rluce commented Dec 30, 2020

rluce commented Dec 30, 2020

uranusjr commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

rluce commented Dec 30, 2020

rluce commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

rluce commented Dec 30, 2020 •

edited

Loading

rluce commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

ronaldoussoren commented Dec 31, 2020

henryiii commented Jan 1, 2021

Tag macosx_10_9_universal2 not accepted when running native arm64 (on macOS 11) #379

Tag macosx_10_9_universal2 not accepted when running native arm64 (on macOS 11) #379

Comments

rluce commented Dec 30, 2020

rluce commented Dec 30, 2020

uranusjr commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

rluce commented Dec 30, 2020

rluce commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

rluce commented Dec 30, 2020 • edited Loading

rluce commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

pradyunsg commented Dec 30, 2020

ronaldoussoren commented Dec 30, 2020

ronaldoussoren commented Dec 31, 2020

henryiii commented Jan 1, 2021

rluce commented Dec 30, 2020 •

edited

Loading