Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fail to install by pip on macOS 11.0.1 Big Sur #78

Closed
applearound opened this issue Nov 30, 2020 · 11 comments
Closed

Fail to install by pip on macOS 11.0.1 Big Sur #78

applearound opened this issue Nov 30, 2020 · 11 comments

Comments

@applearound
Copy link

I tried to install jupyterlab which depends argon2-cffi, but failed at building argon2-cffi.

System:
image

Python version:

> python3 --version
Python 3.8.2

pip version:

> pip3 --version
pip 19.2.3 from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8)

Toolchains:

> clang --version
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: x86_64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Command and Log:

> pip3 install argon2-cffi --user
Collecting argon2-cffi
  Using cached https://files.pythonhosted.org/packages/74/fd/d78e003a79c453e8454197092fce9d1c6099445b7e7da0b04eb4fe1dbab7/argon2-cffi-20.1.0.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: six in /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (from argon2-cffi) (1.15.0)
Collecting cffi>=1.0.0 (from argon2-cffi)
  Using cached https://files.pythonhosted.org/packages/66/6a/98e023b3d11537a5521902ac6b50db470c826c682be6a8c661549cb7717a/cffi-1.14.4.tar.gz
Collecting pycparser (from cffi>=1.0.0->argon2-cffi)
  Using cached https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl
Building wheels for collected packages: argon2-cffi
  Building wheel for argon2-cffi (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Developer/CommandLineTools/usr/bin/python3 /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/j_/c2k540gj5ts_zyfz7__f76yr0000gn/T/tmp45e__06d
       cwd: /private/var/folders/j_/c2k540gj5ts_zyfz7__f76yr0000gn/T/pip-install-db4uhggy/argon2-cffi
  Complete output (90 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.14.6-x86_64-3.8
  creating build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/low_level.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/_ffi_build.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/_password_hasher.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/exceptions.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/_legacy.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/__main__.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  copying src/argon2/_utils.py -> build/lib.macosx-10.14.6-x86_64-3.8/argon2
  running build_clib
  building 'argon2' library
  creating build/temp.macosx-10.14.6-x86_64-3.8
  creating build/temp.macosx-10.14.6-x86_64-3.8/extras
  creating build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2
  creating build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src
  creating build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/blake2
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -msse2 -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/argon2.c -o build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/argon2.o
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -msse2 -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/blake2/blake2b.c -o build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/blake2/blake2b.o
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -msse2 -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/core.c -o build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/core.o
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -msse2 -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/encoding.c -o build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/encoding.o
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -msse2 -Iextras/libargon2/src/../include -Iextras/libargon2/src/blake2 -c extras/libargon2/src/opt.c -o build/temp.macosx-10.14.6-x86_64-3.8/extras/libargon2/src/opt.o
  In file included from extras/libargon2/src/opt.c:26:
  In file included from extras/libargon2/src/blake2/blamka-round-opt.h:23:
  In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/emmintrin.h:13:
  In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/xmmintrin.h:13:
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:50:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_vec_init_v2si(__i, 0);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:129:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_packsswb((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:159:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_packssdw((__v2si)__m1, (__v2si)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:189:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_packuswb((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:216:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpckhbw((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:239:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpckhwd((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:260:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpckhdq((__v2si)__m1, (__v2si)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:287:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpcklbw((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:310:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpcklwd((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:331:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_punpckldq((__v2si)__m1, (__v2si)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:352:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddb((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:373:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddw((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:394:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddd((__v2si)__m1, (__v2si)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:416:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddsb((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:439:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddsw((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:461:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddusb((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:483:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_paddusw((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:504:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_psubb((__v8qi)__m1, (__v8qi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/mmintrin.h:525:12: error: invalid conversion between vector type '__m64' (vector of 1 'long long' value) and integer type 'int' of different size
      return (__m64)__builtin_ia32_psubw((__v4hi)__m1, (__v4hi)__m2);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  20 errors generated.
  error: command 'clang' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for argon2-cffi
  Running setup.py clean for argon2-cffi
Failed to build argon2-cffi
Building wheels for collected packages: cffi
  Building wheel for cffi (setup.py) ... done
  Created wheel for cffi: filename=cffi-1.14.4-cp38-cp38-macosx_10_14_6_x86_64.whl size=258021 sha256=acbb7806e54c30d92c413b20f8dda619fbee180b965ab5727803b524150bb0a7
  Stored in directory: /Users/yezhou/Library/Caches/pip/wheels/0b/5a/01/5220481b9a4717ba183b738c9ee7d773ad32548555a339345a
Successfully built cffi
ERROR: Could not build wheels for argon2-cffi which use PEP 517 and cannot be installed directly

> pip3 install argon2-cffi --user --no-use-pep517
Collecting argon2-cffi
  Using cached https://files.pythonhosted.org/packages/74/fd/d78e003a79c453e8454197092fce9d1c6099445b7e7da0b04eb4fe1dbab7/argon2-cffi-20.1.0.tar.gz
ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml
@hynek
Copy link
Owner

hynek commented Nov 30, 2020

I'm afraid the bigger problem here is that pip doesn't use wheels on Big Sur yet because of the version bump to 11.0: pypa/pip#9138

It looks like an upcoming pip release should fix that and a (ugly) workaround is mentioned in the linked ticket.

As for the build problem that's gonna be a fun one, because cursory googling unveiled that blake2 currenly fails to compile on latest clangs and I don't see any fixes on neither the argon2 nor the blake repos.

@hynek
Copy link
Owner

hynek commented Nov 30, 2020

I should note though, that I can compile it with the same clang – and supposedly CLI tools version.

Would you mind trying to work through https://github.com/hynek/argon2-cffi/blob/master/.github/CONTRIBUTING.rst#local-development-environment within a virtualenv and see if that works? There's no way in hell I'm updating to Big Sur yet, so I can't debug it myself for now.

@hynek
Copy link
Owner

hynek commented Nov 30, 2020

In a hilarious turn of events, pip 20.3 has just been released: https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html

Please re-try to install with it!

@applearound
Copy link
Author

In a hilarious turn of events, pip 20.3 has just been released: https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html

Please re-try to install with it!

Yes, it works with pip 20.3.

@stewjb
Copy link

stewjb commented Jan 26, 2021

Is there an update on this issue? I tried and failed to pip install this package. Here are my specs:
Numpy version 1.19.5
Python version 3.9.1
pip version 20.3.3
wheel version 0.36.2
macOS Big Sur 11.1 (Apple Silicon/M1)
16gb ram

@hynek
Copy link
Owner

hynek commented Jan 26, 2021

Well, it works for others, so you're probably missing some build dependency.

@avibrazil avibrazil mentioned this issue Feb 10, 2021
Closed
@telamonian
Copy link

telamonian commented Feb 10, 2021 •
edited
Loading

@hynek The -msse2 flag you're using during compilation is x86-64 specific:

argon2-cffi/setup.py

Lines 327 to 334 in ce46d6a

objects = self.compiler.compile(
sources,
extra_preargs=["-msse2"] if optimized and not windows else [],
output_dir=self.build_temp,
macros=macros,
include_dirs=include_dirs,
debug=self.debug,
)

@stewjb has the new apple arm chip. arm CPUs don't support msse, they have their own SIMD extension called Neon

@telamonian
Copy link

Actually, apple's own docs make it sound like there's no SIMD extension whatsoever in the new M1 chips: https://developer.apple.com/documentation/apple_silicon/addressing_architectural_differences_in_your_macos_code

@hynek
Copy link
Owner

hynek commented Feb 11, 2021

We only pass that flag if optimized is True and optimized is only True on x86:

argon2-cffi/setup.py

Line 30 in ce46d6a

optimized = platform.machine() in ("i686", "x86", "x86_64", "AMD64")

So I’m gonna need more output with recent Pythons/pips since I can’t test it myself.

@hynek
Copy link
Owner

hynek commented Feb 11, 2021

(Also, most people use Python still under Rosetta which makes -msse2 correct.)

@hynek
Copy link
Owner

hynek commented Dec 8, 2021

This is fixed by the outsourcing to https://github.com/hynek/argon2-cffi-bindings

Feel free to try to pip-install it and report back if it's still broken.

@hynek hynek closed this as completed Dec 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hynek @telamonian @applearound @stewjb