`pip install cryptography` broken on OS X El Capitan

[ctismer]

The simple pip install cryptography command does not work on OS X (only tested on El Capitan).

What I tried:

brew install openssl
pip install cryptography

results in

build/temp.macosx-10.9-intel-2.7/_openssl.c:425:10: fatal error: 'openssl/aes.h' file not found

#include <openssl/aes.h>

         ^

1 error generated.

error: command 'clang' failed with exit status 1

The following workaround helped:

$ ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" \
   CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography

[reaperhulk]

Upgrade your pip as you appear to be using a version that predates support for binary wheels. What does pip --version give you?

[ctismer]

$ pip --version
pip 8.0.2 from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (python 2.7)

No, I use wheel support myself since a long time. There seems to be a reason that prevents
the application of a wheel.
Also I don't think it is correct if it enforces wheels.

[reaperhulk]

Perhaps an explanation of why cryptography prefers wheels would be helpful here.

cryptography ships with an OpenSSL backend. This backend needs to be compiled and linked against libssl and libcrypto. OS X historically has shipped with OpenSSL, but Apple made the decision to deprecate and strongly recommend against its use starting with OS X 10.7. While we previously supported linking against it, this support is being dropped as the 0.9.8 series is no longer supported by the OpenSSL project and is also so old that major features necessary for modern TLS support are not present. Since we need to compile against a newer version then an alternate version of OpenSSL must be installed (typically by either homebrew or macports). Unfortunately, since Apple still ships the 0.9.8 libraries (but not the headers as of OS X El Capitan) these newer OpenSSL libraries can't safely be put in the default lib search path. So, our options were to either force anyone who wanted to install cryptography to install openssl separately and understand the C toolchain enough to provide the proper include/linker flags, or build a static wheel that would allow pip install cryptography to work with no compiler required at all.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them. When calling pip install it preferentially uses them because it dramatically simplifies the install process for the vast majority of our OS X and Windows users.

In general this binary wheel resolution process is pretty reliable, but sometimes edge cases occur. We currently upload two OS X wheels for each Python we support (a 10_6 SDK version and a 10_10). pip 8 altered its wheel compatibility code to prefer the newest compatible wheel. In your case it appears you're running on Mavericks, so the 10_6 SDK wheel should have been downloaded. To test I just booted a Mavericks VM and using pip 8.0.2 I got the 10_6 wheel as expected with pip install cryptography, so I'm not sure what's going on. You may want to open an issue with pip.

[ctismer]

Thanks for the explanation. No, as stated on top, I used El Capitan, which is 10.11.2 .
With which python do you try on OS X?
There is a big difference between a downloaded Python from Python.org and a homebrew python.
They are not compatible, because they use different models, and when I build packages,
I always have to build them twice, because the one or the other will complain (like me :) ).
Don't remember exactly right now, but I think it was universal build (python.org) vs. simple build from homebrew.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them.

The setup script should configure itself and find needed arguments. The problem is that due
to common practice in other packages, cryptography simply gets installed as a dependancy, and
that then fails miserably.

This was what actually happened, when I tried

pip install --pre gitgub3.py

[ctismer]

I think I was causing the problem by using another python version (stackless python).
Sorry about the confusion.
The statement still holds: The build should configure itself, when it claims to be installable
with pip.

Cheers - Chris

[alex]

There's no way for the build to configure itself, if OpenSSL's headers aren't on your platform's include path, there's nothing we can do.

[ctismer]

@alex the installation paths of homebrew et.al. are a de-facto standard,
so I think it should at least be possible to give a useful hint instead of crashing.

[zopyx]

I have a similar issue with installing Plone 5.0 and some add-ons depending on the cryptography module. I see the same include problem running buildout on MacOSX. However the workaround here is to "pip install crypography" inside the local virtual env that is used to run buildout.

[carn1x]

Upgrading from pip==1.3.1 to pip==9.0.1 fixed this for me

[hanxinhisen]

pip install --upgrade pip

[alexislg2]

Thanks @hanxinhisen that worked ;)