pip install cryptography broken on OS X El Capitan

[ctismer]

The simple pip install cryptography command does not work on OS X (only tested on El Capitan).

What I tried:

brew install openssl
pip install cryptography

results in

build/temp.macosx-10.9-intel-2.7/_openssl.c:425:10: fatal error: 'openssl/aes.h' file not found

#include <openssl/aes.h>

         ^

1 error generated.

error: command 'clang' failed with exit status 1

The following workaround helped:

$ ARCHFLAGS="-arch x86_64" LDFLAGS="-L/usr/local/opt/openssl/lib" \
   CFLAGS="-I/usr/local/opt/openssl/include" pip install cryptography

[reaperhulk]

Upgrade your pip as you appear to be using a version that predates support for binary wheels. What does pip --version give you?

[ctismer]

$ pip --version
pip 8.0.2 from /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (python 2.7)

No, I use wheel support myself since a long time. There seems to be a reason that prevents
the application of a wheel.
Also I don't think it is correct if it enforces wheels.

[reaperhulk]

Perhaps an explanation of why cryptography prefers wheels would be helpful here.

cryptography ships with an OpenSSL backend. This backend needs to be compiled and linked against libssl and libcrypto. OS X historically has shipped with OpenSSL, but Apple made the decision to deprecate and strongly recommend against its use starting with OS X 10.7. While we previously supported linking against it, this support is being dropped as the 0.9.8 series is no longer supported by the OpenSSL project and is also so old that major features necessary for modern TLS support are not present. Since we need to compile against a newer version then an alternate version of OpenSSL must be installed (typically by either homebrew or macports). Unfortunately, since Apple still ships the 0.9.8 libraries (but not the headers as of OS X El Capitan) these newer OpenSSL libraries can't safely be put in the default lib search path. So, our options were to either force anyone who wanted to install cryptography to install openssl separately and understand the C toolchain enough to provide the proper include/linker flags, or build a static wheel that would allow pip install cryptography to work with no compiler required at all.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them. When calling pip install it preferentially uses them because it dramatically simplifies the install process for the vast majority of our OS X and Windows users.

In general this binary wheel resolution process is pretty reliable, but sometimes edge cases occur. We currently upload two OS X wheels for each Python we support (a 10_6 SDK version and a 10_10). pip 8 altered its wheel compatibility code to prefer the newest compatible wheel. In your case it appears you're running on Mavericks, so the 10_6 SDK wheel should have been downloaded. To test I just booted a Mavericks VM and using pip 8.0.2 I got the 10_6 wheel as expected with pip install cryptography, so I'm not sure what's going on. You may want to open an issue with pip.

[ctismer]

Thanks for the explanation. No, as stated on top, I used El Capitan, which is 10.11.2 .
With which python do you try on OS X?
There is a big difference between a downloaded Python from Python.org and a homebrew python.
They are not compatible, because they use different models, and when I build packages,
I always have to build them twice, because the one or the other will complain (like me :) ).
Don't remember exactly right now, but I think it was universal build (python.org) vs. simple build from homebrew.

Of course, we do still document how to build it on OS X using LDFLAGS and CFLAGS to dynamically link against a user-installed OpenSSL, so it's not correct to state that we require them.

The setup script should configure itself and find needed arguments. The problem is that due
to common practice in other packages, cryptography simply gets installed as a dependancy, and
that then fails miserably.

This was what actually happened, when I tried

pip install --pre gitgub3.py

[ctismer]

I think I was causing the problem by using another python version (stackless python).
Sorry about the confusion.
The statement still holds: The build should configure itself, when it claims to be installable
with pip.

Cheers - Chris

[alex]

There's no way for the build to configure itself, if OpenSSL's headers aren't on your platform's include path, there's nothing we can do.

[ctismer]

@alex the installation paths of homebrew et.al. are a de-facto standard,
so I think it should at least be possible to give a useful hint instead of crashing.

[zopyx]

I have a similar issue with installing Plone 5.0 and some add-ons depending on the cryptography module. I see the same include problem running buildout on MacOSX. However the workaround here is to "pip install crypography" inside the local virtual env that is used to run buildout.

[carn1x]

Upgrading from pip==1.3.1 to pip==9.0.1 fixed this for me

[hanxinhisen]

pip install --upgrade pip

[alexislg2]

Thanks @hanxinhisen that worked ;)

[iolloyd]

Thanks @hanxinhisen

[dmitrytokarev]

brew to the rescue:

brew install openssl

then brew gives a hint:

Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries

Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include
    PKG_CONFIG_PATH: /usr/local/opt/openssl/lib/pkgconfig

So just export these env vars and rerun your favorite pip install (or better to avoid havoc in future add following to your ~/.bash_profile:

export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

This worked for me like a charm!

[Jay54520]

@dmitrytokarev This worked for me

python3.6/site-packages/wheel/bdist_wheel.py", line 155, in get_tag
      assert tag == supported_tags[0]
  AssertionError
  
  ----------------------------------------
  Failed building wheel for cryptography
  Running setup.py clean for cryptography
Failed to build cryptography
Installing collected packages: cryptography
  Running setup.py install for cryptography ... done
Successfully installed cryptography-1.6


pip list | grep cry
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
cryptography (1.6)
pycrypto (2.6.1)

[reaperhulk]

cryptography 1.7 has a py36 wheel.

[walles]

For workarounds, I liked this one better:
https://solitum.net/openssl-os-x-el-capitan-and-brew/

$ brew doctor (now fix anything that it tells you to fix)
$ brew update
$ brew upgrade
$ brew install openssl
$ cd /usr/local/include
$ ln -s ../opt/openssl/include/openssl

[reaperhulk]

Linking the include headers into /usr/local is not a wise decision. OpenSSL 0.9.8zh is still available in /usr/lib on macOS and the headers will mismatch with the dynamic libs. When linking against OpenSSL from homebrew (or ports) you should always pass CFLAGS and LDFLAGS (you can see an example in our install docs). However, on a mac running the latest pip you do not need a compiler at all because we release binary artifacts for cryptography on mac and windows.

[judy2k]

@reaperhulk I'm running pip 9.0.1 and python 2.7.9, but my Mac is still trying to build from source - can you help me work out why this is?

(Incidentally, I got it to compile using the tip from @dmitrytokarev above)

[reaperhulk]

Due to an issue with the compilation of some of the wheels the current release has a bit less coverage than normal. Do you get a wheel if you do pip install cryptography==1.7.1? If so that's the issue, and it will be resolved with the next cryptography release (sorry!)

[leonsmith]

Just to add another data point, I get a wheel with 1.7.1 but 1.7.2 has no wheel & fails to build.

[reaperhulk]

This will be resolved with the release of 1.8. The original wheels for the pythons having problems were built incorrectly. PyPI doesn't allow you to upload different wheels with the same name (for good security reasons), so we can't upload new ones without either using a build tag or releasing a new version. Unfortunately pip doesn't respect wheel build tags right now, so that fix won't work either. Sorry about this. cryptography 1.8 should be out within the next week.

[leonsmith]

Thanks for clarification!

I assumed that was the case but was still seeing 1.7.2 on pypi so I thought it was worth the clarification (even for others sake!)

[racitup]

@Jay54520 I get a similar error to yours but for pycrypto. Can the AssertionError be safely ignored?

File "/Users/rich/.pyenv/versions/3.5.2/lib/python3.5/site-packages/wheel/bdist_wheel.py", line 155, in get_tag
  assert tag == supported_tags[0]

AssertionError

---

Failed building wheel for pycrypto
Running setup.py clean for pycrypto
Failed to build pycrypto
Installing collected packages: pycrypto
Running setup.py install for pycrypto ... done
Successfully installed pycrypto-2.6.1

[hatgit]

Not sure if it is worth noting for those with python3 that "pip3" may also be used to verify, sharing just in case: pip3 install cryptography

[mradkov]

@dmitrytokarev Solution worked for me like a charm!

pip 9.0.1
python 3.6

[mahmudahsan]

@dmitrytokarev thank you very much. your solution works perfectly.
`
export LDFLAGS="-L/usr/local/opt/openssl/lib"

export CPPFLAGS="-I/usr/local/opt/openssl/include"

export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"
`

[favorinfo]

another solution is add global-option with your requirments
cryptography --global-option=build_ext --global-option="-I/usr/local/opt/openssl/include" --global-option="-L/usr/local/opt/openssl/lib"
with openssl was installed by brew

[ra0x3]

@dmitrytokarev +1. Worked for me on MacOSX 10.12.6. Exported flags in bash, then pip install ...

[reaperhulk]

Anyone experiencing this issue on CPython in macOS should not need to do anything other than upgrade their pip. No magic incantations required unless you're doing something unusual, in which case there is an installation section in our docs that explains how to do this.

Since this issue keeps attracting me toos I'm going to lock this so this comment is the last one and hopefully lead people to the simpler solution 😄