Python 3.7.0 :: The Python ssl extension was not compiled. Missing the OpenSSL lib?

[rmerkushin]

Platform: macOS High Sierra 10.13.5
Pyenv version: Install by git clone
Python: 3.7.0
OpenSSL version: 1.0.2o_2 (installed from brew)

tricks below doesn't works for me:

CFLAGS="-I$(brew --prefix openssl)/include" \
LDFLAGS="-L$(brew --prefix openssl)/lib" \
pyenv install -v 3.7.0

[Packets-n-Python]

Same with Ubuntu 14

CFLAGS=-I/usr/include/openssl
LDFLAGS=-L/usr/local/lib
pyenv install 3.7.0
Downloading Python-3.7.0.tar.xz...
-> https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
Installing Python-3.7.0...
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems

BUILD FAILED (Ubuntu 14.04 using python-build 1.2.5-4-gc057a80)

Inspect or clean up the working tree at /tmp/python-build.20180706171040.46966
Results logged to /tmp/python-build.20180706171040.46966.log

Last 10 log lines:
install|*) ensurepip="" ;;
esac;
./python -E -m ensurepip
$ensurepip --root=/ ;
fi
Looking in links: /tmp/tmpds7w9_se
Collecting setuptools
Collecting pip
Installing collected packages: setuptools, pip
Successfully installed pip-10.0.1 setuptools-39.0.1

[gamesbrainiac]

I have the same issue. I also tried the following:

brew install libffi

Since it seems to do the trick on Ubuntu. This however does not work on macOS.

[mhart]

Seeing the same thing on Amazon Linux (ie, Fedora essentially)

[ladyrick]

still doesn't work yet.
have to use python 3.6.6

[lgw4]

I can successfully install CPython 2.7.15, 3.6.6, and 3.7.0 using pyenv v1.2.5. on macOS 10.13.6. I (briefly) searched for differences between v1.2.5 and v1.2.6 but didn't see anything obvious that would cause this problem.

[lukebakken]

I ran into this issue with openssl 1.0.2o_2 installed. The following allowed me to build and install Python 3.7.0 via pyenv:

brew install 'openssl@1.1'
CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl@1.1)" pyenv install 3.7.0

[lgw4]

@lukebakken I also got it to work with OpenSSL 1.0.2o_2 by passing the same CONFIGURE_OPTS but for the openssl package:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" pyenv install 3.7.0

I wish I knew why this wasn't needed under pyenv 1.2.5.

[lukebakken]

@lgw4 I'll give that a try just for grins. It's interesting that 1.0.2 worked. Prior to installing OpenSSL 1.1 the test for OpenSSL failed with a missing symbol in the dynamic library (SSL_new, I think), which I would have expected to be #ifdef'd for compatibility with older OpenSSL versions 🤷‍♂️

[lgw4]

@lukebakken FWIW, I've also got the following environment variables set:

CFLAGS=-I/usr/local/opt/openssl/include
CPPFLAGS=-I/usr/local/opt/openssl/include
LDFLAGS=-L/usr/local/opt/openssl/lib

[lukebakken]

I uninstalled openssl@1.1 and confirmed only the one OpenSSL version is installed:

$ ll /usr/local/opt/openssl*
lrwxr-xr-x  1 lbakken  admin  26 Jul 31 08:31 /usr/local/opt/openssl -> ../Cellar/openssl/1.0.2o_2
lrwxr-xr-x  1 lbakken  admin  26 Jul 31 08:31 /usr/local/opt/openssl@1.0 -> ../Cellar/openssl/1.0.2o_2

None of the environment variables mentioned here are exported, and I ran the installation as follows:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" pyenv install 3.7.0

Installation succeeds, and you can see the correct compiler and linker arguments in config.log during the feature testing phase:

configure:16986: checking whether compiling and linking against OpenSSL works
Trying link with OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=-I/usr/local/opt/openssl/include
configure:17008: clang -o conftest  -I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/Users/lbakken/.pyenv/versions/3.7.0/include  -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/lbakken/.pyenv/versions/3.7.0/lib  -L/usr/local/opt/openssl/lib conftest.c -lssl -lcrypto -ldl  -framework CoreFoundation >&5
configure:17008: $? = 0
configure:17010: result: yes

So it looks like, for Python 3.7.0 on OS X at least, the --with-openssl argument to configure must be used. Note that I did try the fix proposed in #1191 but it didn't work, probably because I don't believe the pyenv-local OpenSSL was built (?).

[jhunkeler]

#1191 is only partially correct (for now). The system I wrote that patch for was running OSX 10.11 so I didn't see the next edge case coming...

On OSX 10.13 it proceeds to compile OpenSSL 1.0* as you'd except, however it fails to produce a library containing the SSL_new symbol. Python cannot link against it, so everything falls apart at that point.

After a little tinkering I managed to build OpenSSL, albeit the stable release, and it appears to work fine when compiled against Python 3.7.0.

(PS - 3.6.6 fails on 10.13 in the same way. If I switch its build script to use the stable release of OpenSSL it does seem to fix the problem there too. Hmm.)

index 1ff0ac29..2d5552d1 100755
--- a/plugins/python-build/bin/python-build
+++ b/plugins/python-build/bin/python-build
@@ -1504,6 +1504,9 @@ use_homebrew_openssl() {
 }

 build_package_mac_openssl() {
+  OPENSSL_VERSION_LTS='1.0.*'
+  OPENSSL_VERSION_STABLE='1.1.*'
+
   # Install to a subdirectory since we don't want shims for bin/openssl.
   OPENSSL_PREFIX_PATH="${PREFIX_PATH}/openssl"

@@ -1520,12 +1523,25 @@ build_package_mac_openssl() {
   # Hint OpenSSL that we prefer a 64-bit build.
   export KERNEL_BITS="64"
   OPENSSL_CONFIGURE="${OPENSSL_CONFIGURE:-./config}"
+  OPENSSL_OPTIONS="zlib-dynamic shared"

   local nokerberos
-  [[ "$1" != openssl-1.0.* ]] || nokerberos=1
+  [[ "$1" != openssl-${OPENSSL_VERSION_LTS} ]] || nokerberos=1
+
+  local openssl_stable
+  [[ "$1" == openssl-${OPENSSL_VERSION_STABLE} ]] && openssl_stable=1
+
+  # Set additional flags based on LTS versus Stable
+  if [[ ! $openssl_stable ]]; then
+    OPENSSL_OPTIONS="${OPENSSL_OPTIONS} no-ssl2 no-ssl3"
+    [[ $nokerberos ]] && OPENSSL_OPTIONS="${OPENSSL_OPTIONS} no-krb5"
+  else
+    # Deprecated: SSL2, KRB5
+    OPENSSL_OPTIONS="${OPENSSL_OPTIONS} no-ssl3"
+  fi

   # Compile a shared lib with zlib dynamically linked, no kerberos.
-  package_option openssl configure --openssldir="$OPENSSLDIR" zlib-dynamic no-ssl2 no-ssl3 no-krb5 shared
+  package_option openssl configure --openssldir="$OPENSSLDIR" ${OPENSSL_OPTIONS}

   # Default MAKE_OPTS are -j 2 which can confuse the build. Thankfully, make
   # gives precedence to the last -j option, so we can override that.
diff --git a/plugins/python-build/share/python-build/3.7.0 b/plugins/python-build/share/python-build/3.7.0
index f65d2a73..11a8ce7f 100644
--- a/plugins/python-build/share/python-build/3.7.0
+++ b/plugins/python-build/share/python-build/3.7.0
@@ -1,5 +1,5 @@
 #require_gcc
-install_package "openssl-1.0.2k" "https://www.openssl.org/source/openssl-1.0.2k.tar.gz#6b3977c61f2aedf0f96367dcfb5c6e578cf37e7b8d913b4ecb6643c3cb88d8c0" mac_openssl --if has_broken_mac_openssl
+install_package "openssl-1.1.0h" "https://www.openssl.org/source/openssl-1.1.0h.tar.gz#5835626cde9e99656585fc7aaa2302a73a7e1340bf8c14fd635a62c66802a517" mac_openssl --if has_broken_mac_openssl
 install_package "readline-6.3" "https://ftpmirror.gnu.org/readline/readline-6.3.tar.gz#56ba6071b9462f980c5a72ab0023893b65ba6debb4eeb475d7a563dc65cafd43" standard --if has_broken_mac_readline
 if has_tar_xz_support; then
   install_package "Python-3.7.0" "https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz#0382996d1ee6aafe59763426cf0139ffebe36984474d0ec4126dd1c40a8b3549" ldflags_dirs standard verify_py37 ensurepip```

[lgw4]

@jhunkeler Awesome! However, had you switched to https://www.openssl.org/source/openssl-1.0.2o.tar.gz, I bet the LTS would succeed, too.

[jhunkeler]

D'oh! Thank you! Yes, the latest LTS works fine 👍

diff --git a/plugins/python-build/share/python-build/3.7.0 b/plugins/python-build/share/python-build/3.7.0
index f65d2a73..173f3249 100644
--- a/plugins/python-build/share/python-build/3.7.0
+++ b/plugins/python-build/share/python-build/3.7.0
@@ -1,5 +1,5 @@
 #require_gcc
-install_package "openssl-1.0.2k" "https://www.openssl.org/source/openssl-1.0.2k.tar.gz#6b3977c61f2aedf0f96367dcfb5c6e578cf37e7b8d913b4ecb6643c3cb88d8c0" mac_openssl --if has_broken_mac_openssl
+install_package "openssl-1.0.2o" "https://www.openssl.org/source/openssl-1.0.2o.tar.gz#ec3f5c9714ba0fd45cb4e087301eb1336c317e0d20b575a125050470e8089e4d" mac_openssl --if has_broken_mac_openssl
 install_package "readline-6.3" "https://ftpmirror.gnu.org/readline/readline-6.3.tar.gz#56ba6071b9462f980c5a72ab0023893b65ba6debb4eeb475d7a563dc65cafd43" standard --if has_broken_mac_readline
 if has_tar_xz_support; then
   install_package "Python-3.7.0" "https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz#0382996d1ee6aafe59763426cf0139ffebe36984474d0ec4126dd1c40a8b3549" ldflags_dirs standard verify_py37 ensurepip```

[rmerkushin]

Yep, this working for me in zsh:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" pyenv install 3.7.0

But not working in fish :(

set CONFIGURE_OPTS "--with-openssl="(brew --prefix openssl); and pyenv install 3.7.0

[lgw4]

@rmerkushin I think your fish syntax is off a bit.

env CONFIGURE_OPTS "--with-openssl="(brew --prefix openssl)"" pyenv install 3.7.0

I don’t use fish so I can’t test this.

[rmerkushin]

@lgw4 nope, it's not working. I think it's python installation script not properly working with fish :(

[zyong812]

use brew install pyenv instead.
It solved the same problem for me.

[kzidane]

@Packets-n-Python did you end up finding a solution to install 3.7 on Ubuntu 14.04?

[tonal]

see also https://bugs.python.org/issue34028 and travis-ci/travis-ci#9069

[rmerkushin]

@zyong812 Do you export any variables before install Python 3.7? Which version of OpenSSL do you install?

[rmerkushin]

Holy crap! I found a solution for fish!

git clone https://github.com/pyenv/pyenv.git ~/.pyenv
git clone https://github.com/pyenv/pyenv-virtualenv.git (pyenv root)/plugins/pyenv-virtualenv
brew install autoconf pkg-config openssl readline

My config.fish:

set -x PYENV_ROOT ~/.pyenv
set -x PATH ~/.pyenv/bin $PATH

if status --is-interactive
    source (pyenv init - | psub)
    source (pyenv virtualenv-init - | psub)
end

P.S.: No need extra environment variables

[paulswartz]

For fish (@rmerkushin):

env CONFIGURE_OPTS=--with-openssl=(brew --prefix openssl@1.1) pyenv install 3.7.0

[EscapeLife]

MacOS system，please use brew install instead of Basic GitHub Checkout.

$ brew install pyenv

If you force do, can run this. Basic GitHub Checkout installation Python3.7.0 has bug.

$ brew install 'openssl@1.1'
$ CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl@1.1)" pyenv install 3.7.0

@rmerkushin l test can be install Python3.7.0 with pyenv normally，hope to help you!

[luchevoi]

I've faced that issue in Debian 8.11 (jessie) with default "OpenSSL 1.0.1t 3 May 2016" version.
So, we have to make our custom openssl build.
For example we are logged in as user "test".

cd ~
mkdir openssl
cd openssl
wget "https://www.openssl.org/source/openssl-1.0.2p.tar.gz"
tar xf openssl-1.0.2p.tar.gz
cd openssl-1.0.2p

Latest OpenSSL 1.0.2 has been downloaded and extracted into "/home/test/openssl" directory. Now let's build it into the same directory. I took instructions from here.

./config --prefix=/home/test/openssl --openssldir=/home/test/openssl shared zlib
make
make test
make install

If compiling and testing passed well, you are become able to just link your fresh OpenSSL install to pyenv.

LD_RUN_PATH="/home/test/openssl/lib" LDFLAGS="-L/home/test/openssl/lib" CPPFLAGS="-I/home/test/openssl/include" CFLAGS="-I/home/test/openssl/include" CONFIGURE_OPTS="--with-openssl=/home/test/openssl" pyenv install 3.7.0

LD_RUN_PATH is a necessary option!

[studnitskiy]

On macOS with Homebrew, make sure that pkg-config is installed. Installing pyenv from Homebrew actually takes care of that. However, with git clone installation method, necessary dependences need to be installed manually.

I had the issue on one of my Macs, while on another it was building fine. I tried adding OpenSSL 1.0.2, 1.1, LibreSSL paths in CFLAGS and LDFLAGS - no change. Finally, comparing installed Homebrew packages and build logs helped to narrow this down to pkg-config. You actually don't need to add OpenSSL paths manually, pkg-config takes care of that once it's installed (unlike zlib and sqlite, by the way).

[al45tair]

For Mac users, pyenv was supposed to work automatically by downloading and building a copy of OpenSSL specifically for the use of the Python version you're installing. Unfortunately, the code to do that neglected to actually tell the Python build process to use that copy of OpenSSL, which is why a lot of people in this thread advise you to install OpenSSL separately (e.g. using brew). But that shouldn't be necessary — pyenv has already built and installed a version of OpenSSL. The pull request I just submitted fixes things so that pyenv will actually tell Python to use the version it's just downloaded and built.

[liamdawson]

I ran into this issue after installing pyenv via Linuxbrew. After some digging, I noticed I was using pkg-config from Linuxbrew, not my base system, so I added /usr/bin to the front of my path (to ensure it'd use the system pkg-config), then it worked as expected.

e.g.

export PATH="/usr/bin:$PATH"
# or in fish:
set -x PATH '/usr/bin' $PATH

[mattdodge]

This all seems fine and great for installing new pyenv versions on OSX. But what if I already have dozens of pyenv environments created with an existing python version? Brew upgraded openssl to version 1.1 but all of my existing envs are linked to pyenv pythons using openssl 1.0.2. Is there any path to upgrade those or am I stuck recreating all of the environments? Or is there any way to have openssl versions 1.0.2 and 1.1 coexist on the same system?

[hermansje]

@mattdodge
I ran pyenv install for the existing versions and confirmed to continue with y.
The installed packages are still there and the OpenSSL error is gone.

[mattdodge]

@hermansje Awesome! I didn't even think to run the install command to reinstall on the existing versions. That worked great! Thanks

[L0wBat7erY]

LD_RUN_PATH is a necessary option!

Yes, it is!

Thank you!

PS: the commands I used to install 3.7.2 with pyenv on debian 8:

• wget https://www.openssl.org/source/openssl-1.1.0j.tar.gz
• tar -xvzf openssl-1.1.0j.tar.gz
• cd openssl-1.1.0j/
• ./config --prefix=/usr/local/openssl11 --openssldir=/usr/local/openssl11
• make
• make install
• CONFIGURE_OPTS="--with-openssl=/usr/local/openssl11" LDFLAGS="-L/usr/local/openssl11/lib" LD_RUN_PATH="/usr/local/openssl11/lib" CPPFLAGS="-I/usr/local/openssl11/include" CFLAGS="-I/usr/local/openssl11/include" pyenv install -v 3.7.2

Very awesome!!! I just did. It's work. Many thanks

[damnko]

Hi, I'm having the same issue and apparently can't solve it with any of the above.

I'm running on macOS Mojave (10.14.6) and trying to install python 3.5.2.

After installing pyenv with homebew I got this message

openssl@1.1 is keg-only, which means it was not symlinked into /usr/local,
because openssl/libressl is provided by macOS so don't link an incompatible version.

If you need to have openssl@1.1 first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.bash_profile

For compilers to find openssl@1.1 you may need to set:
  export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"
  export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"
  
  export CPPFLAGS="-I$(brew --prefix zlib)/include"


For pkg-config to find openssl@1.1 you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"

I have tried to run all the commands in the https://github.com/pyenv/pyenv/wiki/Common-build-problems page and additionally:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl@1.1)" pyenv install 3.5.2

CONFIGURE_OPTS="--with-openssl=/usr/local/opt/openssl@1.1" LDFLAGS="-L/usr/local/opt/openssl@1.1/lib" LD_RUN_PATH="/usr/local/opt/openssl@1.1/lib" CPPFLAGS="-I/usr/local/opt/openssl@1.1/include" CFLAGS="-I/usr/local/opt/openssl@1.1/include" PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig" pyenv install -v 3.5.2

CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib" pyenv install 3.5.2

But I'm aways getting the same error:

ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?
...
BUILD FAILED (OS X 10.14.6 using python-build 20180424)

Am I missing anything basic here? Do you have any suggestion?
Thanks

[chairco]

@damnko same error, did you find any solution?

[damnko]

@chairco , nope. I ended up using conda create -n p352_env python=3.5.2.

[dmitrii-sim]

@damnko @chairco Ok guys, here is the answer: #1425 (comment)
The long story short - Homebrew installs OpenSSL 1.1 and python 3.5.2 using OpenSSL 1.0.2
And to install python3.5.2 you need to manually download and install OpenSSL 1.0.2
Additional info and how to install OpenSSL 1.0.2 you can find by the link above.
MacOS Catalina 10.15.3

[stefanitsky]

@dmitrii-sim thank you! My headache is finally gone.

[sittitep]

Confirm that this works

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" pyenv install 3.7.0

[bsed]

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl@1.1)" pyenv install 3.7.0

[damnko]

Thanks a lot @dmitrii-sim that worked, I was able to install also python 3.5.2 !

[Center-Sun]

• tps://www.openssl.org/source/openssl-1.1.0j.tar.gz
• tar -xvzf openssl-1.1.0j.tar.gz
• cd openssl-1.1.0j/
• ./config --prefix=/usr/local/openssl11 --openssldir=/usr/local/openssl11
• make
• make install
• CONFIGURE_OPTS="--with-openssl=/usr/local/openssl11" LDFLAGS="-L/usr/local/openssl11/lib" LD_RUN_PATH="/usr/local/openssl11/lib" CPPFLAGS="-I/usr/local/openssl11/include" CFLAGS="-I/usr/local/openssl11/include" pyenv install -v 3.7.2

solved my problem with openssl 1.1.1c & python 3.8.0.b4 on centos 7 , thanks so much~~~

[gorjanradevski]

Seeing the same thing on Amazon Linux (ie, Fedora essentially)

@mhart, I experience the same on Fedora 32. Did you find how to overcome the issue?

[native-api]

Summarized in https://github.com/pyenv/pyenv/wiki/Common-build-problems#error-the-python-ssl-extension-was-not-compiled-missing-the-openssl-lib

[al45tair]

@native-api @alexpulver On the Mac build, it specifically downloads and builds OpenSSL. It's just that it then doesn't set the paths correctly to pick it up. That's why there's a PR (#1241) containing a fix for that issue.

This wasn't the same as the similar Linux issue, and all the talk about Homebrew in this ticket is a distraction from what's actually wrong.

[native-api]

@al45tair The OP's issue was caused by an incompatible OpenSSL in Homebrew. Another version being built and ignored is wrong, of course, but it's a different issue -- that's why I closed this ticket but not your PR.

[jpmckinney]

To install Python 3.6, I followed the instructions at #1768 (comment). However, I got the error from the issue title/description.

I tried to add CONFIGURE_OPTS as described here: #1184 (comment). openssl@1.1 was already installed (it's a dependency of pyenv), so I didn't re-install it.

The relevant command is:

curl -sSL 'https://github.com/python/cpython/commit/8ea6353.patch?full_index=1' | arch -x86_64 -e CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl@1.1)" pyenv install --patch 3.6.15

However, it results in the same error. I'm not sure how others are able to run 3.6 on M1.

Update: I can install 3.6.13! 3.6.15 (although the patch applies) fails with the error in the issue title/description.

[Tapudp]

None of the options work for me on my M1 mac.

It shows openssl version to be LibreSSL 3.3.6

Upon initiating pyenv install 3.10.6 the first line mentions python-build: use openssl@1.1 from homebrew but at the end it fails. :(

[Tapudp]

Also the exact error is as follows:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'readline'
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/divyesh.parmar@postman.com/.pyenv/versions/3.10.6/lib/python3.10/ssl.py", line 99, in <module>
    import _ssl             # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

[huyz]

@Tapudp did you follow https://github.com/pyenv/pyenv/wiki#troubleshooting--faq ?

[144mdgross]

I ran into this on an arm machine with an M2 chip, OSX 13.5.
I tried many different things, followed the troubleshooting doc and had no luck until running this command:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" arch -x86_64 pyenv install 3.8.17