Symbol not found: _deflate from grpc 1.12.0

[ssarangi]

What version of gRPC and what language are you using?

gRPC Version: 1.12.0 Python: 3.6.4 (From Anaconda)

What operating system (Linux, Windows, …) and version?

OSX (High Sierra)

What runtime / compiler are you using (e.g. python version or version of gcc)

Python: 3.6.4 (From Anaconda)

What did you do?

If possible, provide a recipe for reproducing the error. Try being specific and include code snippets if helpful.
A simple grpc tutorial fails to run and fails on import.

What did you expect to see?

I expected the server to run.

What did you see instead?

python server.py
Traceback (most recent call last):
File "server.py", line 1, in
import grpc
File "/Users/satyajits/anaconda3/lib/python3.6/site-packages/grpc/init.py", line 22, in
from grpc._cython import cygrpc as _cygrpc
ImportError: dlopen(/Users/satyajits/anaconda3/lib/python3.6/site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so, 2): Symbol not found: _deflate
Referenced from: /Users/satyajits/anaconda3/lib/python3.6/site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so
Expected in: flat namespace
in /Users/satyajits/anaconda3/lib/python3.6/site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment?

None.

[ssarangi]

I tried this with virtualenv as well and the same issue exists as well. So its not Anaconda specific.

Traceback (most recent call last):
File "server.py", line 1, in
import grpc
File "/Users/satyajits/dev/tmp/venv/lib/python3.6/site-packages/grpc/init.py", line 22, in
from grpc._cython import cygrpc as _cygrpc
ImportError: dlopen(/Users/satyajits/dev/tmp**/venv/lib/python3.6/**site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so, 2): Symbol not found: _deflate
Referenced from: /Users/satyajits/dev/tmp/venv/lib/python3.6/site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so
Expected in: flat namespace
in /Users/satyajits/dev/tmp/venv/lib/python3.6/site-packages/grpc/_cython/cygrpc.cpython-36m-darwin.so
(venv)

[nathanielmanistaatgoogle]

@mehrdada: would you be able to take a look at this in the next day or two?

[mehrdada]

Cannot reproduce on Python3 installed via brew:

Last login: Tue May 22 10:00:24 on ttys006
Carbon:~ m$ python3
-bash: python3: command not found
Carbon:~ m$ brew install python3
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core, caskroom/cask).
==> New Formulae
brew-php-switcher                libbitcoin-protocol              nifi-registry                    wireguard-go
==> Updated Formulae
cmake ✔               dash                  gnome-builder         libbitcoin-explorer   pam-u2f               sops
agedu                 dbxml                 goenv                 libbitcoin-server     pari                  source-to-image
ammonite-repl         dcd                   grafana               libidn2               passenger             spades
angular-cli           debianutils           grpc                  libphonenumber        pdftoedn              spdlog
annie                 dependency-check      gst-editing-services  libqalculate          pdftoipe              spigot
ansible               dhall-json            gst-libav             librealsense          pegtl                 spring-roo
apibuilder-cli        diff-pdf              gst-plugins-bad       libu2f-host           petsc                 srtp
aptly                 diffoscope            gst-plugins-base      libxlsxwriter         pgloader              stella
archey                dlib                  gst-plugins-good      libxo                 pilosa                stunnel
aria2                 dnscrypt-proxy        gst-plugins-ugly      logtalk               plantuml              swiftformat
armadillo             docfx                 gst-python            lxc                   plowshare             telegraf
artifactory           dscanner              gst-rtsp-server       mariadb               poppler               tepl
aws-sdk-cpp           e2fsprogs             gst-validate          maxwell               postgrest             texmath
awscli                emscripten            gstreamer             mercurial             pound                 tippecanoe
babl                  faas-cli              gtksourceview@4       meson                 pre-commit            traefik
bcal                  fabio                 heroku                mill                  proguard              twarc
binaryen              fabric                icarus-verilog        mitmproxy             psqlodbc              txr
bind                  fdroidserver          imagemagick           monero                pumba                 uhd
bitrise               feh                   imagemagick@6         mongo-c-driver        purescript            uncrustify
bmake                 firebase-cli          infer                 mpc                   qjackctl              vala
cake                  fluent-bit            ispc                  mupdf                 qpm                   vegeta
calicoctl             fn                    jenkins               mupdf-tools           qt                    vim
cglm                  folly                 jfrog-cli-go          mutt                  raylib                vte3
cimg                  fork-cleaner          jhiccup               n                     redshift              vtk
citus                 fq                    jsonpp                nano                  restic                watch
clang-format          fribidi               keepassc              neofetch              restview              wireguard-tools
cockroach             gauge                 kitchen-sync          node-build            rocksdb               x265
coffeescript          gegl                  knot                  node@8                root                  xonsh
convox                geoserver             kompose               nsd                   sagittarius-scheme    xqilla
coreos-ct             git-ftp               kubernetes-helm       nss                   shfmt                 you-get
cppcms                git-lfs               lastpass-cli          opensc                skaffold              youtube-dl
cromwell              gitlab-runner         launch4j              osm2pgrouting         snakemake             zopfli
curl                  glib-networking       libbitcoin-client     osquery               solr                  zsh-autosuggestions
==> Renamed Formulae
latexila -> gnome-latex

==> Installing dependencies for python: sphinx-doc, gdbm, openssl, readline, sqlite, xz
==> Installing python dependency: sphinx-doc
==> Downloading https://homebrew.bintray.com/bottles/sphinx-doc-1.7.4.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring sphinx-doc-1.7.4.high_sierra.bottle.tar.gz
==> Caveats
This formula is keg-only, which means it was not symlinked into /Users/m/Library/Homebrew,
because this formula is mainly used internally by other formulae.
Users are advised to use `pip` to install sphinx-doc.

If you need to have this software first in your PATH run:
  echo 'export PATH="/Users/m/Library/Homebrew/opt/sphinx-doc/bin:$PATH"' >> ~/.bash_profile

==> Summary
🍺  /Users/m/Library/Homebrew/Cellar/sphinx-doc/1.7.4: 2,642 files, 36.8MB
==> Installing python dependency: gdbm
==> Downloading https://homebrew.bintray.com/bottles/gdbm-1.14.1_1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring gdbm-1.14.1_1.high_sierra.bottle.tar.gz
🍺  /Users/m/Library/Homebrew/Cellar/gdbm/1.14.1_1: 20 files, 555.7KB
==> Installing python dependency: openssl
==> Downloading https://www.openssl.org/source/openssl-1.0.2o.tar.gz
######################################################################## 100.0%
==> perl ./Configure --prefix=/Users/m/Library/Homebrew/Cellar/openssl/1.0.2o_1 --openssldir=/Users/m/Library/Homebrew/etc/openssl
==> make depend
==> make
==> make test
==> make install MANDIR=/Users/m/Library/Homebrew/Cellar/openssl/1.0.2o_1/share/man MANSUFFIX=ssl
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /Users/m/Library/Homebrew/etc/openssl/certs

and run
  /Users/m/Library/Homebrew/opt/openssl/bin/c_rehash

This formula is keg-only, which means it was not symlinked into /Users/m/Library/Homebrew,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have this software first in your PATH run:
  echo 'export PATH="/Users/m/Library/Homebrew/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find this software you may need to set:
    LDFLAGS:  -L/Users/m/Library/Homebrew/opt/openssl/lib
    CPPFLAGS: -I/Users/m/Library/Homebrew/opt/openssl/include
For pkg-config to find this software you may need to set:
    PKG_CONFIG_PATH: /Users/m/Library/Homebrew/opt/openssl/lib/pkgconfig

==> Summary
🍺  /Users/m/Library/Homebrew/Cellar/openssl/1.0.2o_1: 1,784 files, 12.2MB, built in 5 minutes 40 seconds
==> Installing python dependency: readline
==> Downloading https://homebrew.bintray.com/bottles/readline-7.0.3_1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring readline-7.0.3_1.high_sierra.bottle.tar.gz
==> Caveats
This formula is keg-only, which means it was not symlinked into /Users/m/Library/Homebrew,
because macOS provides the BSD libedit library, which shadows libreadline.
In order to prevent conflicts when programs look for libreadline we are
defaulting this GNU Readline installation to keg-only..

For compilers to find this software you may need to set:
    LDFLAGS:  -L/Users/m/Library/Homebrew/opt/readline/lib
    CPPFLAGS: -I/Users/m/Library/Homebrew/opt/readline/include

==> Summary
🍺  /Users/m/Library/Homebrew/Cellar/readline/7.0.3_1: 46 files, 1.5MB
==> Installing python dependency: sqlite
==> Downloading https://homebrew.bintray.com/bottles/sqlite-3.23.1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring sqlite-3.23.1.high_sierra.bottle.tar.gz
==> Caveats
This formula is keg-only, which means it was not symlinked into /Users/m/Library/Homebrew,
because macOS provides an older sqlite3.

If you need to have this software first in your PATH run:
  echo 'export PATH="/Users/m/Library/Homebrew/opt/sqlite/bin:$PATH"' >> ~/.bash_profile

For compilers to find this software you may need to set:
    LDFLAGS:  -L/Users/m/Library/Homebrew/opt/sqlite/lib
    CPPFLAGS: -I/Users/m/Library/Homebrew/opt/sqlite/include
For pkg-config to find this software you may need to set:
    PKG_CONFIG_PATH: /Users/m/Library/Homebrew/opt/sqlite/lib/pkgconfig

==> Summary
🍺  /Users/m/Library/Homebrew/Cellar/sqlite/3.23.1: 11 files, 3MB
==> Installing python dependency: xz
==> Downloading https://homebrew.bintray.com/bottles/xz-5.2.4.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring xz-5.2.4.high_sierra.bottle.tar.gz
🍺  /Users/m/Library/Homebrew/Cellar/xz/5.2.4: 92 files, 1MB
==> Installing python
==> Downloading https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tar.xz
######################################################################## 100.0%
==> ./configure --prefix=/Users/m/Library/Homebrew/Cellar/python/3.6.5 --enable-ipv6 --datarootdir=/Users/m/Library/Homebrew/Cella
==> make
==> make install PYTHONAPPSDIR=/Users/m/Library/Homebrew/Cellar/python/3.6.5
==> make frameworkinstallextras PYTHONAPPSDIR=/Users/m/Library/Homebrew/Cellar/python/3.6.5/share/python
==> Downloading https://files.pythonhosted.org/packages/72/c2/c09362ab29338413ab687b47dab03bab4a792e2bbb727a1eb5e0a88e3b86/setupto
######################################################################## 100.0%
==> Downloading https://files.pythonhosted.org/packages/ae/e8/2340d46ecadb1692a1e455f13f75e596d4eab3d11a57446f08259dee8f02/pip-10.
######################################################################## 100.0%
==> Downloading https://files.pythonhosted.org/packages/5d/c1/45947333669b31bc6b4933308dd07c2aa2fedcec0a95b14eedae993bd449/wheel-0
######################################################################## 100.0%
==> make html
==> /Users/m/Library/Homebrew/Cellar/python/3.6.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-script
==> /Users/m/Library/Homebrew/Cellar/python/3.6.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-script
==> /Users/m/Library/Homebrew/Cellar/python/3.6.5/bin/python3 -s setup.py --no-user-cfg install --force --verbose --install-script
==> Caveats
Python has been installed as
  /Users/m/Library/Homebrew/bin/python3

Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
  /Users/m/Library/Homebrew/opt/python/libexec/bin

If you need Homebrew's Python 2.7 run
  brew install python@2

Pip, setuptools, and wheel have been installed. To update them run
  pip3 install --upgrade pip setuptools wheel

You can install Python packages with
  pip3 install <package>
They will install into the site-package directory
  /Users/m/Library/Homebrew/lib/python3.6/site-packages

See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺  /Users/m/Library/Homebrew/Cellar/python/3.6.5: 9,139 files, 154.7MB, built in 4 minutes 7 seconds
Carbon:~ m$ python3
Python 3.6.5 (default, May 22 2018, 18:03:28)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
Carbon:~ m$ cd `mktemp -d`
Carbon:tmp.2cTF23AS m$ python3
Python 3.6.5 (default, May 22 2018, 18:03:28)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
Carbon:tmp.2cTF23AS m$ python3 -m pip install virtualenv
Collecting virtualenv
  Downloading https://files.pythonhosted.org/packages/b6/30/96a02b2287098b23b875bc8c2f58071c35d2efe84f747b64d523721dc2b5/virtualenv-16.0.0-py2.py3-none-any.whl (1.9MB)
    100% |████████████████████████████████| 1.9MB 9.4MB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-16.0.0
Carbon:tmp.2cTF23AS m$ python3 -m virtualenv .
Using base prefix '/Users/m/Library/Homebrew/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6'
New python executable in /private/var/folders/qx/zz09fcl11v9c1z9w3rh5g_jc0000gn/T/tmp.2cTF23AS/bin/python3.6
Also creating executable in /private/var/folders/qx/zz09fcl11v9c1z9w3rh5g_jc0000gn/T/tmp.2cTF23AS/bin/python
Installing setuptools, pip, wheel...done.
Carbon:tmp.2cTF23AS m$ bin/pip install grpcio
Collecting grpcio
  Downloading https://files.pythonhosted.org/packages/e3/c4/b984569ed8ccca0da7e4798632344243d41240908c338fb4c60d26d0af06/grpcio-1.12.0-cp36-cp36m-macosx_10_7_intel.whl (1.9MB)
    100% |████████████████████████████████| 1.9MB 3.5MB/s
Collecting six>=1.5.2 (from grpcio)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six, grpcio
Successfully installed grpcio-1.12.0 six-1.11.0
Carbon:tmp.2cTF23AS m$ bin/python
Python 3.6.5 (default, May 22 2018, 18:03:28)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import grpc
>>> grpc.insecure_channel('localhost:13000')
<grpc._channel.Channel object at 0x100cffd30>
>>>

[nicolasnoble]

@mehrdada the reporter was talking about anaconda as a distribution method. https://anaconda.org/anaconda/python - I don't think it's fair to use this label if you don't try to reproduce exactly :-)

[ssarangi]

Hi, as I mentioned I was able to reproduce this with both Anaconda as well as virtualenv... I haven't tried python via brew but I use Anaconda as the primary distribution. However, I see the same issue whether i use conda or pip to install grpc.

[mehrdada]

@nicolasnoble @ssarangi I assume the virtualenv was using "plain" Python; that's why I asked for reporter help with the reproduction? Is the virtualenv using Anaconda as well?
I don't believe we actively test for Anaconda or other variants of Python (I'm not familiar with Anaconda at all). If you can only reproduce with Anaconda, I suggest filing the bug with the Python distribution. They can probably either help fix the issue on their side or at least point us to what the delta is between theirs and the "regular" (e.g. brew) Python install.