Installation fails on OSX with Python 3.5

[metachris]

Hey all,

I'm working on https://github.com/CityOfZion/neo-python and we're using plyvel as DB. The problem is that the plyvel installation from PyPI does not work out of the box on OSX, we're using Python 3.5.

File "/project-dir/neo/Implementations/Blockchains/LevelDB/LevelDBBlockchain.py", line 2, in <module>
    import plyvel
File "/project-dir/venv/lib/python3.5/site-packages/plyvel/__init__.py", line 6, in <module>
    from ._plyvel import (  # noqa
ImportError: dlopen(/project-dir/venv/lib/python3.5/site-packages/plyvel/_plyvel.cpython-35m-darwin.so, 2): Symbol not found: __ZN7leveldb2DB4OpenERKNS_7OptionsERKSsPPS0_
Referenced from: /project-dir/venv/lib/python3.5/site-packages/plyvel/_plyvel.cpython-35m-darwin.so
Expected in: flat namespace
in /project-dir/venv/lib/python3.5/site-packages/plyvel/_plyvel.cpython-35m-darwin.so

To fix this, we need to do these steps:

pip uninstall plyvel
CFLAGS='-mmacosx-version-min=10.7 -stdlib=libc++' pip install --no-use-wheel plyvel --no-cache-dir --global-option=build_ext --global-option="-I/usr/local/Cellar/leveldb/1.20_2/include/" --global-option="-L/usr/local/lib"

Is there any chance we can fix it in this repository?

[wbolster]

currently there are no osx wheels on pypi, so --no-use-wheel at best avoids a local cache. try removing ~/.cache/pip/

[metachris]

Just double checked. For it to work on OSX, the minimum changes needed are those CFLAGS:

CFLAGS='-mmacosx-version-min=10.7 -stdlib=libc++' pip install plyvel

This is the minimum setup I need for the plyvel installation to not show the Symbol not found error reported above. Is there any way to add this as OSX-specific build arguments?

[wbolster]

this looks like

https://stackoverflow.com/questions/23905661/on-mac-g-clang-fails-to-search-usr-local-include-and-usr-local-lib-by-def

...oh wait, you say it does actually search /usr/local/include.

not sure how to add osx specific build args. if you figure out the setup.py magic, let me know. :)

[wbolster]

https://github.com/kgori/treeCl/blob/master/setup.py#L25-L34

might give inspiration

[metachris]

Give me a second, think i almost got it solved

[wbolster]

awesome! i also found https://github.com/pybind/python_example/blob/master/setup.py btw

[metachris]

I think does the trick for me. Would be great to verify this by other OSX users. (requires to have leveldb installed with homebrew - brew install leveldb)

[wbolster]

https://plyvel.readthedocs.io/en/latest/news.html#plyvel-1-0-3

[wbolster]

fwiw, i had a report from an osx user that got errors like this when running a plain pip install plyvel without setting env vars:

  plyvel/_plyvel.cpp:547:10: fatal error: 'leveldb/db.h' file not found
  #include "leveldb/db.h"

the suggestion from
https://stackoverflow.com/a/36165586
seemed to help fixing this:

$ sudo rm -rf /Applications/Xcode.app

(leaving this comment here since it may help others)

[jeffignacio]

Hi @wbolster, @metachris I'm also working on installing [https://github.com/CityOfZion/neo-python] but am having issues installing plyvel as well. I'm seeing this error down below. I've followed this thread as well as issue #34 but am having a hard time solving.

Command "/Users/user/Documents/neo-python-master/venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/k0/4_v8w8zd4rx9gpqkxmk_d29r0000gn/T/pip-build-hlda1ua_/Plyvel/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext -I/usr/local/include -L/usr/local/lib install --record /var/folders/k0/4_v8w8zd4rx9gpqkxmk_d29r0000gn/T/pip-owzbm2nl-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/user/Documents/neo-python-master/venv/include/site/python3.5/Plyvel" failed with error code 1 in /private/var/folders/k0/4_v8w8zd4rx9gpqkxmk_d29r0000gn/T/pip-build-hlda1ua_/Plyvel/

[wbolster]

not sure, the actual error message seems missing from your comment...

[ralexstokes]

ran into this problem and this did fix it so 👍 .

however, if you want to keep Xcode lying around, you can just temporarily move it while you install plyvel

e.g. sudo mv /Applications/Xcode.app /Applications/Xfoo.app, then you can rename it after install

unclear why XCode clobbers the header search path, but it does seem to

fwiw, i had a report from an osx user that got errors like this when running a plain pip install plyvel without setting env vars:

  plyvel/_plyvel.cpp:547:10: fatal error: 'leveldb/db.h' file not found
  #include "leveldb/db.h"

the suggestion from
https://stackoverflow.com/a/36165586
seemed to help fixing this:

$ sudo rm -rf /Applications/Xcode.app

(leaving this comment here since it may help others)

[ra0x3]

Using brew install leveldb as a pre-req, the following worked for me on OSX 10.13 using Python3.7. It's really just a mash up of all the "worked for me" solutions in the thread (none of which worked for me on their own, but combined, they did)

mv /Applications/Xcode.app /Applications/Xcode_cp.app
ls -la /Applications/ | grep Xcode*

leveldb_version=$(ls /usr/local/Cellar/leveldb/ | tail -1)
CFLAGS="-mmacosx-version-min=10.7 -stdlib=libc++" \
pip install plyvel \
--no-cache-dir \
--global-option=build_ext \
--global-option="-I/usr/local/Cellar/leveldb/${leveldb_version}/include/" \
--global-option="-L/usr/local/lib"

mv /Applications/XCode_cp.app /Applications/Xcode.app
ls -la /Applications/ | grep Xcode*

[patricksurry]

fwiw, I ran into similar compilation problems in pip install plyvel with python 3.6 on Mac OSX 10.14, and was getting strange 'C++11 extension' warnings after I'd done brew install leveldb. Adding the -std=c++11 flag to the recipe above worked for me:

CFLAGS='-mmacosx-version-min=10.7 -stdlib=libc++ -std=c++11' pip install plyvel

[croqaz]

This still doesn't work for me, I tried all the methods 😢
I have leveldb installed from brew.
I don't have the /Applications/Xcode.app and nothing similar.
xcode-select -p shows /Library/Developer/CommandLineTools
I have 3 python versions:

• 2.7 from Mac - useless
• 3.7 from brew - useless
• 3.6 from sashkab/python tap, I need Plyvel here, because I really need this specific version and I'm using a virtualenv.

Maybe other people have the same issue in a similar situation.
I also tried pyenv with python 3.6 but the error is exactly the same.

[StuartFarmer]

fwiw, I ran into similar compilation problems in pip install plyvel with python 3.6 on Mac OSX 10.14, and was getting strange 'C++11 extension' warnings after I'd done brew install leveldb. Adding the -std=c++11 flag to the recipe above worked for me:

CFLAGS='-mmacosx-version-min=10.7 -stdlib=libc++ -std=c++11' pip install plyvel

THANK YOU. This works after a brew install leveldb on Python 3.6 with a pip3 install globally.

[schmty]

@StuartFarmer This issue has been resolved on a recent PR.

@wbolster I've really enjoyed using your package and I have gone ahead and prepared a new version in PR #99 and I'm willing to help in any way I can.