Python import fails with "PyThreadState_Get: no current thread"

[tlhr]

• I believe this to be a bug with Open Babel
• This is a feature request

Environment Information

Open Babel version: 2.4.1-py36_3
Operating system and version: macOS 10.13.1 (17B48)

Steps to Reproduce

Install the conda package and import openbabel:
conda install -c openbabel openbabel

$ python3 -c "import openbabel"
Fatal Python error: PyThreadState_Get: no current thread

I am aware that this error results from conflicting python versions, but I can't see how this can happen with a conda build. Here is the output from otool -L:

otool -L ~/anaconda3/lib/python3.6/site-packages/_openbabel.so                                                                        ~
/Users/thomas/anaconda3/lib/python3.6/site-packages/_openbabel.so:
	@rpath/libpython3.6m.dylib (compatibility version 3.6.0, current version 3.6.0)
	@rpath/libopenbabel.5.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	@rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 104.1.0)

This seems to me to be the correct libpython, I am also running this with the correct python version.
I have also tried building openbabel myself with the same result, this was the cmake command:

cmake .. -DPYTHON_BINDINGS=ON -DPYTHON_PREFIX=~/anaconda3/lib/python3.6/site-packages -DPYTHON_INCLUDE_DIR=~/anaconda3/include/python3.6m -DPYTHON_LIBRARY=~/anaconda3/lib/libpython3.6m.dylib -DRUN_SWIG=ON -DBUILD_GUI=ON

[baoilleach]

In your report above you did not describe activating the conda environment. Did you do this?

[tlhr]

I can not seem to reproduce this now (no activation of the conda environment), and the import works as expected.

[baoilleach]

No worries - good to hear.

[tlhr]

After updating to macOS 10.13.2 I'm getting this error again. I can also reproduce it in a fresh conda install on macOS 10.12.6.

[ghutchis]

The error comes from a mismatch between the python executable and the libpython that's linked to the _openbabel.so for the module. While I understand your complaint, if you upgrade (e.g., a "fresh conda install") you need to make sure that _openbabel.so is pointing at the right libpython.

otool -L /path/to/_openbabel.so

I have no problems with this, either on Mac OS 10.13.2 or 10.12.6, with and without conda. These are my development environment.

[ghutchis]

Oh, and please make sure it's the right python which python

[tlhr]

Thank you for your reply @ghutchis! I have now been able to reproduce this on three different machines, in two cases there were no previous conda installations. This is the output from otool -L:

$ otool -L /anaconda/lib/python3.6/site-packages/_openbabel.so
/anaconda/lib/python3.6/site-packages/_openbabel.so:
	@rpath/libpython3.6m.dylib (compatibility version 3.6.0, current version 3.6.0)
	@rpath/libopenbabel.5.dylib (compatibility version 5.0.0, current version 5.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	@rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 104.1.0)

I have tried explicitly replacing the paths with absolute ones to the libraries provided by conda (with install_name_tool), but I'm getting the same error.

$ which python3
/anaconda/bin/python3
$ python3 --version
Python 3.6.3 :: Anaconda custom (64-bit)
$ python3 -c "import openbabel"
Fatal Python error: PyThreadState_Get: no current thread

zsh: abort      python3 -c "import openbabel"

I don't think my PYTHONPATH has any problematic content:

echo $PYTHONPATH
/Users/thomas/opt/openmm-plumed/lib:/Users/thomas/opt/psi4/lib:

[gthh2]

I have reproduced @tlhr 's error on MacOS 10.12.6 with a fresh install of conda.

[tlhr]

@ghutchis could you show me your output from otool -L? That would help me track down the issue. Thanks!

[tlhr]

OK, this seems to be a bug in conda, because I'm getting the same error when trying to use rdkit. See here: rdkit/rdkit#1617
A workaround that worked for me was to downgrade the python version to 3.6.1:
conda install python=3.6.1
Sorry about that!