undefined symbol: _ZTIN4llvm11ObjectCacheE

[ichumuh]

I've upgraded to the newest version (d9bc4a57da57181dff329e5fac2a4e7a384f0ee8) from sources and now symengine.py seems to be broken. I've tried deleted both symengine and symengine.py and tried the following to reinstall symengine.py but it didn't change anything. I didn't see any errors during the installation.

git clone https://github.com/symengine/symengine.git
git clone https://github.com/symengine/symengine.py.git
cd symengine
git checkout `cat ../symengine.py/symengine_version.txt`
cmake -DWITH_LLVM:BOOL=ON .
make
sudo make install
cd ../symengine.py
sudo python setup.py install

If I now try to simply create a symbol the following happens.

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import symengine as sp
>>> sp.Symbols('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Symbols'

And if I try the same in pycharm's console it tells me:

import symengine as sp
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-4c25c92038d8>", line 1, in <module>
    import symengine as sp
  File "/home/stelter/pycharm-2017.2/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 20, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/symengine/__init__.py", line 1, in <module>
    from .lib.symengine_wrapper import (
  File "/home/stelter/pycharm-2017.2/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 20, in do_import
    module = self._system_import(name, *args, **kwargs)
ImportError: /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so: undefined symbol: _ZTIN4llvm11ObjectCacheE

I don't remember my previous symengine.py version. I only remembered that the symengine hash started with fff so I checked out 52ee6b8b14891fe42d82fb10cb705e37d70b1a4a on symengine.py and did the same stuff as above to install symengine.py and now it works again.

Any ideas?

[bjodah]

did you mean sp.Symbol or sp.symbols?

[ichumuh]

I still have the stuff in my terminal, I actually wrote sp.Symbols. But in the next line past me tried

>>> from symengine import Symbol
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name Symbol
>>> 

There was no typo, at least it works right now with the old version.

[ichumuh]

I have noticed that even with the older version, my python interactive shell can't import anything from symengine. However it works if I write a script and execute it.

import symengine as se

a = se.Symbol('a')
print(a*a)

and my output is:

$ python muh.py 
a**2

So I switched back to the latest version and tested running the same script:

$ python muh.py 
Traceback (most recent call last):
  File "muh.py", line 1, in <module>
    import symengine as se
  File "/usr/local/lib/python2.7/dist-packages/symengine/__init__.py", line 1, in <module>
    from .lib.symengine_wrapper import (
ImportError: /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so: undefined symbol: _ZTIN4llvm11ObjectCacheE

Thats the same I got with pycharm so I guess I can rule out my IDE.

The error looks like there is something wrong with llvm so I tried to compile symengine without the -DWITH_LLVM:BOOL=ON option. And now everything works again. Both the script and importing in the interactive shell. Well everything except for my code that compiles expressions with llvm. :(

Edit: my llvm version is 1:4.0-1ubuntu1~16.04.2 and I'm using python 2.7

[bjodah]

You could try to set CMAKE_PREFIX_PATH to include LLVM (especially if you have multiple versions of LLVM on your machine), e.g.:

CMAKE_PREFIX_PATH=/usr/lib/llvm-6.0 cmake ...

look carefully for what LLVM include path gets used. (and then maybe also the symengine dir when building symengine.py):

CMAKE_PREFIX_PATH=/opt/symengine-7b4db85f:/usr/lib/llvm-6.0 python2 setup.py ...

[isuruf]

Can you do ldd /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so and check if there is a libsymengine.so entry and if so, that the correct libsymengine.so is chosen?

[ichumuh]

You could try to set CMAKE_PREFIX_PATH to include LLVM (especially if you have multiple versions of LLVM on your machine)

$ cmake -DWITH_LLVM:BOOL=ON . tells me:

...
LLVM_INCLUDE_DIRS: /usr/lib/llvm-4.0/include
...

which is correct. This is also the only llvm dev version I have installed, should I try 6.0?

look carefully for what LLVM include path gets used. (and then maybe also the symengine dir when building symengine.py):

$ sudo make install says:

...
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/cmake/symengine/SymEngineConfig.cmake
-- Installing: /usr/local/lib/cmake/symengine/SymEngineConfigVersion.cmake
-- Installing: /usr/local/lib/cmake/symengine/SymEngineTargets.cmake
-- Installing: /usr/local/lib/cmake/symengine/SymEngineTargets-release.cmake
-- Installing: /usr/local/lib/libsymengine.a
-- Installing: /usr/local/include/symengine/symengine_config.h
-- Installing: /usr/local/include/symengine/symengine_export.h
-- Installing: /usr/local/include/symengine/add.h
...

and $ sudo python setup.py install says:

running install
running build
running build_ext
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
SymEngine_DIR : /usr/local/lib/cmake/symengine
-- Python include path: /usr/include/python2.7
-- Python version: 2.7
-- Python install path: /usr/local/lib/python2.7/dist-packages
-- Found PYTHON: /usr/lib/x86_64-linux-gnu/libpython2.7.so  
-- Found CYTHON: cython
CMAKE_BUILD_TYPE        : Release
CMAKE_CXX_FLAGS_RELEASE : -Wall -Wextra -Wno-unused-parameter -fno-common -O3 -march=native -ffast-math -funroll-loops -std=c++11 -fPIC -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DNDEBUG -Wno-unknown-pragmas
CMAKE_CXX_FLAGS_DEBUG   : -Wall -Wextra -Wno-unused-parameter -fno-common -g -ggdb -std=c++11 -fPIC -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wno-unknown-pragmas
HAVE_SYMENGINE_MPFR     : False
HAVE_SYMENGINE_MPC      : False
HAVE_SYMENGINE_PIRANHA  : False
HAVE_SYMENGINE_FLINT    : False
HAVE_SYMENGINE_LLVM     : True
Copying source of python wrappers into: /home/stelter/symengine.py/build/lib.linux-x86_64-2.7
-- Configuring done
...

which also looks fine to me.

Can you do ldd /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so and check if there is a libsymengine.so entry and if so, that the correct libsymengine.so is chosen?

$ ldd /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so
	linux-vdso.so.1 =>  (0x00007ffd27d3e000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f770c5c7000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f770c245000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f770bf3c000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f770bd26000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f770b95c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f770cf04000)

no libsymengine it seems.

Edit:
I've just noticed that python setup.py install also says:

-- Generating done
-- Build files have been written to: /home/stelter/symengine.py/build/lib.linux-x86_64-2.7
[ 25%] Cythonizing symengine_wrapper.pyx
Scanning dependencies of target symengine_wrapper
[ 50%] Building CXX object symengine/lib/CMakeFiles/symengine_wrapper.dir/symengine_wrapper.cpp.o
[ 75%] Building CXX object symengine/lib/CMakeFiles/symengine_wrapper.dir/pywrapper.cpp.o
[100%] Linking CXX shared library symengine_wrapper.so
[100%] Built target symengine_wrapper
running install_egg_info
running egg_info
creating symengine.egg-info
writing symengine.egg-info/PKG-INFO
writing top-level names to symengine.egg-info/top_level.txt
writing dependency_links to symengine.egg-info/dependency_links.txt
writing manifest file 'symengine.egg-info/SOURCES.txt'
reading manifest file 'symengine.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.hpp' under directory 'symengine'
warning: no files found matching '*.cmake' under directory 'symengine'
warning: no files found matching '*.pxi' under directory 'symengine'
warning: no files found matching '*.in' under directory 'cmake'
writing manifest file 'symengine.egg-info/SOURCES.txt'
removing '/usr/local/lib/python2.7/dist-packages/symengine-0.3.1.dev1.egg-info' (and everything under it)
Copying symengine.egg-info to /usr/local/lib/python2.7/dist-packages/symengine-0.3.1.dev1.egg-info
running install_scripts
SymEngine_DIR : /usr/local/lib/cmake/symengine
-- Python include path: /usr/include/python2.7
-- Python version: 2.7
-- Python install path: /usr/local/lib/python2.7/dist-packages
-- Found CYTHON: cython
CMAKE_BUILD_TYPE        : Release
CMAKE_CXX_FLAGS_RELEASE : -Wall -Wextra -Wno-unused-parameter -fno-common -O3 -march=native -ffast-math -funroll-loops -std=c++11 -fPIC -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DNDEBUG -Wno-unknown-pragmas
CMAKE_CXX_FLAGS_DEBUG   : -Wall -Wextra -Wno-unused-parameter -fno-common -g -ggdb -std=c++11 -fPIC -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wno-unknown-pragmas
HAVE_SYMENGINE_MPFR     : False
HAVE_SYMENGINE_MPC      : False
HAVE_SYMENGINE_PIRANHA  : False
HAVE_SYMENGINE_FLINT    : False
HAVE_SYMENGINE_LLVM     : True
Copying source of python wrappers into: /home/stelter/symengine.py/build/lib.linux-x86_64-2.7
-- Configuring done
-- Generating done
-- Build files have been written to: /home/stelter/symengine.py/build/lib.linux-x86_64-2.7
[100%] Built target symengine_wrapper
Install the project...
...

This warning: no files found matching '*.hpp' under directory 'symengine' stuff doesn't look good right?

[ichumuh]

I've uninstalled llvm4.0-dev, installed 6.0 and rebuild everything. The problem persists, but now the python shell also complains about this stuff:

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import symengien as se
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named symengien
>>> import symengine as se
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/symengine/__init__.py", line 1, in <module>
    from .lib.symengine_wrapper import (
ImportError: /usr/local/lib/python2.7/dist-packages/symengine/lib/symengine_wrapper.so: undefined symbol: _ZTIN4llvm11ObjectCacheE

All the stuff I wrote in my previous post still looks the same except that llvm4.0 is replaced with 6.0

Edit: I've checked on a second computer where symengine was never installed. Same problem.

[bjodah]

You need to compile symengine with -DBUILD_SHARED_LIBS:BOOL=ON
(edit: or at least I think so, here's what I'm doing in a Dockerfile)
(edit2: if this is the issue we should probably update the README in symengine.py to point this out)

[ichumuh]

That fixed it. Thanks a lot!
Still weird that it worked without the option with the older version though.