Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing a package? Failed Install #122

Closed
brianfm opened this issue Jan 26, 2017 · 30 comments
Closed

Missing a package? Failed Install #122

brianfm opened this issue Jan 26, 2017 · 30 comments

Comments

@brianfm
Copy link

brianfm commented Jan 26, 2017

I just installed anaconda from a clean install and installed blosc using pip install blosc. I am now having this import error:

In [1]: import blosc
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
in ()
----> 1 import blosc

/usr/local/anaconda/lib/python2.7/site-packages/blosc/init.py in ()
11
12 # Blosc C symbols that we want to export
---> 13 from blosc.blosc_extension import (
14 BLOSC_VERSION_STRING as VERSION_STRING,
15 BLOSC_VERSION_DATE as VERSION_DATE,

ImportError: /usr/local/anaconda/lib/python2.7/site-packages/blosc/blosc_extension.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm

I am running Linux Mint 18 Sarah 64-bit Kernel Linux 4.4.0-21-generic x86_64 MATE 1.14.1
Am I missing a package when installing from pip? Thank you.

@esc
Copy link
Member

esc commented Jan 26, 2017 via email

@esc
Copy link
Member

esc commented Jan 26, 2017 via email

@brianfm
Copy link
Author

brianfm commented Jan 26, 2017

I'm fairly confident it's not anaconda to blame.

@esc
Copy link
Member

esc commented Jan 26, 2017

But did you actually try it? ;)

@brianfm
Copy link
Author

brianfm commented Jan 26, 2017

I just tried it with the system python with the same error. pip is not compiling locally its just copying the so files into the site packages directory

@esc
Copy link
Member

esc commented Jan 26, 2017

That's fairly unlikely since there are no binary packages (i.e. no package that contains a *.so file) on PyPi: https://pypi.python.org/pypi/blosc/1.4.4 -- As you can see there is only a source package. This means the package must be compiled locally in order for you to obtain a *.so file.

@esc
Copy link
Member

esc commented Jan 26, 2017

Perhaps pip is caching the results of a previous compilation? Then try using the pip options that disable cache. --no-cache-dir is your friend.

@esc
Copy link
Member

esc commented Jan 26, 2017

Also, which version of pip are you using, pip --version will show that

@brianfm
Copy link
Author

brianfm commented Jan 26, 2017

I removed the cache:
sudo rm ~/.cache/pip

Ran:
sudo pip install blosc --ignore-installed

Output:
Collecting blosc
Downloading blosc-1.4.4.tar.gz (613kB)
100% |████████████████████████████████| 614kB 1.8MB/s
Installing collected packages: blosc
Running setup.py install for blosc ... done
Successfully installed blosc-1.4.4

and in ipython:
In [1]: import blosc

ImportError Traceback (most recent call last)
in ()
----> 1 import blosc

/usr/local/anaconda/lib/python2.7/site-packages/blosc/init.py in ()
11
12 # Blosc C symbols that we want to export
---> 13 from blosc.blosc_extension import (
14 BLOSC_VERSION_STRING as VERSION_STRING,
15 BLOSC_VERSION_DATE as VERSION_DATE,

ImportError: /usr/local/anaconda/lib/python2.7/site-packages/blosc/blosc_extension.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm

Is there a bug in setup.py?

My pip version:
pip 9.0.1

@esc
Copy link
Member

esc commented Jan 26, 2017

Hmmm:

esc@omnitech /tmp % virtualenv venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in /tmp/venv/bin/python2
Also creating executable in /tmp/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
esc@omnitech /tmp % source .venv/bin/activate
esc@omnitech /tmp % pip install blosc
Collecting blosc
  Using cached blosc-1.4.4.tar.gz
Building wheels for collected packages: blosc
  Running setup.py bdist_wheel for blosc ... done
  Stored in directory: /home/esc/.cache/pip/wheels/66/b5/8e/3e67ed59865256f2c3c5d5a914069e1b7f6ef61f4086e98c43
Successfully built blosc
Installing collected packages: blosc
Successfully installed blosc-1.4.4
pip install blosc  15,33s user 0,81s system 97% cpu 16,483 total
esc@omnitech /tmp % python -c "import blosc; print(blosc.__version__)"
1.4.4

It does work for me, so probably nothing to do with the setup.py

@esc
Copy link
Member

esc commented Jan 26, 2017

Here is one with an empty cache:

esc@omnitech /tmp % virtualenv venv
Running virtualenv with interpreter /usr/bin/python2
New python executable in /tmp/venv/bin/python2
Not overwriting existing python script /tmp/venv/bin/python (you must use /tmp/venv/bin/python2)
Installing setuptools, pkg_resources, pip, wheel...done.
esc@omnitech /tmp % source venv/bin/activate 
esc@omnitech /tmp % pip install -U pip
Requirement already up-to-date: pip in ./venv/lib/python2.7/site-packages
esc@omnitech /tmp % pip --version
pip 9.0.1 from /tmp/venv/local/lib/python2.7/site-packages (python 2.7)
esc@omnitech /tmp % pip install blosc
Collecting blosc
  Downloading blosc-1.4.4.tar.gz (613kB)
    100% |████████████████████████████████| 614kB 975kB/s 
Building wheels for collected packages: blosc
  Running setup.py bdist_wheel for blosc ... done
  Stored in directory: /home/esc/.cache/pip/wheels/66/b5/8e/3e67ed59865256f2c3c5d5a914069e1b7f6ef61f4086e98c43
Successfully built blosc
Installing collected packages: blosc
Successfully installed blosc-1.4.4
pip install blosc  15,96s user 0,80s system 94% cpu 17,706 total
esc@omnitech /tmp % python -c "import blosc; print(blosc.__version__)"
1.4.4

@robbmcleod
Copy link
Contributor

robbmcleod commented Jan 27, 2017 via email

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

I figured it was the linking as well. I will try again from tar ball.

To answer your CMake question:
$ cmake --version
cmake version 3.5.1

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

On python-blosc-1.4.4.tar.gz I opened and ran:

python setup.py build_ext --inplace

python setup.py install
SSE2 detected
running install
running bdist_egg
running egg_info
creating blosc.egg-info
writing blosc.egg-info/PKG-INFO
writing top-level names to blosc.egg-info/top_level.txt
writing dependency_links to blosc.egg-info/dependency_links.txt
writing manifest file 'blosc.egg-info/SOURCES.txt'
reading manifest file 'blosc.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.cpp' under directory 'c-blosc'
warning: no files found matching '
.hpp' under directory 'c-blosc'
writing manifest file 'blosc.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying blosc/toplevel.py -> build/lib.linux-x86_64-2.7/blosc
copying blosc/init.py -> build/lib.linux-x86_64-2.7/blosc
copying blosc/test.py -> build/lib.linux-x86_64-2.7/blosc
copying blosc/version.py -> build/lib.linux-x86_64-2.7/blosc
running build_ext
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/toplevel.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/init.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/blosc_extension.so -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/test.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/version.py -> build/bdist.linux-x86_64/egg/blosc
byte-compiling build/bdist.linux-x86_64/egg/blosc/toplevel.py to toplevel.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/test.py to test.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/version.py to version.pyc
creating stub loader for blosc/blosc_extension.so
byte-compiling build/bdist.linux-x86_64/egg/blosc/blosc_extension.py to blosc_extension.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/blosc-1.4.4-py2.7-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing blosc-1.4.4-py2.7-linux-x86_64.egg
Copying blosc-1.4.4-py2.7-linux-x86_64.egg to /usr/local/anaconda/lib/python2.7/site-packages
Adding blosc 1.4.4 to easy-install.pth file

Installed /usr/local/anaconda/lib/python2.7/site-packages/blosc-1.4.4-py2.7-linux-x86_64.egg
Processing dependencies for blosc==1.4.4
Finished processing dependencies for blosc==1.4.4

Tried test.py:

python test.py
Traceback (most recent call last):
File "test.py", line 7, in
import blosc
File "/usr/local/anaconda/lib/python2.7/site-packages/blosc/init.py", line 13, in
from blosc.blosc_extension import (
ImportError: /usr/local/anaconda/lib/python2.7/site-packages/blosc/blosc_extension.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm

@robbmcleod
Copy link
Contributor

robbmcleod commented Jan 27, 2017 via email

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

cmake -DCMAKE_INSTALL_PREFIX="/usr/local"
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Configuring for Blosc version: 1.11.1
-- Using LZ4 internal sources.
-- No snappy found. Using internal sources.
-- Found ZLIB: /usr/local/anaconda/lib/libz.so (found version "1.2.8")
-- No Zstd library found. Using internal sources.
-- No build type specified. Defaulting to 'Release'.
-- Building for system processor x86_64
-- Adding run-time support for SSE2
-- Adding run-time support for AVX2
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Configuring done
CMake Warning (dev) at blosc/CMakeLists.txt:191 (add_library):
Policy CMP0063 is not set: Honor visibility properties for all target
types. Run "cmake --help-policy CMP0063" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.

Target "blosc_static" of type "STATIC_LIBRARY" has the following visibility
properties set for C:

C_VISIBILITY_PRESET

For compatibility CMake is not honoring them for this target.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: ~/Downloads/python-blosc-1.4.4/c-blosc

~/Downloads/python-blosc-1.4.4/c-blosc $ cmake --build .
Scanning dependencies of target blosc_static
[ 1%] Building C object blosc/CMakeFiles/blosc_static.dir/blosc.c.o
[ 2%] Building C object blosc/CMakeFiles/blosc_static.dir/blosclz.c.o
[ 3%] Building C object blosc/CMakeFiles/blosc_static.dir/shuffle-generic.c.o
[ 4%] Building C object blosc/CMakeFiles/blosc_static.dir/bitshuffle-generic.c.o
[ 5%] Building C object blosc/CMakeFiles/blosc_static.dir/shuffle-sse2.c.o
[ 6%] Building C object blosc/CMakeFiles/blosc_static.dir/bitshuffle-sse2.c.o
[ 7%] Building C object blosc/CMakeFiles/blosc_static.dir/shuffle-avx2.c.o
[ 7%] Building C object blosc/CMakeFiles/blosc_static.dir/bitshuffle-avx2.c.o
[ 8%] Building C object blosc/CMakeFiles/blosc_static.dir/shuffle.c.o
[ 9%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/lz4-1.7.2/lz4hc.c.o
[ 10%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/lz4-1.7.2/lz4.c.o
[ 11%] Building CXX object blosc/CMakeFiles/blosc_static.dir//internal-complibs/snappy-1.1.1/snappy-c.cc.o
[ 12%] Building CXX object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/snappy-1.1.1/snappy-stubs-internal.cc.o
[ 13%] Building CXX object blosc/CMakeFiles/blosc_static.dir//internal-complibs/snappy-1.1.1/snappy.cc.o
[ 14%] Building CXX object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/snappy-1.1.1/snappy-sinksource.cc.o
[ 15%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/common/fse_decompress.c.o
[ 16%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/zstd-1.0.0/common/entropy_common.c.o
[ 17%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/common/xxhash.c.o
[ 18%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/zstd-1.0.0/common/zstd_common.c.o
[ 19%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/compress/fse_compress.c.o
[ 20%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/zstd-1.0.0/compress/huf_compress.c.o
[ 21%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/compress/zstd_compress.c.o
[ 22%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/zstd-1.0.0/compress/zbuff_compress.c.o
[ 23%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/decompress/zstd_decompress.c.o
[ 24%] Building C object blosc/CMakeFiles/blosc_static.dir/
/internal-complibs/zstd-1.0.0/decompress/huf_decompress.c.o
[ 25%] Building C object blosc/CMakeFiles/blosc_static.dir//internal-complibs/zstd-1.0.0/decompress/zbuff_decompress.c.o
[ 26%] Linking CXX static library libblosc.a
[ 26%] Built target blosc_static
Scanning dependencies of target blosc_shared_testing
[ 27%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/blosc.c.o
[ 28%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/blosclz.c.o
[ 29%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/shuffle-generic.c.o
[ 30%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/bitshuffle-generic.c.o
[ 31%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/shuffle-sse2.c.o
[ 32%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/bitshuffle-sse2.c.o
[ 33%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/shuffle-avx2.c.o
[ 34%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/bitshuffle-avx2.c.o
[ 35%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/shuffle.c.o
[ 36%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/lz4-1.7.2/lz4hc.c.o
[ 37%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/lz4-1.7.2/lz4.c.o
[ 38%] Building CXX object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/snappy-1.1.1/snappy-c.cc.o
[ 39%] Building CXX object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/snappy-1.1.1/snappy-stubs-internal.cc.o
[ 39%] Building CXX object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/snappy-1.1.1/snappy.cc.o
[ 40%] Building CXX object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/snappy-1.1.1/snappy-sinksource.cc.o
[ 41%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/common/fse_decompress.c.o
[ 42%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/zstd-1.0.0/common/entropy_common.c.o
[ 43%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/common/xxhash.c.o
[ 44%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/zstd-1.0.0/common/zstd_common.c.o
[ 45%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/compress/fse_compress.c.o
[ 46%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/zstd-1.0.0/compress/huf_compress.c.o
[ 47%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/compress/zstd_compress.c.o
[ 48%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/zstd-1.0.0/compress/zbuff_compress.c.o
[ 49%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/decompress/zstd_decompress.c.o
[ 50%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir//internal-complibs/zstd-1.0.0/decompress/huf_decompress.c.o
[ 51%] Building C object blosc/CMakeFiles/blosc_shared_testing.dir/
/internal-complibs/zstd-1.0.0/decompress/zbuff_decompress.c.o
[ 52%] Linking CXX shared library libblosc_testing.so
[ 52%] Built target blosc_shared_testing
Scanning dependencies of target blosc_shared
[ 53%] Building C object blosc/CMakeFiles/blosc_shared.dir/blosc.c.o
[ 54%] Building C object blosc/CMakeFiles/blosc_shared.dir/blosclz.c.o
[ 55%] Building C object blosc/CMakeFiles/blosc_shared.dir/shuffle-generic.c.o
[ 56%] Building C object blosc/CMakeFiles/blosc_shared.dir/bitshuffle-generic.c.o
[ 57%] Building C object blosc/CMakeFiles/blosc_shared.dir/shuffle-sse2.c.o
[ 58%] Building C object blosc/CMakeFiles/blosc_shared.dir/bitshuffle-sse2.c.o
[ 59%] Building C object blosc/CMakeFiles/blosc_shared.dir/shuffle-avx2.c.o
[ 60%] Building C object blosc/CMakeFiles/blosc_shared.dir/bitshuffle-avx2.c.o
[ 61%] Building C object blosc/CMakeFiles/blosc_shared.dir/shuffle.c.o
[ 62%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/lz4-1.7.2/lz4hc.c.o
[ 63%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/lz4-1.7.2/lz4.c.o
[ 64%] Building CXX object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/snappy-1.1.1/snappy-c.cc.o
[ 65%] Building CXX object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/snappy-1.1.1/snappy-stubs-internal.cc.o
[ 66%] Building CXX object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/snappy-1.1.1/snappy.cc.o
[ 67%] Building CXX object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/snappy-1.1.1/snappy-sinksource.cc.o
[ 68%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/zstd-1.0.0/common/fse_decompress.c.o
[ 69%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/zstd-1.0.0/common/entropy_common.c.o
[ 70%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/zstd-1.0.0/common/xxhash.c.o
[ 71%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/zstd-1.0.0/common/zstd_common.c.o
[ 71%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/zstd-1.0.0/compress/fse_compress.c.o
[ 72%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/zstd-1.0.0/compress/huf_compress.c.o
[ 73%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/zstd-1.0.0/compress/zstd_compress.c.o
[ 74%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/zstd-1.0.0/compress/zbuff_compress.c.o
[ 75%] Building C object blosc/CMakeFiles/blosc_shared.dir//internal-complibs/zstd-1.0.0/decompress/zstd_decompress.c.o
[ 76%] Building C object blosc/CMakeFiles/blosc_shared.dir/
/internal-complibs/zstd-1.0.0/decompress/huf_decompress.c.o
[ 77%] Building C object blosc/CMakeFiles/blosc_shared.dir/__/internal-complibs/zstd-1.0.0/decompress/zbuff_decompress.c.o
[ 78%] Linking CXX shared library libblosc.so
[ 78%] Built target blosc_shared
Scanning dependencies of target test_shuffle_roundtrip_avx2
[ 79%] Building C object tests/CMakeFiles/test_shuffle_roundtrip_avx2.dir/test_shuffle_roundtrip_avx2.c.o
~/Downloads/python-blosc-1.4.4/c-blosc/tests/test_shuffle_roundtrip_avx2.c:25:4: warning: #warning AVX2 shuffle tests not enabled. [-Wcpp]
#warning AVX2 shuffle tests not enabled.
^
[ 80%] Linking C executable test_shuffle_roundtrip_avx2
[ 80%] Built target test_shuffle_roundtrip_avx2
Scanning dependencies of target test_shuffle_roundtrip_generic
[ 81%] Building C object tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/test_shuffle_roundtrip_generic.c.o
[ 82%] Linking C executable test_shuffle_roundtrip_generic
~/Downloads/python-blosc-1.4.4/c-blosc/blosc/libblosc_testing.so: undefined reference to std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21' ~/Downloads/python-blosc-1.4.4/c-blosc/blosc/libblosc_testing.so: undefined reference to std::__cxx11::basic_string<char, std::char_traits, std::allocator >::resize(unsigned long, char)@GLIBCXX_3.4.21'
collect2: error: ld returned 1 exit status
tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/build.make:94: recipe for target 'tests/test_shuffle_roundtrip_generic' failed
make[2]: *** [tests/test_shuffle_roundtrip_generic] Error 1
CMakeFiles/Makefile2:307: recipe for target 'tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/all' failed
make[1]: *** [tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

(I confirmed using gcc 5.4.0 and cmake 3.5.1)

@robbmcleod
Copy link
Contributor

robbmcleod commented Jan 27, 2017

Ok, so we see the error now, and as suspected it didn't get linked after compilation. Somehow glibcxx.3.4.21.lib isn't being linked to. I suspect either your system environment variables have been overwritten at some point, or one of your library files has been corrupted. Try:

export LD_LIBRARY_PATH=" /usr/lib/gcc/x86_64-linux-gnu/5.4.0:$LD_LIBRARY_PATH"
cmake .

from the python-blosc-1.4.4/c-blosc directory? I don't have Mint so that path may not be correct, verify that it exists first.

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

From my .bashrc:

export LD_LIBRARY_PATH=/usr/local/lib:/usr/bin:/usr/lib/gcc/x86_64-linux-gnu/5.4.0:$LD_LIBRARY_PATH

~/Downloads/python-blosc-1.4.4/c-blosc $ cmake .
Configuring for Blosc version: 1.11.1
-- Using LZ4 internal sources.
-- No snappy found. Using internal sources.
-- No Zstd library found. Using internal sources.
-- Building for system processor x86_64
-- Adding run-time support for SSE2
-- Adding run-time support for AVX2
-- Configuring done
CMake Warning (dev) at blosc/CMakeLists.txt:191 (add_library):
Policy CMP0063 is not set: Honor visibility properties for all target
types. Run "cmake --help-policy CMP0063" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.

Target "blosc_static" of type "STATIC_LIBRARY" has the following visibility
properties set for C:

C_VISIBILITY_PRESET

For compatibility CMake is not honoring them for this target.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: ~/Downloads/python-blosc-1.4.4/c-blosc

~/Downloads/python-blosc-1.4.4/c-blosc $ cmake --build .
[ 26%] Built target blosc_static
[ 52%] Built target blosc_shared_testing
[ 78%] Built target blosc_shared
[ 80%] Built target test_shuffle_roundtrip_avx2
[ 81%] Linking C executable test_shuffle_roundtrip_generic
~/Downloads/python-blosc-1.4.4/c-blosc/blosc/libblosc_testing.so: undefined reference to std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_append(char const*, unsigned long)@GLIBCXX_3.4.21' ~/Downloads/python-blosc-1.4.4/c-blosc/blosc/libblosc_testing.so: undefined reference to std::__cxx11::basic_string<char, std::char_traits, std::allocator >::resize(unsigned long, char)@GLIBCXX_3.4.21'
collect2: error: ld returned 1 exit status
tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/build.make:94: recipe for target 'tests/test_shuffle_roundtrip_generic' failed
make[2]: *** [tests/test_shuffle_roundtrip_generic] Error 1
CMakeFiles/Makefile2:307: recipe for target 'tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/all' failed
make[1]: *** [tests/CMakeFiles/test_shuffle_roundtrip_generic.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

@esc
Copy link
Member

esc commented Jan 27, 2017

IIRC the setup.py will not invoke CMake (but might be mistaken)

@FrancescAlted
Copy link
Member

Yeah, but CMake failing is an indication that something is not working properly in OP machine. As @robbmcleod said, this smells like an issue with the C++ libraries (which is quite strange). @brianfm could you try with disabling snappy (the only codec that needs C++) with:

cmake -DDEACTIVATE_SNAPPY=ON .

If that works, I suggest to comment this line and set HAVE_SNAPPY=0 in this other line?

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

Hello - thank you all again for the help.

I just ran:
cmake -DDEACTIVATE_SNAPPY=ON .
cmake --build .

And it finished without error (Built target bench). Should I now cmake --build . --target install? Or go back, comment out the code you have set aside, and run from setup.py?

Thank you.

@FrancescAlted
Copy link
Member

Yes, you can do both. If you opt by installing the C library in the system, then you can tell the python installer where to find the library with:

$ python setup.py build_ext --blosc=installed_blosc_dir

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

cmake -DDEACTIVATE_SNAPPY=ON -DCMAKE_INSTALL_PREFIX="/usr/lib" .

ctest:
100% tests passed, 0 tests failed out of 1613
Total Test time (real) = 93.00 sec

cmake --build . --target install
[ 24%] Built target blosc_static
[ 49%] Built target blosc_shared_testing
[ 74%] Built target blosc_shared
[ 76%] Built target test_shuffle_roundtrip_avx2
[ 78%] Built target test_shuffle_roundtrip_generic
[ 80%] Built target test_shuffle_roundtrip_sse2
[ 82%] Built target test_nolock
[ 84%] Built target test_getitem
[ 87%] Built target test_maxout
[ 89%] Built target test_noinit
[ 91%] Built target test_compressor
[ 93%] Built target test_nthreads
[ 95%] Built target test_api
[ 97%] Built target test_compress_roundtrip
[100%] Built target bench
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /usr/local/include/blosc.h
-- Up-to-date: /usr/local/include/blosc-export.h
-- Up-to-date: /usr/local/lib/libblosc.so.1.11.1
-- Up-to-date: /usr/local/lib/libblosc.so.1
-- Up-to-date: /usr/local/lib/libblosc.so
-- Up-to-date: /usr/local/lib/libblosc.a

python setup.py build_ext --blosc="/usr/local"
running build_ext

python setup.py install
SSE2 detected
running install
running bdist_egg
running egg_info
writing blosc.egg-info/PKG-INFO
writing top-level names to blosc.egg-info/top_level.txt
writing dependency_links to blosc.egg-info/dependency_links.txt
reading manifest file 'blosc.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.hpp' under directory 'c-blosc'
writing manifest file 'blosc.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying blosc/version.py -> build/lib.linux-x86_64-2.7/blosc
running build_ext
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/toplevel.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/init.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/blosc_extension.so -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/test.py -> build/bdist.linux-x86_64/egg/blosc
copying build/lib.linux-x86_64-2.7/blosc/version.py -> build/bdist.linux-x86_64/egg/blosc
byte-compiling build/bdist.linux-x86_64/egg/blosc/toplevel.py to toplevel.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/test.py to test.pyc
byte-compiling build/bdist.linux-x86_64/egg/blosc/version.py to version.pyc
creating stub loader for blosc/blosc_extension.so
byte-compiling build/bdist.linux-x86_64/egg/blosc/blosc_extension.py to blosc_extension.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying blosc.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating 'dist/blosc-1.4.4-py2.7-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing blosc-1.4.4-py2.7-linux-x86_64.egg
Removing /usr/local/anaconda/lib/python2.7/site-packages/blosc-1.4.4-py2.7-linux-x86_64.egg
Copying blosc-1.4.4-py2.7-linux-x86_64.egg to /usr/local/anaconda/lib/python2.7/site-packages
blosc 1.4.4 is already the active version in easy-install.pth

Installed /usr/local/anaconda/lib/python2.7/site-packages/blosc-1.4.4-py2.7-linux-x86_64.egg
Processing dependencies for blosc==1.4.4
Finished processing dependencies for blosc==1.4.4

python -c "import blosc; print(blosc.version)"
Traceback (most recent call last):
File "", line 1, in
File "blosc/init.py", line 13, in
from blosc.blosc_extension import (
ImportError: blosc/blosc_extension.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm

@FrancescAlted
Copy link
Member

Mmh, you should either remove the old blosc/blosc_extension.so or go out from the python-blosc/ source dir before trying to import it.

@brianfm
Copy link
Author

brianfm commented Jan 27, 2017

That makes sense. I went out and deleted and cleared cache. After clearing cache, I ran:
python -c "import blosc; print(blosc.version)"

Output:
Traceback (most recent call last):
File "", line 1, in
File "build/bdist.linux-x86_64/egg/blosc/init.py", line 13, in

File "build/bdist.linux-x86_64/egg/blosc/blosc_extension.py", line 7, in
File "build/bdist.linux-x86_64/egg/blosc/blosc_extension.py", line 6, in bootstrap
ImportError: /home/bmccarthy/.cache/Python-Eggs/blosc-1.4.4-py2.7-linux-x86_64.egg-tmp/blosc/blosc_extension.so: undefined symbol: _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE9_M_appendEPKcm

Why is it picking up on the cache again?

Thank you again, I appreciate at all the help.

@FrancescAlted
Copy link
Member

I am not sure how this cache works, but it looks like that the problem with shared libraries is appearing again. I am getting out of ideas, so not sure what to recommend. It might be a good idea to clean everything related with Blosc in your system and start anew.

@brianfm
Copy link
Author

brianfm commented Jan 28, 2017

Thank you for the help. I will try this on Monday and report back.

@brianfm
Copy link
Author

brianfm commented Jan 31, 2017

Thank you all for the help.

@adityapatadia
Copy link

I am also facing this issue with Anaconda. On normal system it works fine. @brianfm did you solve it?

@brianfm
Copy link
Author

brianfm commented Apr 8, 2017

I was able to solve it - finally! This is what I did.

  1. I grep'ed to find any instance of blosc or cblosc on my machine (and corresponding files) from previous attempts and deleted them all.

  2. I installed cblosc from scratch first. https://github.com/Blosc/c-blosc/releases
    From there I moved to the folder to run cmake:

    cmake -DDEACTIVATE_SNAPPY=ON -DCMAKE_INSTALL_PREFIX="/usr/lib" .

then:
ctest

then:
cmake --build . --target install

  1. I downloaded python blosc from scratch:
    https://github.com/Blosc/python-blosc/releases/tag/v1.5.1

From there I ran:
python setup.py build_ext --blosc="/usr/local"
python setup.py install

Lastly, I set up my Library Path in .bashrc to point to:
libblosc.so.1

Once all of that was done, I was able to import blosc in python.

I hope this helps. Took me a while to get it running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants