Shared objects not found

[ccoulombe]

Summary

When building with shared objects (-DBUILD_SHARED_LIBS=ON) and python bindings, importing faiss results in failure to import _swigfaiss shared objects following the installation of the resulting wheel.

(8690) ~ $ python -c "import faiss"
ImportError: cannot import name '_swigfaiss' from 'faiss'

There's two issues that causes the above :

• The RPATH is incorrect as it points to the build directory which is likely to be removed after installation. It should point to the installation directory.
• libfaiss_python_callbacks.so is missing from the wheel.

/home/coulombc/.envs/8690/lib/python3.9/site-packages/faiss/_swigfaiss_avx2.so:
	libfaiss_avx2.so => not found
	libfaiss_python_callbacks.so => not found
/home/coulombc/.envs/8690/lib/python3.9/site-packages/faiss/_swigfaiss.so:
	libfaiss.so => not found
	libfaiss_python_callbacks.so => not found

Platform

OS:

Faiss version: v1.7.1

Installed from: source

Faiss compilation options:

cmake .. -DCMAKE_INSTALL_PREFIX=../inst -DBUILD_SHARED_LIBS=ON -DFAISS_OPT_LEVEL=avx2 -DFAISS_ENABLE_PYTHON=ON -DBUILD_TESTING=OFF

Interface:

• C++
• Python

Reproduction instructions

virtualenv --clear ~/ENV && source ~/ENV/bin/activate
cmake .. -DCMAKE_INSTALL_PREFIX=../inst -DBUILD_SHARED_LIBS=ON -DFAISS_OPT_LEVEL=avx2 -DFAISS_ENABLE_PYTHON=ON -DBUILD_TESTING=OFF
make -j
make install
cd faiss/python
python setup.py bdist_wheel
pip install dist/faiss-1.7.1*.whl
cd
python -c "import faiss"

[ccoulombe]

Fixes

• The RPATH should include the installation directory, not the build directory. To fix the RPATH once installed:

patchelf --set-rpath "\$ORIGIN:<path installation directory>/lib64:$CUDA_HOME/lib" $VIRTUAL_ENV/lib/python3.9/site-packages/faiss/_swigfaiss_avx2.so $VIRTUAL_ENV/lib/python3.9/site-packages/faiss/_swigfaiss.so

• Include libfaiss_python_callbacks.so in python installation. #2062 fixes the missing libfaiss_python_callbacks.so from the python installation.

[ccoulombe]

@mdouze
This is still the case for v1.8.0.

Can this be reopen and/or can my PR #2062 that fixes this be merged ?