ModuleNotFoundError: No module named 'faiss.swigfaiss_avx2'

[nscozzaro]

Hi,

With faiss-cpu on linux I get an error:

Traceback:
env/lib/python3.6/site-packages/faiss/loader.py:31: in <module>
    from .swigfaiss_avx2 import *
E   ModuleNotFoundError: No module named 'faiss.swigfaiss_avx2'

During handling of the above exception, another exception occurred:
env/lib/python3.6/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
xxx/tests/handlers/xxx.py:6: in <module>
    from xxx.handlers import xxx as xxx_handler
xxx/handlers/xxx.py:2: in <module>
    import faiss
env/lib/python3.6/site-packages/faiss/__init__.py:17: in <module>
    from .loader import *
env/lib/python3.6/site-packages/faiss/loader.py:39: in <module>
    from .swigfaiss import *
env/lib/python3.6/site-packages/faiss/swigfaiss.py:13: in <module>
    from . import _swigfaiss
E   ImportError: dlopen: cannot load any more object with static TLS

Seems related to other issues, but I'm still not sure of the solution:
facebookresearch/faiss#866
https://github.com/facebookresearch/faiss/issues?q=is%3Aissue+swigfaiss_avx2

[kyamagu]

Could be a bad interaction with other imported packages. Try changing the import order in python

[MatejNikl]

I have a maybe a similar issue both on linux and windows on versions 1.7.1 and 1.7.1.post1:

INFO:faiss.loader:Loading faiss with AVX2 support.
INFO:faiss.loader:Could not load library with AVX2 support due to:
ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'")

On linux it proceeds with

INFO:faiss.loader:Loading faiss.
INFO:faiss.loader:Successfully loaded faiss.

On windows the loading of faiss fails completely as discussed in #42.

[MatejNikl]

Hey, even though #44 mentions this issue as fixed, now I still get (both on windows and linux, python 3.9):

INFO:faiss.loader:Loading faiss with AVX2 support.
INFO:faiss.loader:Could not load library with AVX2 support due to:
ModuleNotFoundError("No module named 'faiss.swigfaiss_avx2'")
INFO:faiss.loader:Loading faiss.
INFO:faiss.loader:Successfully loaded faiss.

Is that expected behavior?

[kyamagu]

Yes, avx2 extenstion is not included in the wheel for maximum compatibility. You can safely ignore

[yby9212]

try

cd your_python_path/site-packages/faiss
ln -s swigfaiss.py swigfaiss_avx2.py

[ckald]

Yes, avx2 extenstion is not included in the wheel for maximum compatibility. You can safely ignore

Which basically means that faiss will run up to 10 times slower. A couple of years ago we discovered that our production server was not running faiss with AVX2 and had to work our way through debugging faiss and compiling specific avx2-enabled wheels: https://github.com/ckald/faiss-wheels-prophy/ (1.6.5)

I think that this almost silent behavior is very bad 🤷

[ckald]

@yby9212 your suggestion makes even worse -- you remove the error message and faiss still runs slowly. And nobody will know why

[kyamagu]

Feel free to make a PR if you wish to support both plain and avx2 extension in this repository.

You can always use the official conda package. They are likely better optimized.

[NikhilSehgal123]

ln -s swigfaiss.py swigfaiss_avx2.py

Doing this still causes some problems because of the artificial linkage

[zeaver]

see faiss install doc: https://github.com/facebookresearch/faiss/blob/main/INSTALL.md and the issue facebookresearch/faiss#2872 . I meet the same problem when i install it via pip install. I uninstall faiss-gpu and re-install with conda again, it solve it.

pip is not supported to install Faiss. Please use conda.

[eminmtas]

try

cd your_python_path/site-packages/faiss
ln -s swigfaiss.py swigfaiss_avx2.py

this works perfectly, thank you

[lfeges]

Windows Powershell
cd your_python_path/site-packages/faiss
New-Item -ItemType SymbolicLink -Path "swigfaiss_avx2.py" -Target "swigfaiss.py"

[ngbien83]

Windows Powershell cd your_python_path/site-packages/faiss New-Item -ItemType SymbolicLink -Path "swigfaiss_avx2.py" -Target "swigfaiss.py"

Highly recommended for those who don't use Conda