Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Weird behaviour under macOS #2868

Open
Naozumi520 opened this issue May 19, 2023 · 12 comments
Open

Weird behaviour under macOS #2868

Naozumi520 opened this issue May 19, 2023 · 12 comments
Assignees
@algoriddle
Labels
install Platform unconfirmed-bug

Comments

@Naozumi520
Copy link

Summary

Cannot install under macOS, throw error during Installation.

Platform

OS: macOS 13.3.1

Faiss version: 1.7.0

Installed from: pip

Faiss compilation options: none

Reproduction instructions

run pip3 install faiss-cpu==1.7.0

Throw error:

naozumi@Naozumis-MacBook-Pro RVC-beta % pip3 install faiss-cpu==1.7.0  
Collecting faiss-cpu==1.7.0
  Using cached faiss-cpu-1.7.0.tar.gz (34 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: faiss-cpu
  Building wheel for faiss-cpu (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [67 lines of output]
      running bdist_wheel
      running build
      running build_py
      running build_ext
      building 'faiss._swigfaiss' extension
      swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
      swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
      faiss/faiss/python/swigfaiss.i:241: Error: Unable to find 'faiss/impl/platform_macros.h'
      faiss/faiss/python/swigfaiss.i:245: Error: Unable to find 'faiss/utils/ordered_key_value.h'
      faiss/faiss/python/swigfaiss.i:246: Error: Unable to find 'faiss/utils/Heap.h'
      faiss/faiss/python/swigfaiss.i:251: Error: Unable to find 'faiss/utils/AlignedTable.h'
      faiss/faiss/python/swigfaiss.i:252: Error: Unable to find 'faiss/utils/partitioning.h'
      faiss/faiss/python/swigfaiss.i:253: Error: Unable to find 'faiss/utils/hamming.h'
      faiss/faiss/python/swigfaiss.i:356: Error: Unable to find 'faiss/utils/utils.h'
      faiss/faiss/python/swigfaiss.i:357: Error: Unable to find 'faiss/utils/distances.h'
      faiss/faiss/python/swigfaiss.i:358: Error: Unable to find 'faiss/utils/random.h'
      faiss/faiss/python/swigfaiss.i:360: Error: Unable to find 'faiss/MetricType.h'
      faiss/faiss/python/swigfaiss.i:363: Error: Unable to find 'faiss/Index.h'
      faiss/faiss/python/swigfaiss.i:364: Error: Unable to find 'faiss/Clustering.h'
      faiss/faiss/python/swigfaiss.i:366: Error: Unable to find 'faiss/utils/extra_distances.h'
      faiss/faiss/python/swigfaiss.i:370: Error: Unable to find 'faiss/impl/ProductQuantizer.h'
      faiss/faiss/python/swigfaiss.i:372: Error: Unable to find 'faiss/VectorTransform.h'
      faiss/faiss/python/swigfaiss.i:373: Error: Unable to find 'faiss/IndexPreTransform.h'
      faiss/faiss/python/swigfaiss.i:374: Error: Unable to find 'faiss/IndexFlat.h'
      faiss/faiss/python/swigfaiss.i:375: Error: Unable to find 'faiss/IndexRefine.h'
      faiss/faiss/python/swigfaiss.i:376: Error: Unable to find 'faiss/IndexLSH.h'
      faiss/faiss/python/swigfaiss.i:377: Error: Unable to find 'faiss/impl/PolysemousTraining.h'
      faiss/faiss/python/swigfaiss.i:378: Error: Unable to find 'faiss/IndexPQ.h'
      faiss/faiss/python/swigfaiss.i:379: Error: Unable to find 'faiss/impl/io.h'
      faiss/faiss/python/swigfaiss.i:381: Error: Unable to find 'faiss/invlists/InvertedLists.h'
      faiss/faiss/python/swigfaiss.i:382: Error: Unable to find 'faiss/invlists/InvertedListsIOHook.h'
      faiss/faiss/python/swigfaiss.i:384: Error: Unable to find 'faiss/invlists/BlockInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:385: Error: Unable to find 'faiss/invlists/DirectMap.h'
      faiss/faiss/python/swigfaiss.i:388: Error: Unable to find 'faiss/IndexIVF.h'
      faiss/faiss/python/swigfaiss.i:393: Error: Unable to find 'faiss/IVFlib.h'
      faiss/faiss/python/swigfaiss.i:394: Error: Unable to find 'faiss/impl/ScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:395: Error: Unable to find 'faiss/IndexScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:396: Error: Unable to find 'faiss/IndexIVFSpectralHash.h'
      faiss/faiss/python/swigfaiss.i:397: Error: Unable to find 'faiss/impl/HNSW.h'
      faiss/faiss/python/swigfaiss.i:398: Error: Unable to find 'faiss/IndexHNSW.h'
      faiss/faiss/python/swigfaiss.i:399: Error: Unable to find 'faiss/IndexIVFFlat.h'
      faiss/faiss/python/swigfaiss.i:404: Error: Unable to find 'faiss/invlists/OnDiskInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:407: Error: Unable to find 'faiss/impl/lattice_Zn.h'
      faiss/faiss/python/swigfaiss.i:408: Error: Unable to find 'faiss/IndexLattice.h'
      faiss/faiss/python/swigfaiss.i:411: Error: Unable to find 'faiss/IndexIVFPQ.h'
      faiss/faiss/python/swigfaiss.i:412: Error: Unable to find 'faiss/IndexIVFPQR.h'
      faiss/faiss/python/swigfaiss.i:413: Error: Unable to find 'faiss/Index2Layer.h'
      faiss/faiss/python/swigfaiss.i:415: Error: Unable to find 'faiss/IndexPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:416: Error: Unable to find 'faiss/IndexIVFPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:417: Error: Unable to find 'faiss/utils/quantize_lut.h'
      faiss/faiss/python/swigfaiss.i:419: Error: Unable to find 'faiss/IndexBinary.h'
      faiss/faiss/python/swigfaiss.i:420: Error: Unable to find 'faiss/IndexBinaryFlat.h'
      faiss/faiss/python/swigfaiss.i:421: Error: Unable to find 'faiss/IndexBinaryIVF.h'
      faiss/faiss/python/swigfaiss.i:422: Error: Unable to find 'faiss/IndexBinaryFromFloat.h'
      faiss/faiss/python/swigfaiss.i:423: Error: Unable to find 'faiss/IndexBinaryHNSW.h'
      faiss/faiss/python/swigfaiss.i:424: Error: Unable to find 'faiss/IndexBinaryHash.h'
      faiss/faiss/python/swigfaiss.i:426: Error: Unable to find 'faiss/impl/ThreadedIndex.h'
      faiss/faiss/python/swigfaiss.i:430: Error: Unable to find 'faiss/IndexShards.h'
      faiss/faiss/python/swigfaiss.i:434: Error: Unable to find 'faiss/IndexReplicas.h'
      faiss/faiss/python/swigfaiss.i:438: Error: Unable to find 'faiss/MetaIndexes.h'
      faiss/faiss/python/swigfaiss.i:453: Error: Unable to find 'faiss/impl/AuxIndexStructures.h'
      faiss/faiss/python/swigfaiss.i:639: Error: Unable to find 'faiss/index_io.h'
      faiss/faiss/python/swigfaiss.i:640: Error: Unable to find 'faiss/clone_index.h'
      faiss/faiss/python/swigfaiss.i:644: Error: Unable to find 'faiss/AutoTune.h'
      faiss/faiss/python/swigfaiss.i:645: Error: Unable to find 'faiss/index_factory.h'
      faiss/faiss/python/swigfaiss.i:646: Error: Unable to find 'faiss/MatrixStats.h'
      error: command '/usr/local/bin/swig' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faiss-cpu
  Running setup.py clean for faiss-cpu
Failed to build faiss-cpu
ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects
naozumi@Naozumis-MacBook-Pro RVC-beta %
@Naozumi520
Copy link
Author

Naozumi520 commented May 20, 2023 •
edited
Loading

After using conda as the installer, I've succeeded to install faiss. But it made weird errors and the same using conda-forge

(svc01) naozumi@Naozumis-MBP RVC-beta % python infer-web.py                                  
Traceback (most recent call last):
  File "infer-web.py", line 7, in <module>
    import faiss
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/loader.py", line 65, in <module>
    from .swigfaiss import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: dlopen(/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/_swigfaiss.so, 0x0002): Library not loaded: @rpath/libmkl_intel_lp64.1.dylib
  Referenced from: <4A025BD7-6F1F-3CD5-B319-DE475BB38238> /usr/local/anaconda3/envs/svc01/lib/libfaiss.dylib
  Reason: tried: '/usr/local/anaconda3/envs/svc01/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/lib/libmkl_intel_lp64.1.dylib' (no such file, not in dyld cache)

@Naozumi520 Naozumi520 changed the title Cannot install under macOS Weird behaviour under macOS May 20, 2023
@mdouze mdouze added the install label May 22, 2023
@mdouze
Copy link
Contributor

mdouze commented May 22, 2023

There is currently an issue with MKL on the Mac. @algoriddle what is the current status?

@JingweiZuo
Copy link

JingweiZuo commented Aug 7, 2023 •
edited
Loading

The same issue happened to me when running on the Mac. I have a M1 chip.

Do you have any updates for MKL on the Mac? @mdouze @algoriddle

Thank you!

@liyuyuan6969
Copy link

+1

1 similar comment
@ghost
Copy link

ghost commented Aug 23, 2023

+1

@chongzluong
Copy link

chongzluong commented Aug 27, 2023 •
edited
Loading

+1 - got around this by just conda install -c conda-forge faiss-cpu==1.7.3 in the interim

@luchko
Copy link

luchko commented Aug 31, 2023

+1

1 similar comment
@CharlesDDDD
Copy link

+1

@dalssoft
Copy link

the same! +1

> sw_vers

ProductName: macOS
ProductVersion: 14.2.1
BuildVersion: 23C71

> python --version

Python 3.12.0

> pip --version

pip 23.3.1 from /Users/xxx/.pyenv/versions/3.12.0/lib/python3.12/site-packages/pip (python 3.12)

@InSuperposition
Copy link

+1

> pip wheel --no-cache-dir --use-pep517 "faiss-cpu (==1.7.4)"
> sw_vers

ProductName:            macOS
ProductVersion:         14.2.1
BuildVersion:           23C71

> python --version

Python 3.12.0

> pip --version

pip 23.3.1 from /Users/xxx/.pyenv/versions/3.12.0/lib/python3.12/site-packages/pip (python 3.12)

@InSuperposition
Copy link

There is an open PR #3191 for an issue using faiss with python 3.12

@dalssoft dalssoft mentioned this issue Jan 12, 2024
Closed
@dannychn11 dannychn11 mentioned this issue Jan 29, 2024
Merged
@dalssoft
Copy link

dalssoft commented Feb 9, 2024 •
edited
Loading

It worked after changing python to v3.10 and installing FAISS using brew

> brew install faiss --build-from-source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@algoriddle algoriddle

Labels
install Platform unconfirmed-bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests