Experiments in image search using two different datastores...
- Index
- Creates l2-normalized feature embedding for each image
- Extracts exif metadata and file stat
- Stores everything in database
- Search
- Results are returned by similarity
- Search-by-image: create feature embedding and compare it for similarity with image vectors in database
- Search-by-prompt: encode prompt embedding and compare it for similarity with image vectors in database
- Search-by-text: search for text in metadata
Includes shared requirements for both solutions
python -m venv venv
source venv/bin/activate
pip install -r requirements.txtpython imagedb-lancedb.py --help
python imagedb-lance.py --index /home/vlado/dev/images/ --force
python imagedb-lance.py --search /home/vlado/dev/images/Siamese_100.jpg
python imagedb-lance.py --search "photo of siamese cat" --condition "width > 256 AND size > 131072"
Note: More feature-rich than Faiss due to presence of unified store and hybrid search as well as conditional filters
Dependencies:
- OpenClip for embedding extraction
- LanceDB as vector database with hybrid vector and metadata store and search
python imagedb-faiss.py --help
python imagedb-faiss.py --index /home/vlado/dev/images/
python imagedb-faiss.py --search /home/vlado/dev/images/Siamese_100.jpg
Note: Uses separate vector and metadata stores which are searched separately
Dependencies:
- LanceDB: load=3.3sec, index=7384images/138sec its=52.5, search-by-100-images=2.0sec
- Faiss: load=3.0sec index=7384images/119sec its=62.0, search-by-100-images=1.7sec
- OpenCLiP is chosen as its one of the few models families that can do both text and image feature extraction
- Any CLiP model can be used
Defaults: Faiss=ViT-L-14, LanceDB=ViT-B-16 - Most CliP models also have different variants depending on which dataset they were trained on
Defaults: Faiss=laion2b_s32b_b82k, LanceDB=laion2b_s34b_b88k