Image retrieval / similarity

[johnyquest7]

Thanks for this great resource.
Planning to play with your cats and dog example.
I am wondering whether this approach could be used for image retrieval / image similarity?
Have you looked into it?

[lucidrains]

@johnyquest7 hey Johnson! yea absolutely! It works like any other neural net. You can take the embedding (usually the output at the very last attention layer) https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/vit_pytorch.py#L113 and use your favorite nearest neighbor library (for example, Faiss https://github.com/facebookresearch/faiss) and you've got yourself retrieval