v0.1.3

HDF5 files containing ANN-benchmarks-compatible datasets of CLIP features extracted from the COCO dataset.

Extraction Process: Features vectors are the CLS output token of the OpenAI's CLIP with ViT-B/16 architecture (512 dimensions) of the visual or textual encoder.

All features: All extracted features from images and texts and the corresponding COCO IDs are provided in coco-clip-b16-512-features.hdf5. This does not follow the ANN-benchmark format.

Karpathy Split: In files containing karpathy-split, we used the Karpathy's split of COCO 2014 for queries and search sets.

• the search set comprises the 118,287 vectors extracted from the training set images.
• Visual queries (in the i2i version) and textual queries (in the t2i version) are extracted from the validation-set images and relative captions (only the first one out of five available for each image), respectively. Each set of queries is comprised of 5,000 vectors.

Random Split: The other files contain random train/test splits performed by ann-benchmark utilities.

• the search set comprises 113,287 random vectors extracted from all the image features.
• the remaining 10,000 image features are kept as visual queries in the i2i version, and the corresponding first (out of five) captions are taken as textual queries in the t2i version.