Hope you can recommend me more datasets

[code-orangemonster]

Hi, I am a master's student studying nearest neighbor search algorithm. At present, we need more datasets to test the effectiveness of the algorithm. I hope you can recommend me more dataset links and other information. You can contact me at “tragrace0125@gmail.com”. Thank you very much!

[GioiaZheng]

A few standard sources for nearest-neighbor benchmarking, by scale:

Million-scale, ready-made with ground truth:

• ANN-Benchmarks (erikbern/ann-benchmarks) — HDF5 files with precomputed exact neighbors: SIFT-128, GIST-960, GloVe (25/50/100/200d), Fashion-MNIST-784, NYTimes-256, Last.fm-65, DEEP-image-96. Drop-in for recall/QPS curves.
• TEXMEX / corpus-texmex.irisa.fr — the canonical SIFT1M, GIST1M, plus ANN_SIFT10K for quick tests.

Billion-scale:

• big-ann-benchmarks (NeurIPS'21 & '23 tracks) — BIGANN (SIFT1B), Microsoft SPACEV-1B, Microsoft Turing-ANNS-1B, Yandex DEEP-1B, Yandex Text-to-Image-1B, Facebook SimSearchNet++. All come with query sets and ground truth and are what current FAISS / large-scale ANN papers report on.

Text-embedding (if you want semantic rather than synthetic vectors):

• Encode MS MARCO passage or the BEIR corpora with a sentence encoder to get realistic high-dimensional distributions instead of SIFT/GIST.

For algorithm-effectiveness work, the ANN-Benchmarks suite is usually the fastest starting point because the ground truth and the evaluation harness are already provided.