Public datasets v1 — GIST1M and T2Ranking

MO Search Lab public datasets v1

Frozen public packs for fulltext/vector evaluation and investigation:

• GIST filtered v7: 1,000,000 vectors, 960 dimensions, Top-100,
  PRE/POST filtering, 100/10/1% eligibility, repeat stability.
• T2Ranking anli ngram v3: 2,303,643 passages, 500 frozen dev queries,
  TF-IDF/BM25, ngram, anli-shaped SQL and repeat stability.
• GIST/T2 paired SQL v7: shared CSVs, two independent SQL routes.

The data-only archive is approximately 3.99 GB compressed, approximately
15.73 GB extracted. Shared CSVs are stored once using hardlinks. It contains
no executable, model, measured reports or customer records.

Download and extract

Download all three mo-search-lab-datasets-v1.tar.gz.part01/02/03 files,
SHA256SUMS and archive.sha256 to the same directory.

sha256sum -c SHA256SUMS
cat mo-search-lab-datasets-v1.tar.gz.part01 \
    mo-search-lab-datasets-v1.tar.gz.part02 \
    mo-search-lab-datasets-v1.tar.gz.part03 \
    > mo-search-lab-datasets-v1.tar.gz
sha256sum -c archive.sha256
tar -xzf mo-search-lab-datasets-v1.tar.gz
cd mo-search-lab-datasets-v1
sha256sum -c SHA256SUMS

SHA256SUMS also covers the attached README, metadata, notices and verification
record: download those small files too before checking the full list.
Extract the complete archive to one filesystem to retain shared hardlinks.

Run MO Search Lab v0.8.3 with --pack pointing to the desired pack.
For the paired pack also pass --mixed-scenarios vector,fulltext.
Quick Start
· Dataset instructions

Provenance and interpretation

Input SHA-256, source records, frozen query IDs and judgments are preserved.
GIST includes synthetic visibility metadata and independently calculated
filtered Top-100 truth. T2Ranking preserves original passage/query identities;
500 dev queries are an observational sample, not an official full-test result.
Quality, performance and repeat stability are separate measurements.

Original sources: INRIA/IRISA TexMex and
THUIR/T2Ranking.
T2Ranking's publisher identifies Apache-2.0; its license copy and source/change
attribution are included in the archive. See LICENSE_NOTICES.md.

Each release asset is under GitHub's 2 GiB limit. verification.json records
full archive CRC, every file checksum, the two shared CSV hardlinks and all
compressed part hashes. Uploads are verified against GitHub asset digests.