Skip to content

retrieval-bench 0.1.0

Latest

Choose a tag to compare

@technobasant technobasant released this 14 Oct 07:55

First release.

  • Engines: PostgreSQL 18 + pgvector 0.8.7 (HNSW, IVFFlat, iterative index scans),
    Qdrant 1.19.2 (HNSW, payload index), OpenSearch 3.9.0 (Faiss HNSW, BM25, native hybrid
    with the RRF score-ranker processor), under identical limits in Docker Compose.
  • Data: SciFact, ArguAna and Touché 2020 / args.me from BEIR, downloaded on demand,
    licences checked at the original source. One embedding pass with bge-small-en-v1.5
    (and bge-base-en-v1.5 on the two small sets) at pinned revisions.
  • Measurements: recall–latency sweeps, build time and footprint, filtered search at 50%,
    10% and 1% selectivity with each engine's strategies, a concurrency sweep, updates
    while querying, and nDCG@10 / recall@100 for BM25, Postgres full-text, vector and
    hybrid retrieval. See MEASUREMENTS.md.
  • Tooling: rbench CLI, pytest, ruff, mypy strict, CI with a synthetic smoke run against
    all three engines.