Skip to content

v0.3.0 — popular embedders + methods on the board

Choose a tag to compare

View all tags
@sjmoran sjmoran released this 06 Jun 22:10
· 7 commits to main since this release
v0.3.0
3133bb4

What's new

Embedders (7 tabs): OpenAI text-embedding-3 small & large, BGE (bge-base-en-v1.5), E5 (e5-base-v2), GTE (gte-base), alongside mxbai and MiniLM.
Methods: adds float16 (half precision) and int4 (4-bit scalar).

Every row is a measured bitbudget run, reproducible from a pip install. The headline holds across embedders: a 1-bit code with re-rank is lossless at 32× on OpenAI text-embedding-3-small (0.509 nDCG@10 at 192 B).

Protocol: three BEIR corpora (scifact, nfcorpus, arguana), so the full board reproduces in minutes on a laptop.

Engine fixes: embedding cache reuse (+--force), ~14× faster PQ (vectorised k-means + subsample), model caching, a urllib OpenAI embedder (no hard dep), and a doc_prompt for e5-style prefixes.

Leaderboard: https://sjmoran.github.io/bitbudget/

Assets 2
Loading