BitBudget v0.1.0

BitBudget is a reproducible benchmark for embedding compression and indexing, organised by the projection–quantisation–organisation lens. The headline finding: bits beat dimensions.

Live leaderboard

https://sjmoran.github.io/bitbudget/

Install

pip install bitbudget          # core (numpy only)
pip install bitbudget[faiss]   # + the organisation-axis indexes

Two leaderboards

• Compression — quality per byte. 8 methods across the lens axes (float32, int8, binary, binary+rerank, pq, rabitq, matryoshka, pca), scored by nDCG@10 retained per byte on BEIR with per-corpus error bars. Register your own with @method(...) and run bitbudget run. Headline: a one-bit code with re-ranking is lossless at 32× compression, reproduced at RAG scale on 8.8M MS MARCO passages.
• Indexing — recall per query-second. bitbudget bench-index builds an index over the vectors and reports recall@10, QPS and bytes/vec for flat / hnsw / ivfpq (faiss) and bittrie, a van Emde Boas / PATRICIA radix-trie over compact codes with a multithreaded C kernel (compiled on demand; the wheel stays pure-Python). Register your own with @index(...).

Reproducibility

Plugin registries for methods, embedders and indexes. CI green on Python 3.9 / 3.11 / 3.12. MIT licensed. Published to PyPI via Trusted Publishing (no stored tokens).