Skip to content

0.3.0 — pre-vectorized FinanceBench snapshot on HuggingFace Hub

Choose a tag to compare

View all tags
@Rishabhmannu Rishabhmannu released this 01 Jun 18:12
v0.3.0
77e165a

TL;DR

New-user time-to-first-query drops from ~30 min + ~$5-15 (re-embed 360 PDFs through Voyage's API) to ~3 min + $0 (download pre-computed embeddings, bulk-upsert into Qdrant).

pip install --upgrade financebench-rag-agent
financebench setup
financebench seed --from-hf cmpunkmannu/financebench-voyage-finance-2-embeddings

The dataset

Live at https://huggingface.co/datasets/cmpunkmannu/financebench-voyage-finance-2-embeddings (public, CC BY-NC 4.0).
Chunks 68,059
Documents 84 distinct SEC filings (10-K / 10-Q / 8-K / earnings)
Dense vectors voyage/voyage-finance-2 (1024-dim, cosine)
Sparse vectors BM25 tokens (Qdrant fastembed)
Files chunks.parquet (460 MB) + manifest.json + README.md
License CC BY-NC 4.0 (inherited from FinanceBench)

Also usable from any RAG stack without the CLI:

from huggingface_hub import hf_hub_download
import pandas as pd

path = hf_hub_download(
    repo_id=\"cmpunkmannu/financebench-voyage-finance-2-embeddings\",
    filename=\"chunks.parquet\",
    repo_type=\"dataset\",
)
df = pd.read_parquet(path)

What shipped

  • scripts/export_to_hf.py — producer side: Qdrant scroll → parquet + manifest + frozen README. `--upload` mode handles HF Hub publish.
  • `scripts/seed_from_hf.py` — consumer side (runs in container): downloads via huggingface_hub, drops + recreates collection, bulk upserts. Verifies manifest dense_dim matches consumer pipeline; hard-fails on mismatch.
  • `financebench seed --from-hf ` — CLI flag wiring, mutexes with --sample/--dir, threads --collection + --revision.
  • `pyproject.toml [backend]` — explicit pyarrow + huggingface_hub pins (pyarrow was not in the backend tree; leaving it transitive would have silently broken the restore path).

Verification

Round-trip verified end-to-end before release: downloaded the published parquet → restored to a fresh test Qdrant collection → queried a known chunk's vector against both source and restored → top-3 results match exactly. Vectors round-trip without precision loss.

Why parquet, not a native Qdrant snapshot

Parquet is framework-agnostic and has a stable, documented schema. The README's "Direct parquet (any RAG stack)" path wouldn't work with a native snapshot. Trade: ~70s extra to re-index on restore vs the portability gain. See docs/engineering-log.md for the full reasoning.

Multi-arch image

`ghcr.io/rishabhmannu/financebench-rag-agent-api:0.3.0` (linux/amd64 + linux/arm64). Pulled automatically by `financebench upgrade`.

Full notes

See docs/engineering-log.md for the 0.3.0 narrative — frozen-README design, judgment-call moments, round-trip verification details.

Assets 4
Loading