v0.6.0 — Cloudflare Workers AI embeddings

What changed

Production embedding moved from local sentence-transformers (bge-small @ 384-dim, in-process) to Cloudflare Workers AI (@cf/baai/bge-base-en-v1.5, 768-dim, over HTTPS).

Why

The local model carried a real cost on every cold-boot — 75-200s to download + load — which forced auto_stop_machines=off and a 2 GB Fly machine just to keep the embedder's hot pages resident. CF Workers AI is free at our usage tier and lets the runtime image drop everything torch-shaped.

Numbers

	before	after
Runtime image	~4 GB	89 MB
Cold boot embedder prewarm	75-200s	1.07s
Source-only deploy time	12-14 min	2-3 min
Fly machine memory	2 GB	1 GB
Eval gate (24 cases)	24/24	24/24
Warm query latency p50 / p95	—	588ms / 2.3s

Bake-off

Locally evaluated three options before shipping:

• A bge-small @ 384 (local, baseline)
• B bge-base @ 768 (CF) ← shipped
• C bge-m3 @ 1024 (CF)

C looked appealing on paper but regressed the headline PPI loading-limit case (top-1 became 15.3.n audit text instead of 8.2.c loading-limit clause). B matched A on hit rate and on retrieval quality, with the bonus that we get the whole runtime-slimming win.

How embeddings get configured

Three env vars (no secrets) drive provider selection:

RBI_EMBEDDING_PROVIDER  local | cloudflare    (default: local)
RBI_EMBEDDING_MODEL     <model id>
RBI_EMBEDDING_DIM       <int>

Self-hosters who don't have CF creds can stay on the local path with uv sync --extra local-embeddings. The Fly image bakes the cloudflare defaults; CF auth (CF_ACCOUNT_ID, CF_API_TOKEN) lives in Fly secrets.

Weekly refresh

.github/workflows/refresh.yml runs Sundays 02:00 UTC: crawl → 5 indexers → eval gate (>=80%) → sftp + atomic-rename onto the Fly volume. App-scoped FLY_API_TOKEN limits blast radius to this app only.

Migration notes

Schema is forward-compatible — chunks_vec is now DIM-aware (vec0(embedding float[N]) where N comes from embedding_config.DIM). Forks running on the old 384-dim local path keep working without env changes.