Replacing ChromaDB with TurbovecDB - an experiment that proved very good.

[kostadis]

Summary

MemPalace's backend abstraction (RFC 001 BaseBackend / BaseCollection) makes
it possible to swap the vector store. This proposes a second, opt-in backend
built on turbovec (a CPU, 4-bit
TurboQuant ANN) paired with a durable SQLite sidecar, packaged as a standalone
library — turbovecdb — with MemPalace as its first customer. ChromaDB stays
the default; this is selected with MEMPALACE_BACKEND=turbovec.

Motivation

The interesting properties for a local-first, "verbatim, 100% recall" system:

• No HNSW corruption class. A meaningful chunk of the Chroma backend exists to
  detect and quarantine corrupted/stale HNSW segments. A design where the durable
  state is plain SQLite rows and the ANN index is a rebuildable cache sidesteps
  that failure mode entirely.
• "100% recall" is a store property, not an index property. With SQLite as the
  source of truth (exact float32 vectors + verbatim text), a lossy quantized index
  can only reorder neighbours, never change what text comes back — and it can be
  rebuilt from the store after any crash.
• Cheap to build, small, CPU-resident. Useful on modest/local hardware.

What it is

turbovecdb is a small embedded vector DB:

• SQLite is the source of truth — documents, metadata, the string-id↔uint64
  map, and the exact float32 vectors. The turbovec .tvim index is a derived
  cache; if missing or stale it rebuilds from SQLite on open (crash-safe).
• Exact-cosine re-rank — turbovec returns an approximate candidate pool;
  turbovecdb recomputes true cosine from the stored vectors and returns
  distance ∈ [0, 2]. So callers (and the hybrid ranker in searcher.py) get a
  correct cosine distance, and end-to-end quality matches an exact index.
• Full metadata filters — $eq/$ne/$in/$nin/$gt/$gte/$lt/$lte, $and/$or,
  and where_document $contains; unsupported operators raise rather than silently
  match everything.
• Multi-process safe — a cross-process write lock plus lock-free reader cache
  coherence (a store_gen counter), so the hooks miner / MCP server / CLI can
  share a palace.

In MemPalace, backends/turbovec.py is a thin adapter over the library; no engine
code lives in MemPalace itself.

Evidence (measured, local, honest)

Local parity vs ChromaDB through the MemPalace adapter, 15,805 real text
drawers, ONNX MiniLM (384-d), identical vectors fed to both backends:

backend	build s	q p50 ms	q p95 ms	size MB
turbovec	0.80	0.59	0.76	62.0
chromadb	10.74	2.08	2.61	142.1

≈ 13× faster build, ~3.5× faster query (p50/p95), ~2.3× smaller on disk, with
retrieval quality a wash (hybrid hit@k tied within noise). A standalone,
reproducible synthetic benchmark (docs/performance/benchmark.py in turbovecdb)
shows the same build win and recall@10 = 1.0 (via re-rank) vs ChromaDB's
default-ef 0.67.

Caveats, stated plainly:

• The size result is workload-dependent. On a pure-vector workload (no
  documents) turbovecdb is actually slightly larger than Chroma, because it
  stores exact float32 vectors; the 2.3× win above comes from the document-heavy
  regime where Chroma's HNSW graph overhead dominates. Measure on your data.
• Recall numbers are default-vs-default; Chroma's recall is tunable via
  hnsw:search_ef at a latency cost.
• Benchmarks are one machine; ratios shift with N, dim, and hardware.

Compatibility / risk

• Default unchanged. palace.py resolves the backend from MEMPALACE_BACKEND
  (default chroma); a normal run is byte-for-byte identical.
• Tests: full suite 2044 passed, 14 skipped, 1 pre-existing unrelated
  failure (present on base). turbovecdb has its own 39-test suite (contract,
  persistence, filters, embedder, real multi-process concurrency).
• Conforms to the BaseBackend/BaseCollection contract; results are the typed
  QueryResult/GetResult.

Open questions for the list

1. Interest level — is a second, opt-in backend something the project wants to
   carry, or should this live purely as a third-party plugin discovered via the
   mempalace.backends entry point?
2. Dependency packaging — the [turbovec] extra currently points at
   turbovecdb (not yet on PyPI). Preference between (a) publishing turbovecdb to
   PyPI, or (b) a git-URL dependency for now?
3. Contract gaps worth closing before anyone relies on it: batch query, an
   embedder-identity guard (refuse a model/dim change on reopen), and validation
   at 100K+ drawers (MemPalace's stress regime).
4. Conformance — would a parametrized cross-backend conformance suite (run
   against both chroma and turbovec) be welcome? It'd make adding future
   backends safer.

References

• PR: Add turbovec backend (powered by turbovecdb), opt-in via MEMPALACE_BACKEND kostadis/mempalace#18
• Library: https://github.com/kostadis/turbovecdb
  (docs: docs/core/architecture.md, docs/performance/)
• Underlying index: https://github.com/RyanCodrai/turbovec

[kostadis]

If there is interest, I am happy to do more work.

The nice property of this system is it makes mempalace faster, more space efficient and more resilient.

The downside is that - it's a new DB that probably has a billion new bugs.

[williamblair333]

I just got out of Chroma hell. I'm interested. I'll have my AI buddy investigate and maybe I can help out too.

[kostadis]

pip install turbovecdb # pulls turbovec, numpy, filelock If you want to use install it.

…

On Wed, Jun 3, 2026 at 10:49 AM William Blair ***@***.***> wrote: I just got out of Chroma hell <#1685>. I'm interested. — Reply to this email directly, view it on GitHub <#1668?email_source=notifications&email_token=AB6SYUCTOPLFCP2FB3DC2ZD46BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17169911>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6SYUCFOXXA2SXVCIT6ZLL46BQMJAVCNFSM6AAAAACZUDTD2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJWHE4TCMI> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AB6SYUDDDJADFN5TZHQLQTT46BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> and Android <https://github.com/notifications/mobile/android/AB6SYUEP4KC737UUSAP7UT346BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

[kostadis]

If you are interested in installing it and want instructions on how to patch mempalace https://github.com/kostadis/turbovecdb/blob/main/docs/MEMPALACE_TURBOVEC_MIGRATION_HANDOFF.md

…

On Wed, Jun 3, 2026 at 2:33 PM Kostadis Roussos ***@***.***> wrote: pip install turbovecdb # pulls turbovec, numpy, filelock If you want to use install it. On Wed, Jun 3, 2026 at 10:49 AM William Blair ***@***.***> wrote: > I just got out of Chroma hell > <#1685>. I'm > interested. > > — > Reply to this email directly, view it on GitHub > <#1668?email_source=notifications&email_token=AB6SYUCTOPLFCP2FB3DC2ZD46BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17169911>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AB6SYUCFOXXA2SXVCIT6ZLL46BQMJAVCNFSM6AAAAACZUDTD2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJWHE4TCMI> > . > Triage notifications, keep track of coding agent tasks and review pull > requests on the go with GitHub Mobile for iOS > <https://github.com/notifications/mobile/ios/AB6SYUDDDJADFN5TZHQLQTT46BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> > and Android > <https://github.com/notifications/mobile/android/AB6SYUEP4KC737UUSAP7UT346BQMJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRGY4TSMJRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. > Download it today! > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[williamblair333]

Scale test update — weekly benchmark (290K+ drawers, MiniLM 384-d, k=10)

Date	Drawers	Chroma p50ms	Chroma p95ms	TurboVec p50ms	TurboVec p95ms	Recall@10	Queries
2026-06-04	296,595	318.451	323.326	6.532	6.799	0.408	200

Methodology: 200 random drawer vectors sampled from ChromaDB, used as query vectors against both backends. Recall@10 = |top-10 overlap| / 10. Same machine, sequential runs.

[williamblair333]

@kostadis — update: we have it running. Here's what we set up:

Setup: Installed from our patched fork (williamblair333/turbovecdb@fix/security-findings — the security PR we submitted). Running turbovecdb in parallel against our live MemPalace palace at ~/.turbovecdb-eval/ — ChromaDB stays production throughout.

Migration: 296,595 drawers + 274 closets → turbovecdb in 77s (~3,860 rows/s). MiniLM 384-dim float32.

First benchmark (200 random query vectors, k=10):

Backend	p50	p95	recall@10
ChromaDB	318ms	323ms	—
turbovecdb	6.5ms	6.8ms	0.408

49× faster queries. The recall@10=0.408 is the number we want to understand — is that expected at this scale with default ef, or is there a parameter worth tuning? We have nightly sync after the mine cron and a weekly benchmark that auto-posts results to this thread, so we'll have 4–5 data points in a month.

Eval rig PR: williamblair333/Uncle-J-s-Refinery#23 (merged)

[kostadis]

Thank you so much for your work. If you feel like it open an issue. kostadis

…

On Thu, Jun 4, 2026 at 10:57 AM William Blair ***@***.***> wrote: @kostadis <https://github.com/kostadis> — update: we have it running. Here's what we set up: *Setup:* Installed from our patched fork ***@***.*** /security-findings — the security PR we submitted). Running turbovecdb *in parallel* against our live MemPalace palace at ~/.turbovecdb-eval/ — ChromaDB stays production throughout. *Migration:* 296,595 drawers + 274 closets → turbovecdb in 77s (~3,860 rows/s). MiniLM 384-dim float32. *First benchmark* (200 random query vectors, k=10): Backend p50 p95 ***@***.*** ChromaDB 318ms 323ms — turbovecdb 6.5ms 6.8ms 0.408 *49× faster queries.* The ***@***.***=0.408 is the number we want to understand — is that expected at this scale with default ef, or is there a parameter worth tuning? We have nightly sync after the mine cron and a weekly benchmark that auto-posts results to this thread, so we'll have 4–5 data points in a month. — Reply to this email directly, view it on GitHub <#1668?email_source=notifications&email_token=AB6SYUDO2N3LWQMN5FQBF3346GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17184172>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6SYUDHBTAQBYNDWF245A346GZ75AVCNFSM6AAAAACZUDTD2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJYGQYTOMQ> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AB6SYUA7FDCYCVTONJCLR5D46GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSVGM33PORSXEX3JN5ZQ> and Android <https://github.com/notifications/mobile/android/AB6SYUCM6OCTDCJEPI2YHBL46GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>

[williamblair333]

There's a PR open Thank you, William Blair | ***@***.*** | 317-998-8509 ||Be impeccable in your speech | Take nothing personally | Assume nothing | Do your best||

…

On Thu, Jun 4, 2026, 2:47 PM kostadis ***@***.***> wrote: Thank you so much for your work. If you feel like it open an issue. kostadis On Thu, Jun 4, 2026 at 10:57 AM William Blair ***@***.***> wrote: > @kostadis <https://github.com/kostadis> — update: we have it running. > Here's what we set up: > > *Setup:* Installed from our patched fork ***@***.*** > /security-findings — the security PR we submitted). Running turbovecdb *in > parallel* against our live MemPalace palace at ~/.turbovecdb-eval/ — > ChromaDB stays production throughout. > > *Migration:* 296,595 drawers + 274 closets → turbovecdb in 77s (~3,860 > rows/s). MiniLM 384-dim float32. > > *First benchmark* (200 random query vectors, k=10): > Backend p50 p95 ***@***.*** > ChromaDB 318ms 323ms — > turbovecdb 6.5ms 6.8ms 0.408 > > *49× faster queries.* The ***@***.***=0.408 is the number we want to > understand — is that expected at this scale with default ef, or is there > a parameter worth tuning? We have nightly sync after the mine cron and a > weekly benchmark that auto-posts results to this thread, so we'll have 4–5 > data points in a month. > > — > Reply to this email directly, view it on GitHub > < #1668?email_source=notifications&email_token=AB6SYUDO2N3LWQMN5FQBF3346GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17184172>, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AB6SYUDHBTAQBYNDWF245A346GZ75AVCNFSM6AAAAACZUDTD2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJYGQYTOMQ> > . > Triage notifications, keep track of coding agent tasks and review pull > requests on the go with GitHub Mobile for iOS > < https://github.com/notifications/mobile/ios/AB6SYUA7FDCYCVTONJCLR5D46GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSVGM33PORSXEX3JN5ZQ> > and Android > < https://github.com/notifications/mobile/android/AB6SYUCM6OCTDCJEPI2YHBL46GZ75A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DCNZSUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>. > Download it today! > You are receiving this because you were mentioned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#1668?email_source=notifications&email_token=AOOGOKD6PSPHTIBMYBAV7AT46G74DA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZRHA2DKNJYUZZGKYLTN5XKOY3PNVWWK3TUUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17184558>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOOGOKE7OPCGHTEMOJC36JT46G74DAVCNFSM6AAAAACZUDTD2OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTOMJYGQ2TKOA> . You are receiving this because you commented.Message ID: ***@***.***>

[williamblair333]

I'm coming back to report that I'm so done with chromadb now. It's over. I'm not waiting for prelim turbovecdb results. I'm going to switch to it after comparing it to other products. At this point, I'd really push mempalace to get rid of it chromadb entirely.

[kostadis]

Thank you!

…

On Thu, Jun 11, 2026 at 6:18 AM William Blair ***@***.***> wrote: I'm coming back to report that I'm so done with chromadb now. It's over. I'm not waiting for prelim turbovecdb results. I'm going to switch to it after comparing it to other products. At this point, I'd really push mempalace to get rid of it chromadb entirely. — Reply to this email directly, view it on GitHub <#1668?email_source=notifications&email_token=AB6SYUAIIFIV75V2WLWD75347KWTTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZSGY2DOMZWUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-17264736>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6SYUEGJW5RLJNKO65E22347KWTTAVCNFSNUABJKJSXA33TNF2G64TZHMYTEMBRGY2TMMRRGA5UI2LTMN2XG43JN5XDWMJQGE3DSNJTGCQXMAQ> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AB6SYUG2DAMU4XTSDZXZJPD47KWTTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZSGY2DOMZWUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSVGM33PORSXEX3JN5ZQ> and Android <https://github.com/notifications/mobile/android/AB6SYUCBJ6C4BEUB4SNNAOT47KWTTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZSGY2DOMZWUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>