Repository navigation
Optional Milvus vector index for brains with large vector volume and high query load (willing to build and maintain) #9067
yinmin2020
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi Garry and team,
I read #3673 and #5526 and understand why extra vector backends were declined: a second index path doubles DDL, search, doctor checks and CI, and only helps self-hosted users. I'd like to propose a narrower, opt-in shape, and I'm willing to write the PR and own its maintenance.
The problem: vector volume and query load
GBrain is increasingly used as a shared brain across many agents and as a company brain. Two things grow together there:
brain_search/ recall at the same time, vector search competes with every transactional read and write on one database. Scaling search then means scaling the whole Postgres.Proposed shape
searchVectorasks Milvus for candidate chunk ids, then hydrates and filters them in Postgres as today.vector.index = milvusplus a connection URI). PGLite and Supabase installs never load it.gbrain doctor, and fall back to the existing pgvector path.Why Milvus for this
What I'd commit to
If this isn't a fit for core, I'm happy to ship it as an external plugin instead. Would a thin index interface around
searchVectorbe acceptable so it could live outside the core?Background: I'm a Milvus Ambassador ("北辰使者", Milvus's community ambassador program in China) and write about RAG and agent memory. I'm proposing this as an individual contributor, not on behalf of Zilliz.
All reactions