Vector memory with Milvus

Vector memory with Milvus

Set up semantic memory search and automatic recall with a local Milvus in Docker or with Zilliz Cloud.

malac's memory works without a database: memories are Markdown files and search falls back to keywords. Adding a Milvus collection gives you search by meaning and auto recall, where relevant memories are attached to each prompt automatically.

You need two things:

1. A Milvus endpoint that serves the REST API v2 (/v2/vectordb/…): Milvus 2.4 or newer, self-hosted, or Zilliz Cloud.
2. An embedding model on an OpenAI-compatible provider with /embeddings: OpenAI, a gateway that serves embeddings, or a local Ollama.

Option A: Milvus in Docker

Milvus publishes a script that runs Milvus standalone in a single container:

curl -sfL https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh -o standalone_embed.sh
bash standalone_embed.sh start

It listens on localhost:19530, and keeps its data in a volumes/ directory next to the script. Stop it with bash standalone_embed.sh stop. (Milvus also documents a Docker Compose setup; any of them works as long as port 19530 is reachable.)

Check that the REST API answers:

curl -s -X POST http://localhost:19530/v2/vectordb/collections/list -d '{}'

{"code":0,"data":[]}

Configure malac:

[memory.vector]
url = "http://localhost:19530"
collection = "malac_memory"

A fresh standalone Milvus has authentication off. If you turn it on, put user:password in a variable and name it with token_env:

[memory.vector]
url = "http://localhost:19530"
token_env = "MILVUS_TOKEN"    # for example "root:Milvus"

Option B: Zilliz Cloud

Create a cluster in Zilliz Cloud (the free tier is enough) and copy its public endpoint and an API key.

export ZILLIZ_API_KEY=...

[memory.vector]
url = "https://<cluster-id>.<region>.zillizcloud.com"   # the public endpoint
token_env = "ZILLIZ_API_KEY"
database = "default"
collection = "malac_memory"

Configure embeddings

With OpenAI:

[memory.embeddings]
provider = "openai"
model = "text-embedding-3-small"

Or locally with Ollama (ollama pull nomic-embed-text first):

[providers.local]
kind = "openai"
base_url = "http://localhost:11434/v1"

[memory.embeddings]
provider = "local"
model = "nomic-embed-text"

provider is the name of one of your configured providers. It must be an OpenAI-kind provider; the Anthropic kind has no embeddings.

Index existing memories

malac creates the collection the first time it embeds something, sized to your embedding model's vectors, with a cosine index. Index the memories you already have:

malac memory reindex

indexed 12 memories

From now on, saving a memory also upserts it into the collection, and forgetting deletes it.

Try it

malac memory search when do we ship

0.71  release-day: Releases go out on Fridays after the 14:00 standup

The score is the cosine similarity. With auto_recall on (the default), each prompt of at least 8 characters is embedded, and up to recall_limit memories with a similarity of at least 0.4 are attached to it.

How the collection is used

Field	Content
id	global:<name> or project:<project root>:<name>
vector	Embedding of the memory's name, description and body
scope	global or project
project	The project root, for project memories
name	The memory's name

• A search sees global memories and the current project's, never another project's.
• Only ids and vectors live in Milvus. Search results are read from the Markdown files, so they are always current.
• One collection can serve several machines and projects.

Troubleshooting

• Changed the embedding model? The collection's vector size is fixed at creation. Use a new collection name and run malac memory reindex.
• no vector store configured: malac memory reindex needs both [memory.vector] and [memory.embeddings].
• embedding request failed: the embeddings provider has no /embeddings endpoint, the model id is wrong, or the key is missing.
• Milvus … failed: check the URL, the token and, on Zilliz Cloud, the database.
• Run with MALAC_LOG=debug for details.