Repository navigation
v0.3.0 — Benchmarks + Semantic Embeddings
Highlights
Two new public benchmarks and optional semantic-embeddings support
(hybrid BM25 + vector search).
Benchmarks
| Benchmark | Tests | R@5 | Runtime |
|---|---|---|---|
| LongMemEval (ICLR 2025) | 500 | 69.4% | ~31s |
| LoCoMo (Snap Research) | 1986 | 88.2% | ~11s |
| ConvoMem sizes 1–6 (Salesforce) | 7021 | 100.0% | ~15 min |
- LoCoMo per-category R@5: open-domain 92.7%, temporal 85.0%, single-hop
80.5%, multi-hop 59.4%, adversarial 93.0% - ConvoMem confirms "your first 150 conversations don't need RAG"
Semantic Embeddings (optional)
Set MEM_EMBEDDINGS_URL, MEM_EMBEDDINGS_MODEL, MEM_EMBEDDINGS_API_KEY
pointing at any OpenAI-compatible /v1/embeddings endpoint (OpenAI, Voyage,
Cohere compat, Ollama, LM Studio, LocalAI, llama.cpp). Then:
mem reindex # compute vectors for all drawers
mem search "why GraphQL" --mode hybrid # BM25 + vector fused via RRF- Pure Go HTTP client, batch support
- Embeddings stored as BLOBs in the same SQLite file
- Hybrid = Reciprocal Rank Fusion (k=60)
- Entire feature is opt-in; default BM25 behavior unchanged
- No new module dependencies
Install
# Linux amd64
curl -fsSL https://github.com/snow-ghost/mem/releases/download/v0.3.0/mem-linux-amd64.tar.gz | tar xz
sudo mv mem-linux-amd64 /usr/local/bin/memReplace linux-amd64 with linux-arm64, darwin-amd64, or darwin-arm64
for other platforms.
Reproduce benchmarks
go build -o /tmp/longmemeval-bench ./benchmarks/longmemeval/
go build -o /tmp/locomo-bench ./benchmarks/locomo/
go build -o /tmp/convomem-bench ./benchmarks/convomem/See benchmarks/README.md
for dataset download URLs.