qmd-memory-stack

Local, GPU-accelerated RAG memory for coding agents — Claude Code and OpenClaw — built on qmd with fully local Qwen3 GGUF models (no cloud embeddings) and optional Intel/NVIDIA GPU offload.

It gives an agent a hierarchical, retrieval-augmented memory:

• L1 — MEMORY.md: a lean always-loaded index of one-line facts.
• L2 — topic leaf files (infra/, projects/, …): retrieved on demand via semantic + keyword search.
• L3 — sessions (optional): long-form transcripts.

Recall is wired into Claude Code through hooks (auto-injected on every prompt) and an MCP server (on-demand deep recall), and can share OpenClaw's memory read-only so both agents see the same facts.

Why

• Private & offline: embeddings/expansion/reranking run on local GGUF models (Qwen3-Embedding-0.6B, a 1.7B query-expansion model, Qwen3-Reranker-0.6B). Nothing leaves the box. Works with providers that have no embeddings endpoint (e.g. Core42 Compass).
• Fast where it matters: the always-on auto-inject path is pure BM25 (~0.4s, no model load); heavier semantic/hybrid recall is on-demand. On an Intel Arc iGPU via Vulkan, full hybrid query went 110.7s → 5.7s (~19×) — see docs/OPTIMIZATIONS.md.
• Fail-open: memory recall never blocks or breaks a turn (hard timeouts, swallow errors, exit 0).

Quickstart

git clone https://github.com/mairp/qmd-memory-stack.git
cd qmd-memory-stack
./install.sh                       # auto-detects OS, qmd, GPU, memory dir, OpenClaw
# then start a NEW Claude Code session (loads the hooks + MCP server)

Prerequisites on the target machine: node and the qmd binary on PATH (npm i -g @tobilu/qmd), and Claude Code. OpenClaw is optional (auto-detected for the read-only share).

Useful flags:

./install.sh --dry-run             # show everything it would do, change nothing
./install.sh --no-gpu              # force CPU mode
./install.sh --memory-dir DIR      # override the Claude auto-memory dir
./install.sh --with-openclaw       # also let OpenClaw recall Claude's memory (symmetric share)
./uninstall.sh                     # clean revert (your memory notes are NOT deleted)

Support matrix

	Memory stack	GPU offload
OS	Debian/Ubuntu (apt), Fedora (dnf), Arch (pacman), openSUSE (zypper)	same
Intel iGPU (Arc/Xe)	✅	✅ Vulkan (Mesa ANV) — tested
NVIDIA	✅	⚠️ CUDA via node-llama-cpp — implemented; on the reference host the NVIDIA card (RTX 3090) is reserved for inference and qmd is pinned off it, so CUDA-for-qmd stays unverified by choice. Also visible to Vulkan once its ICD is installed
AMD	✅	✅ Vulkan (Mesa RADV) — best-effort
iGPU + discrete card	✅	✅ tested — pin qmd to the iGPU with --igpu (auto-pins if a discrete NVIDIA ICD is present). Live on the reference host: Intel Arc iGPU + RTX 3090 eGPU, qmd stays on the Arc — see docs/GPU-VULKAN.md
CPU-only	✅	n/a (QMD_FORCE_CPU=1)

Docs

• docs/ARCHITECTURE.md — how the memory layers, RAG modes, and hooks fit together.
• docs/TUTORIAL.md — understand it, install it, verify it, troubleshoot it.
• docs/GPU-VULKAN.md — GPU offload (Intel/NVIDIA/AMD), the shared-RAM model, reverting.
• docs/OPTIMIZATIONS.md — the full tuning changelog + benchmark numbers.

How it works (one diagram)

 user prompt ──► UserPromptSubmit hook ──► claude-recall.sh -m fast ──► qmd search (BM25, ~0.4s)
                                                                          └─► <memory-recall> injected
 on demand   ──► MCP "qmd" tools / claude-recall.sh -m deep|query ─────► qmd vector / hybrid (+GPU)
 memory write ─► PostToolUse hook ───────► reindex.sh (flock) ─────────► qmd update && embed
 session start ► SessionStart hook ──────► refresh GPU flag + background reindex

 qmd index (sqlite, FTS5 + sqlite-vec)  ◄── collections: claude-memory + openclaw-memory (read-only)
 models: Qwen3-Embedding-0.6B · qmd-query-expansion-1.7B · Qwen3-Reranker-0.6B  (local GGUF, CPU/GPU)

On the mairp host there is also a shared, always-on, writable gateway layer that fans this same qmd out to every coding harness (Claude/bebop, pi, OpenClaw, CLIs):

 any agent (MCP)   ──► qmd-gateway.service ──► mcp-proxy SSE :8190 ──► one warm `qmd mcp`
 pi / CLI (bash)   ──► recall.sh (read) ────────────────────────────► shared qmd-shared index
 any agent (write) ──► qmd-remember.sh ──► /root/fleet-memory/** ──► reindex ──► fleet-ops (RW)

The shared index adds a writable fleet-ops collection (cross-agent memory) alongside read-only shares of Claude + OpenClaw memory. See docs/ARCHITECTURE.md and /root/qmd-gateway/README.md for the attach matrix. This repo remains the source of the recall/reindex logic both layers use.

Benchmark & vector-space visualization

The bench/ harness measures retrieval quality (recall@k / MRR / F1 / latency) across bm25 / vector / hybrid / full backends over a hand-built gold set, and evaluates an adaptive "grading/retry" escalation policy. It also projects the memory corpus + benchmark queries into 2D/3D (PCA of the Qwen3 embeddings) — docs colored by collection, queries by type (open marker = retrieval miss):

Finding: on this corpus plain vector search wins (recall@3 ≈ 0.82, ~100 ms) and beats the expansion+rerank pipelines (≈ 0.77, ~22 s) — so a grading/retry wrapper isn't worth it. See bench/report.md. Regenerate the viz with node bench/embed-project.mjs and the demo GIF with bench/render-demo.sh; a live Grafana version ships as bench/grafana/.

License

MIT — see LICENSE. No secrets or host-specific credentials are included; configure via flags or a local .env (see .env.example).