Embedding models

Embedding models

Hybrid recall pairs keyword search with dense vectors from a small embedding model. The release binaries embed with candle and default to BAAI/bge-small-en-v1.5 (about 130 MB), downloaded from HuggingFace once on first use and cached.

dmem doctor          # active embedder, model, cache dir, whether it is already cached, CPU features
dmem doctor --json   # the same, machine-parseable

Offline / air-gapped

Pre-populate the model cache once on a connected machine, then carry it over (or point at a shared path):

# Pre-warm the cache (run once with network), then start dmem offline:
HF_HOME=/srv/hf-cache python -c \
  "from huggingface_hub import snapshot_download; snapshot_download('BAAI/bge-small-en-v1.5')"
HF_HOME=/srv/hf-cache dmem serve --addr 127.0.0.1:8088

dmem honours HF_HOME and HUGGINGFACE_HUB_CACHE (the standard HuggingFace cache), and dmem serve logs the cache dir and model on startup. dmem doctor prints the exact directory it expects and whether the model is present, so you know up front if a first run needs network.

Multilingual and custom models

The default model is English. If your memory is bilingual (queries in one language should recall records written in another), swap in a multilingual checkpoint by env, no recompile. Any 384-d BertModel-architecture checkpoint on HuggingFace works. Needs a build newer than 0.1.1.

export DM_CANDLE_MODEL="intfloat/multilingual-e5-small"
export DM_CANDLE_POOLING=mean            # e5 and paraphrase-multilingual are mean-pooled (default: CLS, the bge way)
export DM_CANDLE_PREFIX_QUERY="query: "  # e5-style role prefixes
export DM_CANDLE_PREFIX_DOC="passage: "

Set the pooling and prefixes to whatever your model's card documents; the defaults (CLS pooling, no prefixes) match the bge family exactly. The doc prefix is injected into every chunk window of a long record, not just the first, so late windows keep their role marker. With a model override the embedder reports itself as candle-custom in dmem doctor and the logs, since similarity scores are model-relative and should not be read against bge-calibrated expectations.

These variables take effect wherever the embedding happens: the server process in client/server mode, or your local process in embedded mode. Set them for the service (systemd/launchd unit) if you run one, not just your shell.

After switching models, re-embed the existing records so old and new vectors live in one space:

dmem reindex-embeddings

Footprint

DM_CANDLE_F16=1 loads the weights as f16, halving the model's RAM. The fastembed build feature is an alternative embedder behind its own flag; the release binaries ship candle.