-
Notifications
You must be signed in to change notification settings - Fork 1
MiniLM Embeddings
Chris Sweet edited this page Jul 9, 2026
·
1 revision
type: concept up: "Markov-Retrieval-Tutorial" tags: [embeddings, model, encoder] related: ["Heterogeneous-Graph", "Vanilla-RAG-Failure-Mode"]
The tutorial uses all-MiniLM-L6-v2 from sentence-transformers as its single embedding model.
The same model encodes both chunk text and concept strings. This means chunk-to-chunk, concept-to-concept, and chunk-to-concept similarities all live in one shared cosine-similarity space, with no learned cross-encoder or reranker between them.
| Item | Value |
|---|---|
| Model | sentence-transformers/all-MiniLM-L6-v2 |
| Embedding dim | 384 |
| Cache location | ~/.cache/huggingface |
| First-run download | About 90MB |
| Subsequent runs | Offline |
-
graph.pyembeds every chunk and every concept once. -
build_transition_matrixinmarkov.pyturns pairwise cosine similarity into edge weights, applying a sparsification threshold so only sufficiently similar pairs become edges. -
DIAGNOSTIC_CONCEPT_PAIRSincorpus.pyrecords pairs whose cosine similarity is printed as a diagnostic, useful for understanding what the encoder actually treats as related.
MiniLM is small and general-purpose. It will mark some plausibly related concepts as far apart and some unrelated concepts as close. The diagnostic pairs are how the tutorial exposes this: when retrieval misses a connection, the path-length decomposition and the concept similarity printout often together explain why.