SearchBasedRAG

---

title: Search-based RAG radar_quadrant: Techniques radar_ring: Assess radar_position: inner

Search-based RAG

Search-based RAG is the practice of using keyword (BM25/full-text) search rather than vector/semantic search as the retrieval layer in a retrieval-augmented generation pipeline. The user's question is first translated into one or more search queries by an LLM, those queries are run against a full-text search index, and the top results are fed back to the LLM to generate an answer. The technique was demonstrated by Simon Willison in June 2024 using SQLite FTS via Datasette as the search backend and Claude 3.5 Sonnet as the generation model.

Implementation Pattern

The retrieval step uses SQLite's built-in FTS5 (BM25 ranking) rather than an embedding model and vector store:

SELECT id, title, body, created FROM documents
JOIN documents_fts ON documents_fts.rowid = documents.rowid
WHERE documents_fts MATCH :search
ORDER BY rank LIMIT 10

An LLM generates the search queries from the user's natural-language question before this step runs. The LLM-as-query-generator compensates for BM25's lack of semantic understanding: "What did Einstein say about time?" becomes Einstein time relativity as a search term.

When to Choose Keyword Search over Vector Search

Keyword (BM25) search outperforms semantic/vector search in several conditions:

• Factual, structured queries — proper nouns, technical terms, model names, version numbers, error codes
• Code search — function names and identifiers match exactly; embeddings add noise
• Small-to-medium corpora — vector search infrastructure overhead is not justified
• Zero additional dependencies — SQLite FTS is bundled in Python's standard library; no embedding model, no vector store, no GPU

Vector search outperforms BM25 for conceptual or paraphrased queries where the exact words don't appear in the documents.

Hybrid RAG

The strongest production retrieval combines both: BM25 for precision on known terms, vector search for semantic recall on conceptual queries. Results from both retrievers are merged and re-ranked before the LLM generation step. This pattern is documented in the Year of Building with LLMs blip.

Radar Assessment

Placed in Techniques / Assess / inner.

Search-based RAG is the overlooked counterpart to vector search. Most RAG tutorials default to embeddings without considering whether keyword search would suffice or outperform for the specific query distribution. For corpora of technical documentation, code, or named entities — common in internal tooling — BM25 precision often exceeds semantic recall.

SQLite FTS requires no new dependencies, no embedding model, and no chunking strategy to tune. It is immediately applicable to any project that already stores text in SQLite (or can). The LLM-as-query-generator step is the only addition to a standard search pipeline.

Complements DuckDB Vector Search (semantic retrieval), RAG Chunking Strategies (chunk preparation for vector RAG), and LLM Evaluation Methodology (measuring retrieval precision across approaches). Together these three blips define the decision space for RAG retrieval layer design.

Trial gate: a working RAG pipeline using SQLite FTS as the retrieval backend, with retrieval quality compared against at least one alternative approach (vector search or naive context stuffing) on a representative query set.