GRAG

Generic RAG

Structure

Sub-package implementation when langchain doesn't provide a adequate implementation

src/ init.py cli.py - Command line interface for direct calls port_1/ init.py - Interface ABC adapter_1.py - Implementation adapter_2.py - Implementation port_2/ init.py - Interface ABC adapter/ v1.py - Implementation v2.py - Implementation v3.py - Implementation adapter_2.py - Implementation

Current packages:

src/ chunker/ - Chunking interfaces and naive implementation document/ - Document container embedding/ - Embedding interfaces and naive fastembed adapter querying/ - Querying interfaces and naive retriever reader/ - Source readers store/ - Store interfaces and Chroma adapter

Key decisions

• Avoid LlamaIndex, too much magic and no fine control over the process
• We will develop custom connectors, because LangChain connectors are not enough advanced for complex documents

Important concepts within RAG

• Loading: this refers to getting your data from where it lives — whether it’s text files, PDFs, another website, a database, or an API — into your workflow.
• Indexing: this means creating a data structure that allows for querying the data. For LLMs this nearly always means creating vector embeddings, numerical representations of the meaning of your data, as well as numerous other metadata strategies to make it easy to accurately find contextually relevant data.
• Storing: once your data is indexed you will almost always want to store your index, as well as other metadata, to avoid having to re-index it.
• Querying: for any given indexing strategy there are many ways you can utilize LLMs and data structures to query, including sub-queries, multi-step queries and hybrid strategies.
• Evaluation: a critical step in any flow is checking how effective it is relative to other strategies, or when you make changes. Evaluation provides objective measures of how accurate, faithful and fast your responses to queries are.

There are also some terms you’ll encounter that refer to steps within each of these stages.

Loading stage

Nodes and Documents: A Document is a container around any data source - for instance, a PDF, an API output, or retrieve data from a database. A Node is the atomic unit of data and represents a “chunk” of a source Document. Nodes have metadata that relate them to the document they are in and to other nodes.

Connectors: A data connector (often called a Reader) ingests data from different data sources and data formats into Documents and Nodes.

Indexing Stage

Indexes: Once you’ve ingested your data, It is transformed into a structure that’s easy to retrieve. This usually involves generating vector embeddings which are stored in a specialized database called a vector store. Indexes can also store a variety of metadata about your data.

Embeddings: LLMs generate numerical representations of data called embeddings. When filtering your data for relevance, Embedder will convert queries into embeddings, and your vector store will find data that is numerically similar to the embedding of your query.

Querying Stage

Retrievers: A retriever defines how to efficiently retrieve relevant context from an index when given a query. Your retrieval strategy is key to the relevancy of the data retrieved and the efficiency with which it’s done.

Routers: A router determines which retriever will be used to retrieve relevant context from the knowledge base. More specifically, the RouterRetriever class, is responsible for selecting one or multiple candidate retrievers to execute a query. They use a selector to choose the best option based on each candidate’s metadata and the query.

Node Postprocessors: A node postprocessor takes in a set of retrieved nodes and applies transformations, filtering, or re-ranking logic to them.

Response Synthesizers: A response synthesizer generates a response from an LLM, using a user query and a given set of retrieved text chunks.

Querying baseline

The first querying implementation is intentionally naive and LLM-free. It exists as a baseline for future retrieval iterations.

• src.querying.QueryRequest stores the query text, top_k, and optional metadata filters.
• src.querying.RetrievedNode is the normalized retrieved context shape: id, text, metadata, rank, and distance.
• src.querying.QueryResult returns the original query and retrieved nodes.
• src.querying.Retriever is the main querying port.
• src.querying.NodePostprocessor is a future hook for filtering, transformation, or re-ranking.
• src.querying.Router is a future hook for selecting one or more retrievers.
• src.querying.naive.NaiveRetriever calls Store.search(...) and maps store results into RetrievedNode objects.

The baseline retrieves stored document records. If chunk-level retrieval is required, the indexing stage should store chunk-sized Document objects before querying.

Tests

The querying baseline has three test classes:

• Normal tests use local fixture data from tests/fixtures/querying and deterministic embeddings.
• Real tests use live external sources and are skipped unless REAL_TESTS=1 is set.
• Evaluation tests use RAGAS retrieval-only metrics and local statistics to compare implementations over the same fixture data.

Evaluation is focused on retrieval accuracy and metadata accuracy. It does not evaluate generated answers and does not use an LLM.

Useful commands:

pytest
pytest -m evaluation -s
REAL_TESTS=1 pytest

Reference

• https://bytevagabond.com/post/how-to-build-enterprise-ai-rag/
• https://github.com/NirDiamant/RAG_Techniques
• https://arxiv.org/abs/2210.03629
• https://developers.llamaindex.ai/python/framework/understanding/rag/