Can a RAG reference back to its source document?

[bilogic]

Basically as the title says, because the FileVectorStore stores by lines, even knowing which line was the source will help

[ilvalerione]

The RAG component injects the retreived document into the system prompt along with its souce type and source name: https://github.com/inspector-apm/neuron-ai/blob/main/src/RAG/RAG.php#L125

Be sure to have the latest version 1.15.14

With a bit of good prompting I think you can get what you want asking the agent for a structured output.

The structured output class should ask for the "citations". Somthing like:

namespace App\Output;

class CitedAnswer
{
    #[SchemaProperty(description: 'The answer to the user question', required: true)]
    public string $answer;

    /**
     * @var \App\Output\Citations[]
     */
    #[ArrayOf(Citations::class)]
    public array $citations:
}

namespace App\Output;

class Citations
{
    #[SchemaProperty(description: 'The type of the source', required: true)]
    public string $type;

    #[SchemaProperty(description: 'The name of the source', required: true)]
    public string $name:
}

And than call the agent with structured method to enforce the agent includes also the reference to the original document:

$response = MyRAG::make()->structured(
    new UserMessage(...),
    CitedAnswer::class
);

var_dump($response);

The agent should be able to attach the source information of the documents used to generate the final answer.

It is just as an inspiration. You can play with this to catch the structure that guarantees you the details you want.

[bilogic]

Ok, thanks, that clears up alot, for a moment, I was wondering how big tech managed to agree on making embeddings inter-operatable. It also seems to me that LLMs are basically stateless and the reason it has context is because we fed it with instructions and the chat history.

1. Are all these context information re-uploaded to the LLMs with each new interaction with the user?
2. Would you also be able to explain how the LLM is able to search our vector store? (Is it re-uploaded with every request?)

[ilvalerione]

Yes LLMs are completely stateless, so all the contextual information (system instructions, chat history, tools definition, etc) are always sent to te model on every interactions.

It's not the LLM that searches into the vector store, it's NeuronAI. That's why you use a framework to create a RAG system.

[bilogic]

Yes LLMs are completely stateless, so all the contextual information (system instructions, chat history, tools definition, etc) are always sent to te model on every interactions.
It's not the LLM that searches into the vector store, it's NeuronAI. That's why you use a framework to create a RAG system.

I guess my question then becomes, does the whole vector store gets uploaded on every interaction as well? If yes, that seems to be a bottleneck if I had to greatly increase my documents correct?

[ilvalerione]

If you need to learn the details of how RAG works you should look into the Neuron codebase. When you call the chat method of a RAG agent Neuron first perfoms a search request against the vector store and includes only the topK documents into the system instructions to give the LLM the necessary information to answer the question: https://github.com/inspector-apm/neuron-ai/blob/develop/src/RAG/RAG.php#L68

That's why the vector store components in NeuronAI have the "topk" arguments in the constructor. It's the way you can customize the number of documents to provide to the LLM and it obviously affect tokens consumption.

[bilogic]

Got it, thank you so much!