[FEAT]: Automatically pin documents that have been found based on vector search for improved RAG

[illov]

What would you like to see?

A option in the workspace: If a related snippet is found in the vector search then that document is fed to the LLM in full as a pinned document to get the full context. Also in this approach the snippet limit should me removed because all documents that have info should be found.

[timothycarambat]

This would blow up almost any persons local LLM use (or even cloud) since you could go from n tokens with y context, but the document y comes from is 100K tokens your next request would be n + 100K suddenly.

For cloud users -> This would inflate the per query costs to something insane for long-turn chats.
For Local -> This would spike all resources and result in a full lockup or a crash since we would be max-context filling.

This also ignores how we do pruning to make sure context works - so now you will lose history, system prompt content, and document snippets.

For this, turn on re-ranking. It is the closest you can get to full-context without blowing up your LLM costs or your server running the LLM. The nuance to enable this is too complex for a toggle for a user - so that is why we show a big warning model when you pin a document for the first time. Pinning should always be a last resort or a tool for people who know what they are doing.