Full content of document not returned

[mhughes2k]

Bug Report

Description

Bug Summary:
Full content of documents is not returned due to "RAG"ging of the internal content.

I appreciate that this is suitable for summarisation across documents, and the internal chunks should be generated, this seems to be incorrect behavior / context to provide where the task is potentially working upon the whole content of a document, and thus a similarity based approach may not be appropriate.

Steps to Reproduce:

1. A CSV file with approximately 159 rows in it was uploaded to the document area
2. A new chat was started using the llama3 model
3. The csv file was referenced by an # statement to tell the prompt to use that single document
4. Prompt to "summarise this csv" file
5. LLM responses with a statement "...There are five records in total,..."
6. openwebui log indicates context passed to the LLM following RAG, has been a similarity search of the contents of the file, and thus limited the top-k results:
   INFO:apps.rag.utils:query_doc:result {'ids': [['f83b3943-9854-4806-8112-3c1b1d276a99', 'f85bea6f-e3b1-4c27-835a-dea22f986d97', 'a79509b0-ea09-41bf-b155-ca18c495ba65', '4b0b10b9-ce41-4439-9742-532de27ca972', '47d39c80-14e1-40a3-9816-5423c88b98a9']], 'distances': [[1.431112589434609, 1.4432387351989746, 1.4436806440353394, 1.443746566772461, 1.4461387395858765]], 'metadatas': [[{'row': 102, 'source': '/app/backend/data/uploads/charging_sessions.csv', 'start_index': 0}, {'row': 42, 'source': '/app/backend/data/uploads/charging_sessions.csv', 'start_index': 0}, {'row': 2, 'source': '/app/backend/data/uploads/charging_sessions.csv', 'start_index': 0}, {'row': 32, 'source': '/app/backend/data/uploads/charging_sessions.csv', 'start_index': 0}, {'row': 1, 'source': '/app/backend/data/uploads/charging_sessions.csv', 'start_index': 0}]], 'embeddings': None, 'documents': [['Start: 2023-09-07 16:16:20\nEnd: 2023-09-07 18:53:43\nDuration: 02:37:23\nSite: Craigpark\nConnector Type: AC\nPayment Status: Paid\nConsum(kWh): 28.04\nCurrency: £\nAmount: 12.22', 'Start: 2024-02-11 18:34:42\nEnd: 2024-02-11 21:09:04\nDuration: 02:34:22\nSite: Craigpark\nConnector Type: AC\nPayment Status: Paid\nConsum(kWh): 28.28\nCurrency: £\nAmount: 12.31', 'Start: 2024-05-08 20:06:46\nEnd: 2024-05-08 22:06:23\nDuration: 01:59:37\nSite: Craigpark\nConnector Type: AC\nPayment Status: Paid\nConsum(kWh): 22.02\nCurrency: £\nAmount: 9.81', 'Start: 2024-03-09 11:14:00\nEnd: 2024-03-09 13:51:40\nDuration: 02:37:40\nSite: Craigpark\nConnector Type: AC\nPayment Status: Paid\nConsum(kWh): 29.32\nCurrency: £\nAmount: 12.73', 'Start: 2024-05-11 09:15:14\nEnd: 2024-05-11 11:05:00\nDuration: 01:49:46\nSite: Craigpark\nConnector Type: AC\nPayment Status: Paid\nConsum(kWh): 20.29\nCurrency: £\nAmount: 9.12']], 'uris': None, 'data': None}

Expected Behavior:
Given the single file nature, the contents of the file should be processed.

Actual Behavior:
LLM reponds with statement indicating fewer rows in the document that reality.

Environment

• Open WebUI Version: 0.1.124
• Ollama (if applicable): 0.1.37
• Operating System: Windows, WSL & Docker
• Browser (if applicable): Brave Version 1.65.133 Chromium: 124.0.6367.208 (Official Build) (64-bit)

Reproduction Details

Confirmation:

• I have read and followed all the instructions provided in the README.md.
• I am on the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
browser.log

Docker Container Logs:
docker.logs.txt

Screenshots (if applicable):

LLM resonse

Sample of CSV format > 5 rows identified in response.

Installation Method

Docker

Additional Information

[tjbck]

Would adjusting these params suffice to cover your use case?

[mhughes2k]

I amended it to a chunk size of 3000 and it still returned only 5 rows.

I don't think the issue is the size of the chunks per se, but the fact that the process appears to be using the user prompt to similarity match the items from within the "whole" document, which if it has been split into smaller chunks (at upload), because it is using the query_doc() function, which is inherently going to return only the "k" number of items from the vector store.

Again "fine" for textual content, but less for for data formats, where each row (potentially) isn't relevant on it's own, but part of the "holistic" whole CSV file.

However there may be cases where treating a CSV "like" a document does make sense, and if the "intention" can't be inferred then I guess there may need to be some sort of mechanism for the user to specify that their are desiring the "file" to be used as the context, not the RAG / vector search to be.

I've not (yet) looked into what processing is done to a CSV file on upload to put it in to the underlying vector store database, from what I've experienced here, and the output that is logged from query_doc() it suggests that each row in the CSV is turned into a document, of which the content has been synthesied from the CSV header row values and the values for each row.

This is based at looking at the serialised data in the log and the "documents" element.

If this is the case then the context provided to the LLM can never be the full CSV data, but only the "k" rows that are a "close" match to the user's prompt...which seems to be counter-productive.

[mhughes2k]

Looking at https://python.langchain.com/v0.1/docs/modules/data_connection/document_loaders/csv/ which does indicate the CSV is split in to a series of documents per row with a "source_column" attribute to collate them together, this would suggest that solely relying on the RAG search results for context is insufficient for this "sort" of query against a (single) file.

So in the case of using the # function (or upload, as this seems to happen with a directly file upload in a chat, as well as a pre-loaded one in the documents area) to link a document or document collection to the chat, where the specified target is a "single" item, then that is loaded in addition to RAG results or in preference to RAG results, and where the selected target is a "collection" then RAG is used exclusively.

[apasila]

Very similar problems here. I can't upload a document from chat, need to do it from documents. When I upload 20 PDFs it looks like everything is uploaded very fast but when I start talking with the docs it just uses the first page of the first document and rest it doesn't seem to know anything about. How do I know RAG has ingested my documents and they are read?

[apasila]

Now I received this:

I apologize, but since there is no provided context within the XML tags, I'm unable to summarize any content.
If you meant to provide additional context or attach a file, please feel free to do so, and I'll be happy to assist you.

Couild I have screwed up some setting?

[mhughes2k]

Any movement on the thoughts on this?