feat: Option to set "Full context mode" for file upload and knowledge retrieval individually.

[deliciousbob]

Check Existing Issues

• I have searched all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Problem Description

Focus Retrieval mode does not work well on uploaded documents, we noticed that switching to full context / full document mode at the uploaded document fixed that issue and requests give good answers.

As discussed in the Issue #18581 the Option would be to enable full context mode:

• Settings -> Documents -> "enable full context mode" -> enable Option
  - this is then also disabling RAG + Hybrid Search (reranking)

If you upload a single document in the chat, it still showes that focused retrieval mode is active (maybe a bug)
It does the "Retrieval Query Generation" still, but actually it pass the full document context to the LLM.
Just checked the LiteLLM Logs. Showed me 65K tokens on the request. That is what we want, nice!

BUT..... for Knowledge Collection this is causing a retrieval of ALL Documents from the Collection.
This is somehow expected as the RAG + Hybrid Search is disabled too.

So the "enable full context mode" does actually work for Document upload (only shows wrong if you click onto the file)
But i cannot activate it, as it would cause RAG for Knowledge Collections to not work anymore.

Desired Solution you'd like

So, having an option to set that individually for file uploads and knowledge collections would be a feature i would desire a lot.

Thx, best regards, Robert

Alternatives Considered

We could also discuss if disabling RAG on Knowledge Collections makes sense at all (maybe made sense some early versions) ?
Cannot imagine using Knowledge collections with passing all documents to the LLM makes any sense to anyone.
So maybe it would be wise to rename the Option from "full context mode" to "full context mode for uploaded files" and only apply that to the file upload retrieval without touching the RAG + Hybrid Search Options for Knowledge Collections.

Additional Context

No response

[deliciousbob]

We use OWUI since more than a year now in production.
We recently did a survey with our users about our internal OWUI Solution, to see what could be improved.
The biggest complaints we got about file uploads. Some went into detail and shared their experience with us:

• File upload does not work - most of the time the response is: "there is no document..."
• I tried to use the AI, but it would not recognize files very well, ChatGPT does a great job
• I would like to work with my files, but often I get a general response, it does not answer.

We get these responses all the time, I tried to upload this and ask this, but it did not work well, can you help me, what did I do wrong.
Most of the time we tell the users to use the "entire document" and then it magically works well for them.
I cannot tell how often I told the users to use the "entire" document... It would save a lot of work if we have an option to make it default to pass through the entire document.

If there was a logic, that checks after extracting the text:

IF (doc.length + 10% > model.context-length)
    USE retrieval-mode
ELSE use entire-docuemnt-mode

If there would be an option in the Admin-Settings to set the "entire-document-threshold", in thousand tokens.
You could steer that yourself -> If threshold is reached -> use retrieval-mode otherwise use the full context of a document.

Making full context for uploaded documents the default would be a 5 star quality improvement for me and all my users.

[mahenning]

You can manually enable full context mode for uploaded files, but right now it can not be set as default.

The logic check sounds nice, but the problem is that at the document extraction step (the step where the document length would be known), OWUI does not know the model context length or the model tokenizer. Also, what if the document is uploaded mid-chat? How is it handled then with the length? Would it be okay if the document fills 90% of the context window?

[deliciousbob]

Hi @mahenning Marko,
I would be totally happy to have a Environment Variable that defines that "entire document" mode is the default for uploaded files. So users have to switch to "retrieval mode" manually everytime.
That would be a 5 star quality improvement 👍

I totally agree with your points that you cannot dynamically calculate that when there is already some content on the chat.
Also It is further complex if multiple documents are uploaded within the chat.

[deliciousbob]

The point is, we can teach the users to change to "full context" everytime they upload a file, but they will forget about that and run into issues. Working with uploaded documents works very well with full context mode, we have very good response from our users.

Retrieval mode for asking against a "book-sized" pdfs is necessary, i totally agree, but for smaller pdfs it causes a lot of issues.

I could imagine the following would also work well:
If a document has for example more then 30K Tokens in size, and flag is displayed to remind the user to "use to retrieval mode as the LLM could be overloaded with context". So the user can decide, which mode to use.

[mahenning]

I could imagine the following would also work well:
If a document has for example more then 30K Tokens in size, and flag is displayed to remind the user to "use to retrieval mode as the LLM could be overloaded with context". So the user can decide, which mode to use.

While the idea is nice, the problem is that at the point where the length of the document is known the document is already uploaded and submitted. So to display a warning that the user should switch to focus mode, the document has to be processed already at the "dropped in for chatting about" step, before submitting it with a message. This potentially includes tools like Docling/OCR, too.

I don't really see a way to implement the auto decide for full context / focused retrieval. There are too many changing variables for that. When there is a fixed threshold in the admin settings, what if users have model A with 8K context and model B with 128K context available? What if the chat is already running with thousands of token? What if multiple documents are uploaded, or worse: multiple documents uploaded one after the other? Does doc 2 potentially kick out doc 1 when consuming the whole context window?

I support a env var / setting to default uploaded documents to full context. I have the same problem with my users, that the setting for full context mode is hidden and very easily forgotten. Also, they need a rule of thumb when to use it and when not to use it (like "only for documents with less than 20 pages of text")

[deliciousbob]

OK, i get it, the tokens are not yet known, but the lines seem to be available after the upload:

Then you could define the logic -> if document > 2000 lines --> then show a flag that informs the user that the context might be to big.
But as I already mentioned :) that would be the cherry on the cake.

To have an Env. Variable to steer the behavior of having full context or retrieval mode by default would be awesome and would help a lot on usability.

[wsomm]

I have the same issue as deliciousbob.

It is far more easy to tell a small number of users "your file is too big, you have to do this", than to tell a large number of users "right click on the file and choose full context".

I would also appreciate a env variable for file upload "retrieval/full context" very much!

Is it that hard?

[mahenning]

I made a PR to at least set the default for uploaded documents to full content mode: #19166
Collections/Knowledge bases are still focused retrieval by default and ignore this setting. Users can still switch back to focused retrieval for each document.

[deliciousbob]

Thx Marko, I did not expect that quick rework :) but your PR is very welcome. I hope that Tim is willing to commit your changes very soon.
Best regards, Robert

[Classic298]

I have a related PR that allows admins to configure interface Default settings for users

Besides i understand Tim's reasoning. This should probably be a user sided toggle

[tjbck]

• If anything one thing you could do to immediately mitigate this "issue" would be creating a Filter Function for this

[deliciousbob]

I also understand Tim's reasons, some settings should be user-related, but I cannot count anymore how often I wished to have a full context by default. As we get the requests all the time...
For most standard questions like "summarize that pls", "what is the document about",... there is no related context to that in most documents... response from the AI --> "no document found".
The full context solves all these issues....

Most users just silently drop that system if it does not work as expected, for them, it is the most basic thing to get a document summarized...
So why don't we just allow full context for all, it is the admins decision to spend more tokens for better usability or save some tokens.
By today's standards, 128K context is more than enough to handle bigger PDFs, a year ago 8K was the standard, I fully understand why that was necessary to have retrieval mode as default.
Just give us the options to change the default behavior, an Environment Variable alone would be all I could wish... but I know that a lot of other users feel the same pain with uploaded documents... it simply works well with full context mode.

[deliciousbob]

To be honest, I am fine with a user side toggle as long as it gets implemented. :)
I will probably find a way to override that in the pg database to get that enabled for all users.

[mahenning]

See my comment in my PR. I believe that most of my users will not set it, as they dont know what it does, or does not find it. If there is a default value that is good for 90% of my userbase, I want to set it as default for everyone, not telling everyone to set it separately. That a user can toggle it back for themself is totally fine, but I dont see any harm to toggle the default as an admin.

[Classic298]

@mahenning my PR allows you to override the default values for all users.

[mahenning]

Then ignore my comment, I will re-read you PR. Great to hear! But will this setting also be in your PR? Just wanted to make sure as I didn't find it in the text and wasn't sure this here counts as a "user setting" as not even user can set the default now.

[Classic298]

yes it is already

[Classic298]

@mahenning my PR introduces a button to override all manual changes a user has made to the interface settings and force the admin-configured default settings - and also it introduces an permission toggle to hide the interface settings from users for example.

But yes, most importantly, you can change the interface settings.

[Classic298]

Soooooo

@deliciousbob @mahenning

you guys wanna make a new PR to implement this as a ... user sided interface setting after my PR for modifying the interface settings from admin panel was merged? sounds like the ideal solution for you then? :D

[mahenning]

Sure, does sound easy. Will watch your PR status.

[wsomm]

Did I miss something? I cannot see, that there is a solution in 0.6.37. What did I overlook?

[mahenning]

You didn't overlook anything, there is no solution yet merged. I made a fast PR (#19166) that got rejected in favor of a broader solution. Currently there is an open PR (#19295) in which an admin could overwrite every default of a user setting. When that PR is merged, the "focused / full content" default has to be added as a user setting in another PR, but then an admin can change that default for every user because of #19295.

This will take longer than merging my PR, but it will be a cleaner solution which lays the foundation to do this kind of stuff with other setting options too.

[deliciousbob]

Hi Guys :) really looking forward to the PR is getting merged into Prod! (#19954)

[Classic298]

FYI everyone: #18684

and with this comment here: #18684 (comment) you can set the interface settings for all users for now

[d1ffuser]

I'm sorry if there is some kind of solution I missed, but what about having two options when dragging files into the chat windows — like a splitscreen (left, query mode; right, full context)? I don't if it's possible to filter for file extensions...

In any case, if there is no way to have RAG as well as attach full files as default, this is an issue.

[Classic298]

@d1ffuser after you drag a file into the chat, you can click on the file and enable full context mode OR query mode as you want already

[d1ffuser]

I'm aware — but you have to do this every time you upload a file. I think that's the point of this discussion.
Other LLM interfaces treat file upload differently and users are either confused or forget to switch it on. Overall, it's a bad user experience.

[wsomm]

@Classic298

after you drag a file into the chat, you can click on the file and enable full context mode OR query mode as you want already

Yes, that works.

But my user base have zero technical expertise. It isn't an issue for "us", but they will not learn it.
(Most of them struggle even to choose another model than the default option Gemma 4...)

That is the reason, why I have the wish, to have the ability to change the default to "full context".
Just that. Just change the default. Somewhere.

For now, my workaround is to patch utils.py and hard code it to "full context".
But the possibility to change the default would be the clean solution.

[Classic298]

@wsomm i agree

should continue discussion here

#18431

[Classic298]

but i wanna add setting it to default full context might be dangerous. Our users upload PDFs with 20000 pages. thats simply too much