issue: num_keep Setting Doesn't Limit Context Retention as Expected

[weissfl]

Check Existing Issues

• I have searched the existing issues and discussions.
• I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.5.20

Ollama Version (if applicable)

No response

Operating System

Windows 10

Browser (if applicable)

No response

Confirmation

• I have read and followed all instructions in README.md.
• I am using the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.
• I have listed steps to reproduce the bug in detail.

Expected Behavior

When setting "Tokens To Keep On Context Refresh (num_keep)" to 1, the model should retain only 1 token from previous messages. If asked to recall earlier information, it should not remember details beyond this limit.

Actual Behavior

Despite setting "Tokens To Keep On Context Refresh (num_keep)" to 1, the model retains more than 1 token and successfully recalls information from previous messages.

Steps to Reproduce

1. Go to Chat Controls / Settings / Advanced Params (I tried all three options).
2. Find the "Tokens To Keep On Context Refresh (num_keep)" setting.
3. Change it from default to custom and set it to 1.
4. Start a conversation and mention your name in one of the first messages.
5. Send a few more messages.
6. Ask the model to recall your name.

Unexpected Behavior:
The model still remembers the name, indicating that more than 1 token is retained, despite the setting.

Logs & Screenshots

Additional Information

No response

[frenzybiscuit]

Are you using OpenAI API or Ollama?

Because the answer to this question is that it's likely an ollama only feature that wasn't properly marked as such.

[weissfl]

Oh, I'm using the OpenAI API. If that's an Ollama-only feature, that explains why it didn't work. Though honestly, it'd be great to have this in the API too - there's no need to shuttle the entire context back and forth every time, plus it just burns through tokens.

[weissfl]

That said, the num_predict parameter works fine with the OpenAI API, so maybe this isn’t strictly an Ollama thing after all?

[rgaricano]

and bad explained, it keep first N of the conversation.

Advanced Params
#3794

[weissfl]

Thanks for the clarification! That makes a lot more sense now. So if I got it right, it keeps the beginning of the conversation to improve accuracy? I guess I mixed it up with Context Length then.