[Bug]: Models ignore configured context window size

[The-unknown-Shadowman]

It appears the context window setting is not being respected; based on my observations, responses to prompts containing longer text snippets do not seem to consider the entirety of that text content regardless of the size of the context window configured in the settings. Only the Gemma 3 models exhibit behavior suggesting they are aware of a limited context window ("Completion failed: Context is full"), implying other models are effectively ignoring the specified parameter, which significantly impacts usability for tasks requiring understanding of longer documents.

[The-unknown-Shadowman]

It's better now, but with Gemma 3 12B I still get the message. Maybe it's because there is no rolling context window or truncate middle? Or is there an option I don't see?

[a-ghorbani]

text only or with image?

[a-ghorbani]

It's better now, but with Gemma 3 12B I still get the message. Maybe it's because there is no rolling context window or truncate middle? Or is there an option I don't see?

What error do you see, is it “Completion failed: Context is full”?

I tested https://huggingface.co/bartowski/google_gemma-3-12b-it-GGUF/blob/main/google_gemma-3-12b-it-Q5_K_M.gguf The generated text here is around 2k tokens, and I haven’t run into any issues yet. Could you let me know how to reproduce the problem you see?

Simulator.Screen.Recording.-.iPhone.16.Pro.-.2025-08-20.at.20.43.12.mov

[The-unknown-Shadowman]

In the settings you can set the context size for the model. I set mine to 4096 token. After filling it up with text and multimodal use with images it responds “Completion failed: Context is full”. I assuem it's because there is no cutting off the first messages in a rolling manner or truncating the middle of the chat. I can send you an export of a chat json file.