Adjustable context length for local models

[A-Rutledge]

I’ve been testing the meeting notes feature with local models, and the Chat tab can’t handle a normal-length meeting. A 30-minute 1:1 transcript was enough to hit this error on the first question:

request (18052 tokens) exceeds the available context size (16384 tokens), try increasing it

I couldn’t find a context length setting anywhere in the UI, so it seems fixed at 16K for local models.

Request: an adjustable context length for local models, ideally per tab (Chat and Note Formatting need it most; Dictation Cleanup is fine at 16K). A warning about the extra VRAM it uses would make sense too. As an alternative, automatically chunking or summarizing long transcripts before sending them to chat would also solve it.

Setup: OpenWhispr 1.10.2 on Windows 10, local Qwen3.5 4B with GPU acceleration (Vulkan) on an RTX 2070 8GB.

[naborajs]

Yeah, I really think we should also have an option to adjust the context window size. I use a local model for speech-to-text that's only around 864 MB in size, yet it still consumes approximately 1.5 GB+ of VRAM. Having control over the context window would help us reduce unnecessary VRAM usage and optimize performance, especially when running smaller local models.