Add context window size to Custom Model

[ackalker]

Is your feature request related to a problem? Please describe.
The current documentation in https://github.com/logancyang/obsidian-copilot/blob/master/local_copilot.md instructs Ollama users to use ollama run <model> and /set parameter num_ctx <value> to change the context window size for Ollama models. This is IMHO way too cumbersome and hardly necessary, as the desired context size can be specified in each API request. The Ollama server will (re)initialize the model with new parameter values (including num_ctx) when it detects a change from their default or previous values.

Describe the solution you'd like

• Add a Context window size field in the Ollama section of the plugin settings where users can fill in their desired context window size.
• If the field is non-empty, send its value as a key/value pair "num_ctx": <value> in the options dict in every chat API request (remember to send it in every request, or else the server will revert back to default model parameter settings for any parameters left out). See: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion (Advanced parameters) and here: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values .
• If the field is left empty, don't send num_ctx in the chat API request's options dict, the server will use the model default in that case.

Describe alternatives you've considered
Explain to users how they can create their own Ollama models derived from base models using ollama create <model> with a Modelfile containing the desired changes. This can even be done programmatically, instead of running each and every model in turn using ollama run <model> to set parameters.

Additional context

[logancyang]

This was my request to ollama some time ago and seems it's possible now.

One question is that when you see this num_ctx in copilot settings it will be sent for all requests, and since each model has a different max context length, when this setting is bigger than the max it will fail silently. If that's something people can live with, sure why not.

[user82622]

Is there any progress on this front? This is such an annoying problem for the end user, and it kinda prevents the offline workflow from working properly if the notes are just a bit longer than a page or so.

[logancyang]

@Emt-lin @zeroliu We could add a new field to Custom Model for builtin models in the constant, and add an optional input for the user to set the context length of a model when they add new custom models. After that, there can be other logic like truncation or warning in the UI when input / context is too long before a user sends the request. What do you think?

Ollama-specific logic can be implemented after this change.

[jsynowiec]

I also suggest adding temperature, repeat penalty, and possibly top-p, top-k, frequency penalty, and presence penalty in a similar manner. I often use different models for various tasks, and specific actions frequently require fine-tuning these parameters to maintain control over the outputs. Specific values for these parameters can vary depending on the model.

[ichts]

This is related to future self-host mode.

[pawl]

This is still a significant issue. Running Qwen3.5-9B (Q8) on an RTX 5060 Ti (16GB VRAM), Copilot sends the model's full context length (131072) as num_ctx to Ollama, even though I've set Token Limit to 8100 in Copilot settings and created a custom Modelfile with num_ctx 8192.

This causes Ollama to spill 19% of the model to CPU/RAM, which tanks inference speed. The same model loaded from CLI respects the Modelfile default and keeps 8192 context, 100% GPU, 12GB.

Until Copilot either passes num_ctx in API requests or stops overriding the model's Modelfile default, large-context models on consumer GPUs are a no-go.

# From Copilot:
ollama ps → 16 GB, 19%/81% CPU/GPU, context 131072
# From CLI (same model):
ollama ps → 12 GB, 100% GPU, context 8192