Ollama truncating input prompt context size

[joelam33-coder]

ollama version is 0.12.5
Ragflow 0.20.5

In RagFlow Model Provider: For Ollama gemma3:4b Max Token set to 8192
In Chat Setting Max Token set to 8192
When I run Ollam PS, it shows the default size 4096. It seems like Ragflow is not asking or calculating the context size and pass it to ollama before running it.

Due to truncated text, it causes the model to hallucinate because it is cutting off the instruction.

The question I have is that, is the Max Token configuration in the RagFlow Model Provider passing the context size to ollama?

Error I received:
time=2025-10-11T15:34:53.835Z level=WARN source=runner.go:159 msg="truncating input prompt" limit=4096 prompt=4339 keep=4 new=4096

People with similar problem:
ollama/ollama#10368

my temporary solution. I have to create a model derived from gemma3:4b and set it to 8192 in Ollama which I do not prefer.

[dosubot[bot]]

Hi @joelam33-coder! I'm Dosu and I’m helping the ragflow team.

RagFlow's Max Token configuration does not reliably pass the context size to Ollama. Even if you set 8192 in RagFlow, Ollama will use its default (4096) unless the underlying model in Ollama is configured for a higher context window. This is why you see prompt truncation and hallucination issues. The workaround—creating a derived model in Ollama with the desired context size—is necessary because RagFlow's settings alone do not update Ollama's context window. This gap is a known issue and may be addressed in future updates, but for now, changing RagFlow's settings is not enough without also updating the Ollama model itself. More details are discussed here and here.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[Ganesh-403]

@joelam33-coder Based on the behavior you're seeing, I don't think the Max Token setting in RAGFlow's Model Provider is intended to control Ollama's context window (num_ctx).

The warning from Ollama:

truncating input prompt
limit=4096
prompt=4339

indicates that the request reached Ollama with a context window of 4096, so Ollama truncated the prompt before inference. Ollama uses num_ctx (or the server/model default) to determine the context window, not the maximum output tokens. (docs.ollama.com :contentReference[oaicite:0]{index=0})

So your observation makes sense:

• Setting Max Token = 8192 in RAGFlow does not necessarily mean RAGFlow is sending options.num_ctx = 8192 to Ollama.
• If ollama ps still shows a context of 4096, then Ollama is likely using its default/model configuration rather than receiving a larger num_ctx from the client.

Your workaround of creating a derived model with:

PARAMETER num_ctx 8192

is a valid solution because it changes the model's default context window. Alternatively, Ollama also supports configuring the context length via the API (options.num_ctx) or by setting the OLLAMA_CONTEXT_LENGTH environment variable for the server. (docs.ollama.com :contentReference[oaicite:1]{index=1})

One question that would help confirm whether this is a RAGFlow limitation:

• If you enable debug logging or capture the request that RAGFlow sends to Ollama, do you see an options.num_ctx field in the JSON payload?

If that field is missing, then it would strongly suggest that the current Ollama integration isn't forwarding the configured context size, and your workaround (or setting OLLAMA_CONTEXT_LENGTH) is the correct approach until support is added.

If this solves your problem, feel free to mark it as the accepted answer so others can find it easily.

[cyberjoe-ai]

Thank you. I used the workaround method.

[Ganesh-403]

@cyberjoe-ai welcome! please mark answer as accepted to close this discussion