Summary
When using a local Ollama instance with Hermes Agent, the model runs with Ollama's default num_ctx (4096 tokens) even though the GGUF metadata reports a much larger context (e.g. 131,072 for Gemma 4). Hermes reads the GGUF value and stores it in _ollama_num_ctx, but never actually sends it to Ollama's /v1/chat/completions endpoint — because the OpenAI-compat route ignores the extra_body.options.num_ctx field that the custom provider profile injects.
The result: once the conversation grows (many turns + tool definitions), the input token count exceeds 4096 and every response returns finish_reason="length" with empty or truncated content — even when the Hermes UI shows 0 / 131.1K context usage. The truncation-retry loop then concatenates three broken partial responses into garbled output.
Summary
When using a local Ollama instance with Hermes Agent, the model runs with Ollama's default
num_ctx(4096 tokens) even though the GGUF metadata reports a much larger context (e.g. 131,072 for Gemma 4). Hermes reads the GGUF value and stores it in_ollama_num_ctx, but never actually sends it to Ollama's/v1/chat/completionsendpoint — because the OpenAI-compat route ignores theextra_body.options.num_ctxfield that thecustomprovider profile injects.The result: once the conversation grows (many turns + tool definitions), the input token count exceeds 4096 and every response returns
finish_reason="length"with empty or truncated content — even when the Hermes UI shows0 / 131.1Kcontext usage. The truncation-retry loop then concatenates three broken partial responses into garbled output.