Skip to content

v1.12.1 — issue #3: real fix for thinking-channel empty content

Latest

Choose a tag to compare

@a-radwan-20 a-radwan-20 released this 22 Jun 13:12

The real fix for #3. Thinking-channel models (gpt-oss, glm, minimax-m3, …) now reliably produce optimized prompts instead of intermittently returning empty content. The 1.7.1 mitigation surfaced the failure loudly and degraded gracefully; this makes the models actually work.

The root cause was wrong in the docs

It was never "Ollama's /v1 shim drops the harmony final channel." These models emit a large reasoning trace before the final answer, so a tight max_tokens is spent entirely in the thinking channel and content comes back "". Higher reasoning effort makes it worse; a bigger budget fixes it.

The fix — two levers, applied together (families honor different ones)

  • A max_tokens floor (8192) for reasoning models — the universal lever; the only thing that works for glm, which ignores reasoning_effort entirely. It's a ceiling, not a target, so no added latency.
  • reasoning_effort: "low" — for families that respect it (gpt-oss), also trimming latency/cost. Tune with the new LLM_REASONING_EFFORT env (low|medium|high).

The previously-proposed "switch to Ollama's native /api/chat" was a dead end/api/chat think:false still returns empty for gpt-oss.

Detection is robust, not a hardcoded model list

New model-capabilities.ts characterizes "is this a thinking model?" by (1) Ollama's /api/show capabilities (authoritative + self-updating — detects minimax-m3:cloud with no name match), (2) learning from responses (a reasoning trace or empty-with-tokens → thinking, any provider), (3) a small name hint as last resort, with the empty-content retry as the final name-agnostic backstop.

Verified (all non-empty on the first call)

gpt-oss:20b-cloud 4/4 · glm-5.2:cloud 4/4 · minimax-m3:cloud 3/3 · compose_prompt 3/3 real prompts · test:thinking 7 groups · integration 9/9 on gemma4 · non-reasoning path byte-identical.

Full notes: CHANGELOG.md

🤖 Generated with Claude Code