Force gemma4 to reason - #95
Merged
Merged
Conversation
Adds a per-model "Force Reasoning in Gemma4-series models" toggle in the
Reasoning section of the model settings. When enabled and the model runs
on the local llama.cpp engine, the assistant reply is prefilled with an
opening thought channel so Gemma4-series models are forced to reason
before answering.
Frontend:
- New forceGemma4Reasoning field in the advanced model settings schema,
editor controller, and form mapping.
- Toggle is only rendered for the llamacpp provider and is reset when the
provider is switched to anything else, so it can never linger active on
a remote provider.
- i18n keys added and translated across all 20 locales; parameter-support
labels updated.
Backend:
- AdvancedModelSettings gains force_gemma4_reasoning.
- build_provider_extra_fields injects a forceGemma4Reasoning body flag,
gated on provider_id == "llamacpp".
- The llama.cpp desktop path carries the flag through
OpenAICompatPromptOptions and appends the GEMMA4_REASONING_PREFILL
("<|channel>thought") to the built prompt after the generation prompt.
resolve_base_url fell back to https://api.openai.com for any provider not in the config list. The embedded llama.cpp engine is not in that list, so its (never-used) request URL resolved to the OpenAI endpoint, which showed up in the debug view and misleadingly implied traffic was sent to OpenAI. Every real HTTP provider — OpenAI included — has its own entry with an explicit default_base_url, so this fallback only ever fires for unregistered providers. Default it to http://localhost instead, so the debug view reflects that llama.cpp requests stay local.
…Gemma4 reasoning When the forced-reasoning toggle is active on llama.cpp, the system prompt now automatically starts with a "<|think|>\n" opener, so users no longer have to paste it into the system prompt by hand. The opener is prepended into the single leading system message rather than added as a separate one, since Gemma chat templates reject conversations with more than one system message. This complements the existing "<|channel>thought" reply prefill driven by the same toggle.
…d-to-end
The forced-reasoning toggle did not actually work: the forceGemma4Reasoning
body flag was silently dropped, and even once delivered the reasoning block
leaked into the visible message.
- Allowlist the forceGemma4Reasoning extra-body key for llamacpp, so the
flag survives strip_provider_incompatible_extra_fields and reaches the
local engine (previously it was filtered out and never arrived).
- Give the reply prefill a trailing newline ("<|channel>thought\n") so the
model starts at the content position instead of the opener/newline
boundary where it sometimes closed the channel with an empty thought.
- Because the opener is prefilled into the prompt, the model output begins
mid-thought and never emits the open tag. Start the thinking parser
already inside the reasoning block (closing at "<channel|>") for both the
streamed deltas and the final message assembly, so the reasoning goes to
the reasoning field instead of leaking into the saved/visible content.
- Surface the final prompt (including the appended prefill) in the
llama_runtime debug payload, and apply the prefill to the built prompt
before it is logged/tokenized, to make the injected text inspectable.
llamaAdaptiveTarget and llamaAdaptiveDecay were emitted by build_llama_extra_fields but never added to the llamacpp allowlist, so strip_provider_incompatible_extra_fields dropped them from every request and the adaptive-p sampler silently ran without its target/decay controls. Add both keys to the allowlist, and populate them in the coverage test fixture (count 42 -> 44) so the allowlist guard catches a missing entry like this in the future.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a per-model "Force Reasoning in Gemma4-series models" toggle in the Reasoning section of the model settings, only when llama.cpp is the active provider. When enabled and the model runs on the local llama.cpp engine, the assistant reply is prefilled with an opening thought channel so Gemma4-series models are forced to reason before answering.
Also, the toggle removes the necessity to create duplicate systemprompts to force Gemma4-series models to reason.
And the general fallback-URL is set to http://localhost/ to prevent people from heart-attacks when looking at the debug-window.