Gemma 4 template warning: 26B MoE performance drop and broken reasoning tags (1.117) #2328
perdimanoklboliyezeevich-sketch
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone, I'm getting a recurring warning in the console when running Gemma 4 models (using the built-in Gemma 4 Instruct presets) on the recent KoboldCPP versions: "common_chat_try_specialized_template: detected an outdated gemma4 chat template, applying compatibility workarounds. Consider updating to the official template." While the dense 31B model handles this workaround fine (chugging along stably at a heavy but consistent pace), my experience with the 26B MoE version degraded significantly:
Massive Performance Drop: The generation speed on my hardware (GTX 1660 Super with partial CPU offloading) dropped nearly by half — from a smooth ~17 t/s on older versions down to 8-10 t/s now. It feels like the compatibility workaround breaks the optimized dynamic MoE expert offloading, forcing the backend to push execution entirely to the CPU.
Broken Thinking Blocks: The model frequently fails to hide its reasoning process. The tags and internal English drafts either leak directly into the visible response or the reasoning phase gets completely skipped/disabled by the backend to save context.
It looks like the llama.cpp core under the hood expects the new official Gemma 4 token/channel formatting, but Kobold's built-in UI presets are lagging behind, triggering this fallback mode that hits MoE routing hard.
Could we get the built-in Gemma 4 template configurations updated to match the latest official layout natively?
Thanks for all your hard work on the project!
Beta Was this translation helpful? Give feedback.
All reactions