Skip to content

Gemma 4 Thinking permanently disabled until model reload #2125

Description

@MasterMoritz

Expected Behavior
Using Text Completion API in instruct mode, I prepend the system prompt with <|think|> to enable reasoning. I want to temporarily disable it for some testing by removing <|think|> from the system prompt and then re-enable it afterwards. I would expect it to start reasoning again after reintroducing <|think|>.

Issue
What actually happens is that reasoning stops working forever after removing the <|think|> and sending a message. Reintroducing <|think|> to the system prompt does nothing at all. After restarting koboldcpp, thinking works again as expected.

Steps to reproduce

  1. add <|think|>
  2. send message (response with reasoning)
  3. disable <|think|>
  4. send message (response without reasoning)
  5. add <|think|>
  6. send message (response without reasoning)
  7. restart koboldcpp
  8. send message (response with reasoning)

Further Observed Behavior

  1. If I start without <|think|> and then add it afterwards, it actually starts to think. But once it has thought, turning it off and on leads to above described issue.
  2. Injecting <|think|> into a separate System message right after user turn interestingly works every time (but does not fix thinking capability through the system prompt)

Model config
Probably makes no difference but I use bartowski/google_gemma-4-26B-A4B-it-GGUF (Q4_K)
KoboldCpp v1.111.2
swa enabled, fastforwarding enabled
{ "threads": 5, "usecuda": null, "usevulkan": [ 0 ], "usecpu": false, "contextsize": 20480, "gpulayers": -1, "tensor_split": null, "autofit": false, "maingpu": -1, "batchsize": 512, "blasthreads": null, "lora": null, "loramult": 1.0, "noshift": true, "nofastforward": false, "useswa": true, "smartcache": 0, "ropeconfig": [ 0.0, 10000.0 ], "overridenativecontext": 0, "usemmap": false, "usemlock": false, "noavx2": false, "failsafe": false, "debugmode": 0, "onready": "", "benchmark": null, "prompt": "", "cli": false, "genlimit": 0, "multiuser": 1, "multiplayer": false, "websearch": false, "remotetunnel": false, "highpriority": false, "foreground": false, "preloadstory": null, "savedatafile": null, "quiet": false, "ssl": null, "nocertify": false, "mmproj": null, "mmprojcpu": false, "visionmaxres": 1024, "draftmodel": null, "draftamount": 8, "draftgpulayers": 999, "draftgpusplit": null, "password": null, "ratelimit": 0, "ignoremissing": false, "chatcompletionsadapter": "AutoGuess", "jinja": false, "jinja_tools": false, "jinja_kwargs": "", "noflashattention": false, "lowvram": false, "quantkv": 0, "smartcontext": false, "nomodel": false, "moeexperts": -1, "moecpu": 0, "defaultgenamt": 1024, "nobostoken": false, "enableguidance": false, "maxrequestsize": 32, "overridekv": null, "overridetensors": null, "showgui": false, "skiplauncher": false, "singleinstance": false, "nopipelineparallel": true, "gendefaults": {}, "gendefaultsoverwrite": false, "mcpfile": null, "device": "", "downloaddir": "", "autofitpadding": 1024, "hordemodelname": "", "hordeworkername": "", "hordekey": "", "hordemaxctx": 0, "hordegenlen": 0, "sdmodel": "", "sdthreads": 5, "sdclamped": 0, "sdclampedsoft": 0, "sdt5xxl": "", "sdclip1": "", "sdclip2": "", "sdphotomaker": "", "sdupscaler": "", "sdflashattention": false, "sdoffloadcpu": false, "sdvaecpu": false, "sdclipgpu": false, "sdconvdirect": "off", "sdvae": "", "sdvaeauto": false, "sdquant": 0, "sdlora": [], "sdloramult": [ 1.0 ], "sdtiledvae": 768, "sdmaingpu": -1, "whispermodel": "", "ttsmodel": "", "ttswavtokenizer": "", "ttsgpu": false, "ttsmaxlen": 4096, "ttsthreads": 0, "ttsdir": "", "musicllm": "", "musicembeddings": "", "musicdiffusion": "", "musicvae": "", "musiclowvram": false, "embeddingsmodel": "", "embeddingsmaxctx": 0, "embeddingsgpu": false, "admin": false, "adminpassword": "", "admindir": "", "adminunloadtimeout": 0, "routermode": false, "autoswapmode": false, "hordeconfig": null, "sdconfig": null, "noblas": false, "nommap": false, "pipelineparallel": false, "sdnotile": false, "forceversion": false, "sdgendefaults": false, "flashattention": false, "istemplate": false }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions