Name and Version
version: 9573 (f0152ef)
Operating systems
Windows
GGML backends
CUDA
Hardware
2*3060 12G
Models
gemma-4-12B-it-heretic-QAT-UD-Q4_K_XL.gguf
https://huggingface.co/SC117/gemma-4-12B-it-heretic-QAT-GGUF
Problem description & steps to reproduce
@echo off
set CUDA_DEVICE_ORDER=PCI_BUS_ID
start /affinity FFF0 /b /wait F:\ai_system\llama-cpp-cuda\llama-server.exe ^
-m F:\ai_models\gemma4\12B\gemma-4-12B-it-heretic-QAT-UD-Q4_K_XL.gguf ^
--spec-draft-model F:\ai_models\gemma4\12B\MTP\gemma-4-12B-it-qat-assistant-MTP-ONLY_Q4_0.gguf ^
--spec-type draft-mtp ^
--spec-draft-n-max 2 ^
-ngl 99 ^
-ts 50,50 ^
--threads 12 ^
-c 204800 ^
-b 1024 ^
-ub 128 ^
-np 1 ^
-ctk q8_0 ^
-ctv q8_0 ^
-fa on ^
--temp 1 ^
--top-k 64 ^
--top-p 0.95 ^
--min-p 0.00 ^
--presence_penalty 1.0 ^
--repeat-penalty 1.0^
--frequency-penalty 0.05 ^
--xtc-probability 0 ^
--mirostat 0 ^
--samplers "top_k;top_p;min_p;temperature" ^
--reasoning on ^
--seed -1 ^
--host 0.0.0.0 ^
--port 8000 ^
--jinja ^
--chat-template-file F:\ai_models\gemma4\12B\custom_pub_chat_template_gemma4.jinja ^
--ctx-checkpoints 2 ^
--warmup ^
--mlock ^
--no-mmap ^
---fit on ^
-v
But there is no problem when fit off.
First Bad Commit
No response
Relevant log output
0.01.198.364 E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)
0.01.264.271 W srv load_model: [spec] failed to measure draft model memory: failed to create llama_context from model
Name and Version
version: 9573 (f0152ef)
Operating systems
Windows
GGML backends
CUDA
Hardware
2*3060 12G
Models
gemma-4-12B-it-heretic-QAT-UD-Q4_K_XL.gguf
https://huggingface.co/SC117/gemma-4-12B-it-heretic-QAT-GGUF
Problem description & steps to reproduce
@echo off
set CUDA_DEVICE_ORDER=PCI_BUS_ID
start /affinity FFF0 /b /wait F:\ai_system\llama-cpp-cuda\llama-server.exe ^
-m F:\ai_models\gemma4\12B\gemma-4-12B-it-heretic-QAT-UD-Q4_K_XL.gguf ^
--spec-draft-model F:\ai_models\gemma4\12B\MTP\gemma-4-12B-it-qat-assistant-MTP-ONLY_Q4_0.gguf ^
--spec-type draft-mtp ^
--spec-draft-n-max 2 ^
-ngl 99 ^
-ts 50,50 ^
--threads 12 ^
-c 204800 ^
-b 1024 ^
-ub 128 ^
-np 1 ^
-ctk q8_0 ^
-ctv q8_0 ^
-fa on ^
--temp 1 ^
--top-k 64 ^
--top-p 0.95 ^
--min-p 0.00 ^
--presence_penalty 1.0 ^
--repeat-penalty 1.0^
--frequency-penalty 0.05 ^
--xtc-probability 0 ^
--mirostat 0 ^
--samplers "top_k;top_p;min_p;temperature" ^
--reasoning on ^
--seed -1 ^
--host 0.0.0.0 ^
--port 8000 ^
--jinja ^
--chat-template-file F:\ai_models\gemma4\12B\custom_pub_chat_template_gemma4.jinja ^
--ctx-checkpoints 2 ^
--warmup ^
--mlock ^
--no-mmap ^
---fit on ^
-v
But there is no problem when fit off.
First Bad Commit
No response
Relevant log output
0.01.198.364 E llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this is normal during memory fitting)
0.01.264.271 W srv load_model: [spec] failed to measure draft model memory: failed to create llama_context from model