Repository navigation
MPT Qwen 3.8 27b Error #28199
Unanswered
brxperience
asked this question in
Q&A
Replies: 2 comments
|
Resolved in b10751 |
0 replies
This comment was marked as low quality.
This comment was marked as low quality.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
In recent versions of llama.cpp, Qwen's MTP has stopped loading and is throwing an error. Can anyone help me?
C:\llama\llama-server.exe --version
version: 0.3.0-dev (build 10745, commit c845263)
built with MSVC 19.44.35227.0 for Windows AMD64
C:\llama\llama-server.exe -m D:\Modelos\Qwen3.8-27B-UD-Q4_K_XL.gguf -c 133120 -ngl 99 -t 8 -np 1 -b 3072 -ub 512 --temp 0.9 --top-p 0.95 --top-k 20 --min-p 0.01 --repeat-penalty 1.0 --reasoning on --cache-type-v q8_0 --flash-attn on --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0.75 --spec-draft-p-split 0.10 --host 0.0.0.0 --port 8083 --cont-batching --metrics --reasoning-preserve --image-min-tokens 1024 --fit off --spec-draft-ngl 99 --reasoning-effort xhigh
0.00.021.769 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the
-lv NCLI arg)0.00.035.498 I srv load_model: loading model 'D:\Modelos\Qwen3.8-27B-UD-Q4_K_XL.gguf'
0.06.409.026 I cmn init: llama threadpool init, n_threads = 8
0.06.490.169 I common_speculative_init_result: creating MTP draft context against the target model 'D:\Modelos\Qwen3.8-27B-UD-Q4_K_XL.gguf'
0.06.492.013 E llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache
0.06.492.018 E common_speculative_init_result: failed to create MTP context
0.06.492.023 E srv load_model: failed to create MTP context
0.06.492.031 I srv operator (): operator (): cleaning up before exit...
0.06.492.678 E srv llama_server: exiting due to model loading error
All reactions