Llama.cpp token generation speed is roughly one third of speed in MLX for Qwen 3 coder next for Apple Silicon

[ManasInd]

Name and Version

Compiled llama.cpp code b7935

Operating systems

Mac

Which llama.cpp modules do you know to be affected?

llama-server

Command line

./llama-server \
    --model ../models/Qwen3-Coder-Next-Q4_K_M.gguf  \
    --alias "unsloth/Qwen3-Coder-Next" \
    --temp 1.0 \
    --top-p 0.95 \
    --min-p 0.01 \
    --top-k 40 \
    --port 8033 \
    --ctx-size 40000 \
    --predict 10000 \
    --gpu-layers all \
    --parallel 1 \
    --cont-batching

Problem description & steps to reproduce

Using llama.cpp, the Qwen3 next coder loads fine and generates accurate text, but the speed of token generation is unusually slow compared to MLX format in Apple Silicon.

For example, in M4 Pro, the token generation speed of Qwen3 Next Q4_K_M unsloth variant is roughly 24 t/s using the latest llama.cpp code. Running the MLX 4bit version of Qwen3 Next has roughly 60 t/s. Existing filled context was negligible as models were just loaded and then asked some random code related questions over chat.

Also tried running the same gguf in LMStudio but the token gen speed was low.

First Bad Commit

No response

Relevant log output

Logs

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.