Qwen3.8-27B and DGX Spark ( GB10 )

[phr0gz]

Hello,

I’m using the latest available llama.cpp Docker image and trying to run the latest Qwen3.8 model with MTP. I’m seeing an average generation speed of approximately 15 tokens/s.

I also checked the NVIDIA forum for DGX Spark / GB10, where users reported similar performance with vLLM. That seemed consistent. Until I saw an official SGLang post reporting 38 tokens/s decode throughput on DGX Spark ( https://x.com/sgl_project/status/2088281320422322413 ).

Am I missing a configuration or optimization that could explain this difference? Or is this model/platform not yet fully optimized in llama.cpp?

For reference, here is my Docker command. It is essentially the same configuration I use for Qwen3.5 122B:

docker run -d --name llama-cpp-qwen38-27b-MTP \
	--gpus all -p 8000:8080 \
	-v /config/llamacpp/models:/models \
	-e LLAMA_CACHE=/models \
	ghcr.io/ggml-org/llama.cpp:full-cuda -s  -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL \
		--host 0.0.0.0 --port 8080 \
		--no-mmap \
		-ngl 99 \
		-fa on \
		-t 8 \
		--cache-type-k q8_0 --cache-type-v q8_0 \
		-b 4096 -ub 2048 \
		-np 1 \
		--spec-type draft-mtp \
		--spec-draft-n-max 2

[ggerganov]

Try this on the DGX Spark:

llama serve \
 -hf  ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
 -hfd ggml-org/Qwen3.8-27B-GGUF:Q4_0 \
 --spec-default \
 --spec-type draft-mtp \
 --reasoning-preserve

[phr0gz]

Thanks for the feedback, I ran another with this:

docker run -d --name llama-cpp-qwen38-27b-test \
	--gpus all -p 8000:8080 \
	-v /config/llamacpp/models:/models \
	-e LLAMA_CACHE=/models \
	ghcr.io/ggml-org/llama.cpp:full-cuda -s \
		-hf  ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
		-hfd ggml-org/Qwen3.8-27B-GGUF:Q4_0 \
		--host 0.0.0.0 --port 8080 \
		--spec-default \
		--spec-type draft-mtp \
		--reasoning-preserve

I have a small increase:

22.12.854.508 I slot print_timing: id  2 | task 2 | n_gen =  22141, tg =  18.07 t/s, tg_3s =  14.02 t/s
22.15.873.882 I slot print_timing: id  2 | task 2 | n_gen =  22202, tg =  18.07 t/s, tg_3s =  20.20 t/s
22.18.998.030 I slot print_timing: id  2 | task 2 | n_gen =  22270, tg =  18.08 t/s, tg_3s =  21.77 t/s
22.22.131.568 I slot print_timing: id  2 | task 2 | n_gen =  22338, tg =  18.09 t/s, tg_3s =  21.70 t/s
22.24.806.445 I slot print_timing: id  2 | task 2 | prompt eval time =   12753.97 ms /  6231 tokens (    2.05 ms per token,   488.55 tokens per second)
22.24.806.448 I slot print_timing: id  2 | task 2 |        eval time = 1237284.46 ms / 22373 tokens (   55.31 ms per token,    18.08 tokens per second)
22.24.806.448 I slot print_timing: id  2 | task 2 |       total time = 1250038.43 ms / 28604 tokens
22.24.806.450 I slot print_timing: id  2 | task 2 |    graphs reused =       9239
22.24.806.462 I slot print_timing: id  2 | task 2 | draft acceptance = 0.45567 (13020 accepted / 28573 generated), mean len =  2.39
22.24.806.799 I slot      release: id  2 | task 2 | stop processing: n_tokens = 28605, truncated = 0