llama-bench Strix Halo (gfx1151) ROCm 7.2.3 + Qwen3.5 122B A10B MTP UD-Q6_K_XL

[muslimpribadi]

Run llama-bench using unsloth/Qwen3.5-122B-A10B-MTP-GGUF UD-Q6_K_XL.

❌ Currently no llama-bench test parameters for MTP (i.e. llama-server --spec-type draft-mtp --spec-draft-n-max 2).

Header llama-bench output:

ggml_cuda_init: found 1 ROCm devices (Total VRAM: 126976 MiB):
  Device 0: AMD Radeon 8060S Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32, VRAM: 126976 MiB

Footer llama-bench output:

build: ad2775726 (9261)

Text generation Qwen3.5 122B A10B with gemma 4 31B (dense)

llama-bench -ngl 999 -fa 1 -mmp 0 -p 0 -n 128,256,512

🟢 Benchmark result

model	size	params	backend	test	t/s
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	tg128	19.14 ± 0.21
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	tg256	19.21 ± 0.07
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	tg512	19.17 ± 0.04
gemma4 31B Q4_K - Medium	17.46 GiB	30.70 B	ROCm	tg128	10.48 ± 0.01
gemma4 31B Q4_K - Medium	17.46 GiB	30.70 B	ROCm	tg256	10.48 ± 0.00
gemma4 31B Q4_K - Medium	17.46 GiB	30.70 B	ROCm	tg512	10.37 ± 0.01

ℹ️ Token generation averaging at 19 t/s.
default batch-size 2048 ubatch-size 512
Dense model expected slower than MoE.

With MTP flag it can reach 26.86t/s :

Prompt processing with different batch sizes

llama-bench -ngl 999 -fa 1 -mmp 0 -n 0 -p 1024 -b 128,256,512,1024

🟢 Benchmark result

model	size	params	backend	n_batch	test	t/s
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	128	pp1024	137.83 ± 1.52
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	256	pp1024	195.93 ± 1.99
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	512	pp1024	✅266.48 ± 1.81
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	1024	pp1024	258.55 ± 3.35

✅ batch-size 512 with default u-batch-size 512 reach 266.48 t/s
default ubatch-size 512

Different numbers of threads

llama-bench -ngl 999 -fa 1 -mmp 0 -n 0 -n 16 -p 64 -t 1,2,4,8,16,32

🟢 Benchmark result

model	size	params	backend	threads	test	t/s
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	1	pp64	124.46 ± 35.28
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	1	tg16	17.73 ± 2.48
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	2	pp64	145.82 ± 6.29
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	2	tg16	18.75 ± 0.20
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	4	pp64	141.33 ± 2.20
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	4	tg16	18.98 ± 0.08
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	8	pp64	✅148.48 ± 5.75
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	8	tg16	18.94 ± 0.07
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	16	pp64	146.02 ± 3.59
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	16	tg16	18.96 ± 0.08
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	32	pp64	146.40 ± 3.97
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	32	tg16	18.96 ± 0.10

✅ threads 8 148.48 t/s
default batch-size 2048 ubatch-size 512

Different numbers of ubatch and threads

llama-bench -ngl 999 -fa 1 -mmp 0 -n 0 -p 2048 -ub 512,1024 -t 1,2,4,8,16,32

🟢 Benchmark result

model	size	params	backend	threads	n_ubatch	test	t/s
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	1	512	pp2048	252.49 ± 6.22
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	2	512	pp2048	✅258.64 ± 0.62
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	4	512	pp2048	255.27 ± 2.75
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	8	512	pp2048	243.32 ± 2.15
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	16	512	pp2048	232.14 ± 1.11
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	32	512	pp2048	226.60 ± 1.29
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	1	1024	pp2048	✅255.35 ± 0.66
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	2	1024	pp2048	253.47 ± 1.12
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	4	1024	pp2048	252.79 ± 0.93
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	8	1024	pp2048	252.47 ± 0.98
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	16	1024	pp2048	252.86 ± 1.00
qwen35moe 122B.A10B Q6_K	106.04 GiB	124.64 B	ROCm	32	1024	pp2048	251.93 ± 0.79

✅ ubatch-size 512 threads 2 258.64 t/s
✅ ubatch-size 1024 threads 1 255.35 t/s
default batch-size 2048

Environment

$ hostnamectl
    Operating System: Fedora Linux 43 (Server Edition)
              Kernel: Linux 7.0.8-100.fc43.x86_64
        Architecture: x86-64
     Hardware Vendor: GEEKOM
      Hardware Model: A9 Mega
        Hardware SKU: GEEKOM A9 Mega
    Hardware Version: Standard
    Firmware Version: 0.12

llama-server --version
version: 9261 (ad2775726)
built with GNU 15.2.1 for Linux x86_64

Container: podman image docker pull kyuz0/amd-strix-halo-toolboxes:rocm-7.2.3 ROCM 7.2.3

see Strix Halo Toolboxes