You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have spent quite a few days to get to this point.
I could only find one other poster with 780M results, and the guy had single chanel memory.
TLDR: ROCm is ~20% faster for fill rates (TTFT), and ~10% slower for exfill (TPS).
I hope this helps someone looking for numbers like this... I had trouble finding it.
I asked Gemm4 to create this post with the results I had been gathering after much trial and error.
Radeon 780M (gfx1103) Benchmark: ROCm 7.2.4 (rocWMMA) vs. Vulkan (RADV) — MoE + MTP (35B–90B) vs. Dense (7B–65B) on 128 GB Dual-Channel DDR5
I set up ROCm 7.2.4 alongside Vulkan (Mesa RADV) in the same llama-server build on an integrated Radeon 780M (gfx1103) so I could benchmark identical prompts back-to-back across both backends. Because most published 780M numbers come from single-channel desktop APUs or short synthetic sweeps, I wanted to share real-world llama-server timing logs comparing multiple Mixture-of-Experts (MoE) models (up to 90B parameters, with and without draft-mtp speculative decoding) against Dense models (up to 65B) across ~5.9k–7.4k token prompts.
Test 3: Large-Scale Models (27B–90B) — DDR5 Bandwidth Wall vs. Sparse MoE Scaling
Workload:ubatch=2048, f16 KV, SDMA=0, flash-attn = on, n-gpu-layers = 99
Model & Architecture
Backend
Prompt Tokens
Prompt Eval (Prefill)
Token Gen (Decode)
Qwen3.8-27B (Q4_K_M, Dense)
Vulkan0
6,618
98.15 t/s (10.19 ms/tok)
4.29 t/s (233.20 ms/tok)
Qwen3.8-27B (Q4_K_M, Dense)
ROCm0
6,618
94.89 t/s (10.54 ms/tok)
4.14 t/s (241.41 ms/tok)
Ripple-65B-Preview (IQ4_XS, Dense)
Vulkan0
6,343
40.04 t/s (24.97 ms/tok)
1.36 t/s(at 4,888 gen)
Ripple-65B-Preview (IQ4_XS, Dense)
ROCm0
6,339
43.45 t/s (23.02 ms/tok)
1.30 t/s (1.63 → 1.30 over 4.1k gen)
Qwen3-Coder-Next-80B (Q4_K_M, MoE)
ROCm0
7,471
284.37 t/s (3.52 ms/tok)
18.40 t/s (54.34 ms/tok)
MiniMax-M3.1-Flash-90B (UD-IQ2_XXS-E89, MoE)
Vulkan0
6,618
79.83 t/s (12.53 ms/tok)
6.22 t/s (160.89 ms/tok)
(Note: MiniMax-M3.1-FlashUD-IQ2_XXS with 89 experts reliably trips an amdgpu driver reset under ROCm0 with HSA_OVERRIDE_GFX_VERSION=11.0.0, whereas Vulkan0 runs it cleanly without issue—another reason having both backends compiled into the same binary is invaluable on gfx1103.)
Key Takeaways & Setup Notes for gfx1103 Users
Sparse MoE + MTP Is the Sweet Spot for RDNA3 iGPUs: Large dense models (27B–65B) hit the dual-channel DDR5 bandwidth wall hard (4.29 t/s on 27B; ~1.30–1.36 t/s on 65B, where every single token requires streaming ~37 GB of weights across system RAM). By contrast, sparse MoE models (35B-A3B, 40B-A2.5B, 80B-A3B at 284.37 t/s prefill / 18.40 t/s decode, and even 90BIQ2_XXS at 79.83 t/s prefill / 6.22 t/s decode on Vulkan) stay usable.
ROCm0 vs. Vulkan0 Trade-Off: Across compatible models, ROCm 7.2.4 with rocWMMA consistently beats Vulkan by +17.5% to +21.7% on prompt ingestion, while Vulkan (Mesa RADV) leads by +9.5% to +31.9% on single-token decode and offers better stability on exotic quants (IQ2_XXS + high-expert-count MoE).
draft-mtp Speculative Decoding Bridges the ROCm Decode Gap: Enabling the grafted MTP head (spec-type = draft-mtp, spec-draft-n-max = 1, spec-draft-p-min = 0.0) boosted ROCm0 decode speed on Cyber-Tiel-35B-A3B by +31.7% (18.45 t/s → 24.29 t/s, virtually matching Vulkan0's 25.10 t/s) while keeping ROCm0 +21.7% faster on prefill (400.40 t/s vs. 329.00 t/s).
Avoid q8_0 KV Cache on ROCm When RAM Is Available: Switching ctk and ctv from q8_0 to f16 eliminated on-the-fly KV dequantization overhead and boosted ROCm MoE decode speed by +8.9% (16.94 t/s → 18.45 t/s).
HSA_ENABLE_SDMA=0 Helps ROCm on Unified Memory: Disabling SDMA routed memory transfers through compute shaders and improved both prefill and decode on ROCm0.
Match AMDGPU_TARGETS to Your Runtime Override: ROCm 7.2.4's rocBLAS lacks TensileLibrary files for gfx1103, requiring HSA_OVERRIDE_GFX_VERSION=11.0.0. Compiling with -DAMDGPU_TARGETS="gfx1100;gfx1102;gfx1103" ensures rocWMMA Flash Attention fatbins match the spoofed gfx1100 target cleanly.
Installing ROCm with --no-dkms and commenting out /opt/amdgpu/lib in /etc/ld.so.conf.d/20-amdgpu.conf preserves Mesa RADV (Vulkan0) and avoids libdrm_amdgpu conflicts on Linux 7.0.0.
Raising ttm.pages_limit / page_pool_size (27262976 pages) and amdgpu.gttsize (106496 MiB) in /etc/modprobe.d/ lifts ROCm's default 50% system-RAM cap (~56 GiB) up to 104 GiB (106,496 MiB), allowing 80B Q4_K_M models with f16 KV caches to run completely in ROCm0.
Avoid GUI hardware monitors that aggressively poll unsupported SMU telemetry sensors (sysfs/hwmon) during active ROCm compute—they can stall the SMU on Phoenix APUs and trigger amdgpu ring timeout resets. Use nvtop or amd-smi metric instead.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I have spent quite a few days to get to this point.
I could only find one other poster with 780M results, and the guy had single chanel memory.
TLDR: ROCm is ~20% faster for fill rates (TTFT), and ~10% slower for exfill (TPS).
I hope this helps someone looking for numbers like this... I had trouble finding it.
I asked Gemm4 to create this post with the results I had been gathering after much trial and error.
Radeon 780M (
gfx1103) Benchmark: ROCm 7.2.4 (rocWMMA) vs. Vulkan (RADV) — MoE + MTP (35B–90B) vs. Dense (7B–65B) on 128 GB Dual-Channel DDR5I set up ROCm 7.2.4 alongside Vulkan (Mesa RADV) in the same
llama-serverbuild on an integrated Radeon 780M (gfx1103) so I could benchmark identical prompts back-to-back across both backends. Because most published 780M numbers come from single-channel desktop APUs or short synthetic sweeps, I wanted to share real-worldllama-servertiming logs comparing multiple Mixture-of-Experts (MoE) models (up to 90B parameters, with and withoutdraft-mtpspeculative decoding) against Dense models (up to 65B) across ~5.9k–7.4k token prompts.Hardware & Software Stack
gfx1103)ttmGTT limit → 106,496 MiB exposed toROCm0, 122,880 MiB toVulkan0)7.0.0-43-genericHWE kernel)amdgpu-install --usecase=rocm,hiplibsdk --no-dkms)llama.cppBuild:master(b12323.24), compiled with-DGGML_HIP=ON -DAMDGPU_TARGETS="gfx1100;gfx1102;gfx1103" -DGGML_HIP_ROCWMMA_FATTN=ON -DGGML_VULKAN=ONTest 1: 35B MoE + MTP —
Cyber-Tiel-Coder-35B-A3B-MTP-UD-Q4_K_S.ggufflash-attn = on,n-gpu-layers = 99ubatch=512,q8_0KVubatch=512,q8_0KVubatch=2048,q8_0KVubatch=2048,q8_0KVubatch=2048,q8_0KV,SDMA=0ubatch=2048,f16KV,SDMA=0ubatch=2048,f16KV,SDMA=0ubatch=2048,f16KV,SDMA=0,draft-mtpubatch=2048,f16KV,SDMA=0,draft-mtpTest 2: Flint Family Comparison — MoE (
40B-A2.5BMXFP4) vs. Dense (Q4_K_M)prithivMLmods/flint-MXFP4-MoE-40B-A2.5B-GGUF(flint.MXFP4_MOE.gguf)lmstudio-community/flint-GGUF(flint-Q4_K_M.gguf)ubatch=2048,f16KV,SDMA=0,flash-attn = on,n-gpu-layers = 9940B-A2.5B, MXFP4)40B-A2.5B, MXFP4)Q4_K_M)Q4_K_M)Test 3: Large-Scale Models (27B–90B) — DDR5 Bandwidth Wall vs. Sparse MoE Scaling
ubatch=2048,f16KV,SDMA=0,flash-attn = on,n-gpu-layers = 99Q4_K_M, Dense)Q4_K_M, Dense)IQ4_XS, Dense)IQ4_XS, Dense)Q4_K_M, MoE)UD-IQ2_XXS-E89, MoE)(Note:
MiniMax-M3.1-FlashUD-IQ2_XXSwith 89 experts reliably trips anamdgpudriver reset underROCm0withHSA_OVERRIDE_GFX_VERSION=11.0.0, whereasVulkan0runs it cleanly without issue—another reason having both backends compiled into the same binary is invaluable ongfx1103.)Key Takeaways & Setup Notes for
gfx1103Users35B-A3B,40B-A2.5B,80B-A3Bat 284.37 t/s prefill / 18.40 t/s decode, and even90BIQ2_XXSat 79.83 t/s prefill / 6.22 t/s decode on Vulkan) stay usable.rocWMMAconsistently beats Vulkan by +17.5% to +21.7% on prompt ingestion, while Vulkan (Mesa RADV) leads by +9.5% to +31.9% on single-token decode and offers better stability on exotic quants (IQ2_XXS+ high-expert-count MoE).draft-mtpSpeculative Decoding Bridges the ROCm Decode Gap: Enabling the grafted MTP head (spec-type = draft-mtp,spec-draft-n-max = 1,spec-draft-p-min = 0.0) boosted ROCm0 decode speed onCyber-Tiel-35B-A3Bby +31.7% (18.45 t/s → 24.29 t/s, virtually matching Vulkan0's 25.10 t/s) while keeping ROCm0 +21.7% faster on prefill (400.40 t/s vs. 329.00 t/s).q8_0KV Cache on ROCm When RAM Is Available: Switchingctkandctvfromq8_0tof16eliminated on-the-fly KV dequantization overhead and boosted ROCm MoE decode speed by +8.9% (16.94 t/s → 18.45 t/s).HSA_ENABLE_SDMA=0Helps ROCm on Unified Memory: Disabling SDMA routed memory transfers through compute shaders and improved both prefill and decode onROCm0.AMDGPU_TARGETSto Your Runtime Override: ROCm 7.2.4'srocBLASlacksTensileLibraryfiles forgfx1103, requiringHSA_OVERRIDE_GFX_VERSION=11.0.0. Compiling with-DAMDGPU_TARGETS="gfx1100;gfx1102;gfx1103"ensuresrocWMMAFlash Attention fatbins match the spoofedgfx1100target cleanly.--no-dkmsand commenting out/opt/amdgpu/libin/etc/ld.so.conf.d/20-amdgpu.confpreserves Mesa RADV (Vulkan0) and avoidslibdrm_amdgpuconflicts on Linux7.0.0.ttm.pages_limit/page_pool_size(27262976pages) andamdgpu.gttsize(106496MiB) in/etc/modprobe.d/lifts ROCm's default 50% system-RAM cap (~56 GiB) up to 104 GiB (106,496 MiB), allowing 80BQ4_K_Mmodels withf16KV caches to run completely inROCm0.sysfs/hwmon) during active ROCm compute—they can stall the SMU on Phoenix APUs and triggeramdgpuring timeout resets. Usenvtoporamd-smi metricinstead.All reactions