Eval bug: Qwen 3.5 degraded pp performance on HIP/ROCm

[riveroon]

Name and Version

riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232$ ./llama-cli --version
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
load_backend: loaded ROCm backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-hip.so
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-rpc.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-cpu-haswell.so
version: 8232 (6fce5c6)
built with GNU 11.4.0 for Linux x86_64

Operating systems

Linux, Windows

GGML backends

HIP

Hardware

CPU: Ryzen 5 5700X
GPU: Radeon RX 9070 XT 16GB (gfx1201), ROCm 7.2

Models

Qwen 3.5 Model family (size & quant seems irrelevant)

Problem description & steps to reproduce

Qwen 3.5 models exhibit severe pp performance degradation when run on HIP/ROCm vs. Vulkan, regardless of context depth.
Standard builds for Windows & Linux from https://github.com/ggml-org/llama.cpp/releases , as well as those compiled natively (on Linux, gcc 13.3.0), consistently showed degraded performance.

The benchmark below was performed on an Ubuntu 24.04.4 (Linux 6.17.0) with ROCm 7.2 system with build 8232 from the release page. (Llama 2 and Qwen 3 models were benchmarked for comparison & proof of system integrity)

model	size	params	backend	ngl	fa	test	t/s
llama 7B Q4_0	3.56 GiB	6.74 B	Vulkan	99	1	pp512	4018.63 ± 17.08
llama 7B Q4_0	3.56 GiB	6.74 B	Vulkan	99	1	tg128	127.74 ± 2.60
llama 7B Q4_0	3.56 GiB	6.74 B	ROCm	99	1	pp512	5146.36 ± 20.74
llama 7B Q4_0	3.56 GiB	6.74 B	ROCm	99	1	tg128	93.88 ± 0.09
qwen3 4B Q4_0	2.21 GiB	4.02 B	Vulkan	99	1	pp512	6169.16 ± 9.47
qwen3 4B Q4_0	2.21 GiB	4.02 B	Vulkan	99	1	tg128	171.08 ± 0.22
qwen3 4B Q4_0	2.21 GiB	4.02 B	ROCm	99	1	pp512	7152.43 ± 100.78
qwen3 4B Q4_0	2.21 GiB	4.02 B	ROCm	99	1	tg128	131.16 ± 1.22
qwen35 4B Q4_0	2.40 GiB	4.21 B	Vulkan	99	1	pp512	3760.47 ± 7.70
qwen35 4B Q4_0	2.40 GiB	4.21 B	Vulkan	99	1	tg128	102.54 ± 0.32
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	pp512	50.27 ± 1.73
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	tg128	80.95 ± 0.44
qwen35 2B BF16	3.51 GiB	1.88 B	Vulkan	99	1	pp512	8162.26 ± 51.99
qwen35 2B BF16	3.51 GiB	1.88 B	Vulkan	99	1	tg128	107.37 ± 0.02
qwen35 2B BF16	3.51 GiB	1.88 B	ROCm	99	1	pp512	69.60 ± 0.23
qwen35 2B BF16	3.51 GiB	1.88 B	ROCm	99	1	tg128	86.69 ± 0.51

Additionally, building with flags -DGGML_HIP_GRAPHS=ON partially restores performance back to 4 digits at lower context depth, but falls apart at higher context depth (build 8233):

model	size	params	backend	ngl	fa	test	t/s
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	pp512	3391.79 ± 0.00
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	tg128	103.32 ± 0.00
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	pp512 @ d8192	51.58 ± 0.00
qwen35 4B Q4_0	2.40 GiB	4.21 B	ROCm	99	1	tg128 @ d8192	99.26 ± 0.00

First Bad Commit

No response

Relevant log output

Logs

riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/llama-2-7b.Q4_0.gguf" -ngl 99 -fa 0,1
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
load_backend: loaded ROCm backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-hip.so
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-rpc.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | ROCm       |  99 |  0 |           pp512 |      4751.54 ± 71.00 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | ROCm       |  99 |  0 |           tg128 |         86.15 ± 0.17 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | ROCm       |  99 |  1 |           pp512 |      5146.36 ± 20.74 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | ROCm       |  99 |  1 |           tg128 |         93.88 ± 0.09 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/Qwen3-4B-Q4_0.gguf" -ngl 99 -fa 0,1
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
load_backend: loaded ROCm backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-hip.so
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-rpc.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | ROCm       |  99 |  0 |           pp512 |        533.74 ± 2.29 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | ROCm       |  99 |  0 |           tg128 |        104.92 ± 0.47 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | ROCm       |  99 |  1 |           pp512 |     7152.43 ± 100.78 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | ROCm       |  99 |  1 |           tg128 |        131.16 ± 1.22 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/Qwen3.5-4B-Q4_0.gguf" -ngl 99 -fa 0,1
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
load_backend: loaded ROCm backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-hip.so
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-rpc.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  0 |           pp512 |         48.23 ± 1.27 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  0 |           tg128 |         79.37 ± 0.35 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |           pp512 |         50.27 ± 1.73 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |           tg128 |         80.95 ± 0.44 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232$ ./llama-bench -m ~/Downloads/Qwen3.5-2B-BF16.gguf -ngl 99 -fa 0,1
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
load_backend: loaded ROCm backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-hip.so
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-rpc.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-rocm-7.2-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | ROCm       |  99 |  0 |           pp512 |         68.11 ± 0.22 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | ROCm       |  99 |  0 |           tg128 |         85.63 ± 0.32 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | ROCm       |  99 |  1 |           pp512 |         69.60 ± 0.23 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | ROCm       |  99 |  1 |           tg128 |         86.69 ± 0.51 |

build: 6fce5c6a7 (8232)



riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/Qwen3.5-4B-Q4_0.gguf" -ngl 99 -fa 0,1
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-rpc.so
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 9070 XT (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-vulkan.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | Vulkan     |  99 |  0 |           pp512 |      3735.58 ± 32.89 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | Vulkan     |  99 |  0 |           tg128 |        101.67 ± 0.51 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | Vulkan     |  99 |  1 |           pp512 |       3760.47 ± 7.70 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | Vulkan     |  99 |  1 |           tg128 |        102.54 ± 0.32 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/Qwen3-4B-Q4_0.gguf" -ngl 99 -fa 0,1
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-rpc.so
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 9070 XT (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-vulkan.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | Vulkan     |  99 |  0 |           pp512 |     5922.46 ± 125.76 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | Vulkan     |  99 |  0 |           tg128 |        165.92 ± 0.82 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | Vulkan     |  99 |  1 |           pp512 |       6169.16 ± 9.47 |
| qwen3 4B Q4_0                  |   2.21 GiB |     4.02 B | Vulkan     |  99 |  1 |           tg128 |        171.08 ± 0.22 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232$ ./llama-bench -m "/media/riveroon/347E19C47E198032/llama-2-7b.Q4_0.gguf" -ngl 99 -fa 0,1
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-rpc.so
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 9070 XT (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-vulkan.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | Vulkan     |  99 |  0 |           pp512 |      3835.78 ± 25.51 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | Vulkan     |  99 |  0 |           tg128 |        123.66 ± 0.26 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | Vulkan     |  99 |  1 |           pp512 |      4018.63 ± 17.08 |
| llama 7B Q4_0                  |   3.56 GiB |     6.74 B | Vulkan     |  99 |  1 |           tg128 |        127.74 ± 2.60 |

build: 6fce5c6a7 (8232)
riveroon@riveroon-MS-7B89:~/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232$ ./llama-bench -m ~/Downloads/Qwen3.5-2B-BF16.gguf -ngl 99 -fa 0,1
load_backend: loaded RPC backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-rpc.so
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 9070 XT (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-vulkan.so
load_backend: loaded CPU backend from /home/riveroon/Downloads/llama-b8232-bin-ubuntu-vulkan-x64/llama-b8232/libggml-cpu-haswell.so
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | Vulkan     |  99 |  0 |           pp512 |      8024.57 ± 22.07 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | Vulkan     |  99 |  0 |           tg128 |        106.99 ± 0.12 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | Vulkan     |  99 |  1 |           pp512 |      8162.26 ± 51.99 |
| qwen35 2B BF16                 |   3.51 GiB |     1.88 B | Vulkan     |  99 |  1 |           tg128 |        107.37 ± 0.02 |

build: 6fce5c6a7 (8232)

$ HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
    cmake -S . -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1201 -DGGML_HIP_GRAPHS=ON -DCMAKE_HIP_FLAGS=" --rocm-path=/opt/rocm -mllvm --amdgpu-unroll-threshold-local=600" -DCMAKE_BUILD_TYPE=Release \
    && cmake --build build --config Release -- -j 16
[...]
$ ./llama-bench -m "/media/riveroon/347E19C47E198032/Qwen3.5-4B-Q4_0.gguf" -ngl 99 -fa 1 -d 0,8192 -r 1
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 9070 XT, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model                          |       size |     params | backend    | ngl | fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |           pp512 |       3391.79 ± 0.00 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |           tg128 |        103.32 ± 0.00 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |   pp512 @ d8192 |         51.58 ± 0.00 |
| qwen35 4B Q4_0                 |   2.40 GiB |     4.21 B | ROCm       |  99 |  1 |   tg128 @ d8192 |         99.26 ± 0.00 |

build: c5a778891 (8233)

[InternetOfTofu]

I hit the same bug.

[thejacer]

A comment on Reddit, in the post announcing this PR merge a user commented that ROCm saw a regression: #19504

[am17an]

BTW that PR doesn't even touch the PP path, it is purely for the TG case. Not sure what is happening here

[thejacer]

BTW that PR doesn't even touch the PP path, it is purely for the TG case. Not sure what is happening here

I didn’t see at first that this was exclusively pp. my issue is (20237) is exclusively tg. I’ll move this back to that one.

[InternetOfTofu]

This is how I fixed it: InternetOfTofu@72a76b7

it passes my manual testing with 10x+ speed up for my pp.

[riveroon]

Further investigation:

a) On Windows (HIP), which has ROCm 7.1.1 installed, the degraded performance seems to be a regression which appeared between b8149~b8152.
b8149 (a96a112):

model	size	params	backend	ngl	test	t/s
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	pp512	4009.50 ± 29.73
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	tg128	109.15 ± 0.70

b8152 (d7d826b):

model	size	params	backend	ngl	test	t/s
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	pp512	27.96 ± 0.48
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	tg128	112.40 ± 0.13

Both benchmarks are from the standard (pre-compiled) releases.

b) On the contrary, Linux (ROCm 7.2) seems to suffer from this issue from the start of Qwen 3.5 support:
b7990 (fc0fe40):

model	size	params	backend	ngl	test	t/s
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	pp512	65.75 ± 0.66
qwen35 ?B Q4_0	2.40 GiB	4.21 B	ROCm	99	tg128	74.28 ± 0.28

Since the only thing that catches my eye between b8149 and b8152 seems to be 8fdf269, I'm guessing this is a new upstream (ROCm / rocWMMA 7.2) bug?

[meven3000]

Works fine on vulkan. Issue on ROCM (7.x tested) appears for me after a 2nd prompt then gets works.

start at 120pp +
second at 21.39 pp
third at 27 tps (but too 3-4 minutes of 'processing')

output TPS is stable the whole time. 3x GPU at 100%

will continue investigating.

container.zip

[williamtwomey]

Same issue here

Profiled with rocprof --stats on pp128:

Metric	Value
Total kernel dispatches	14,977
Total GPU compute time	0.039 seconds
Total wall time	~4.1 seconds
Time in dispatch overhead	4.06 seconds (99%)
Overhead per dispatch	~271 µs

The GPU is idle 99% of the time. top shows 100% CPU during prompt processing, suspect the HIP runtime is dispatching kernels one by one.

[ethanmc22]

just wanted to add my experience, a recent update to llama.cpp within the last few days (pretty sure it was the Added GATED_DELTA_NET op), i went from 50 T/s (generation) on qwen3.5 35b down to 35 T/s in the most recent version after that update.

The older version (i reverted back to b8232) is ~40% faster than mainline, so its a huge regression in performance.

And i noticed the regression in performance for all of the qwen 3.5 family and qwen 3 next (which is still extremely slow on ROCm btw, 25T/s full GPU offload)

I am running ROCm backend on two AMD MI50 32Gb's