kompute : llama-bench support and ggml_cpu_has_kompute()

[cebtenzzre]

I didn't realize that the Kompute backend should have been added in these places.

[slaren]

It doesn't really need to be added to ggml.c, eventually all of the backend code will be removed from there, and the llama.cpp change does nothing since the CPU backend is no longer running at the same time as the GPU backends, it's just a leftover from the pre-ggml-backend implementation that I forget to remove. Anyway it doesn't really matter, the changes to llama-bench and common are good.

[cebtenzzre]

@slaren I removed that code from llama.cpp, does that seem right?

[stduhpf]

Somehow this PR huts performance for the other Vulkan backend? This doesn't make sense when I look at the changes, but the difference with the previous commit on master is very significant. (It's not just with llama-bench.)

Before merge:

ggml_vulkan: Using AMD Radeon RX 5700 XT | uma: 0 | fp16: 1 | warp size: 64

model	size	params	backend	ngl	test	t/s
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	Vulkan	33	pp 512	225.55 ± 1.52
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	Vulkan	33	tg 128	43.33 ± 0.35

build: e0085fd (2026)

After merge:

ggml_vulkan: Using AMD Radeon RX 5700 XT | uma: 0 | fp16: 1 | warp size: 64

model	size	params	backend	ngl	test	t/s
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	Vulkan	33	pp 512	197.63 ± 16.47
llama 7B Q4_K - Medium	4.07 GiB	7.24 B	Vulkan	33	tg 128	21.54 ± 2.20

build: e8dc55d (2027)

EDIT: reverting 3536cf6 fixes it.

[slaren]

The only way I can imagine this could make a difference is there is a large overhead for launching the extra threads for the get_rows operation that still runs on the CPU. Are you on Windows?

[stduhpf]

I observe a simillar drop of performance with the Kompute backend

Lastest master

model	size	params	backend	ngl	test	t/s
llama 7B Q4_0	3.56 GiB	6.74 B	Kompute	33	pp 512	68.04 ± 1.70
llama 7B Q4_0	3.56 GiB	6.74 B	Kompute	33	tg 128	15.42 ± 2.18

With 3536cf6 reverted

model	size	params	backend	ngl	test	t/s
llama 7B Q4_0	3.56 GiB	6.74 B	Kompute	33	pp 512	71.44 ± 0.25
llama 7B Q4_0	3.56 GiB	6.74 B	Kompute	33	tg 128	20.79 ± 2.75

[stduhpf]

The only way I can imagine this could make a difference is there is a large overhead for launching the extra threads for the get_rows operation that still runs on the CPU. Are you on Windows?

Yes, on Windows 10

[stduhpf]

Setting -t 1 manually works too.