Tweaking tile geometry for MoE on AMD KHR_coopmat (Vulkan)

[accaldwell]

I've been testing a change to the MUL_MAT_ID Vulkan kernel, adding a non-square tile for AMD hardware with coopmat support. Currently the M tile (64x64) is used for the workloads I was exploring and the original L tile (128x128) is disabled. I confirmed 128x128 performs worse than 64x64 on my hardware, so this seems sensible. However, for some MoE workloads, 128x32 is a decent amount faster than 64x64 - heavy users of MUL_MAT_ID see 7-10% increases in prompt processing speed on Strix Halo:

Model	Before	After	Change	Comment
Qwen3.6-35B-A3B Q6_K	971.44	1045.36	+7.6%
Qwen3.6-35B-A3B Q4_K_M	1111.55	1228.61	+10.5%
Qwen3.6-35B-A3B Q8_0	1159.18	1276.72	+10.1%
Nemotron-Cascade-2-30B-A3B Q4_K_M	1150.78	1262.10	+9.7%
gpt-oss-20b mxfp4	1385.48	1381.11	-0.3%	MoE, doesnt use this new tile
Qwen3.6-27B Q6_K	289.97	289.47	-0.2%	dense model

The math here is unchanged, so perplexity and token generation are identical before and after this change. Token generation speed is unchanged.

Any thoughts on if this is worth continuing to work on? The code was written with LLM assistance, which I understand is frowned upon here, but its also a fairly small change (+40 lines, about half of which are comments). I reviewed the code, did all the benchmarking and other testing, and wrote this post.

My change is currently gated to AMD hardware with KHR_coopmat, if anyone is interested in testing, the code is here: https://github.com/accaldwell/llama.cpp/tree/ac/vulkan-mmid-l-tile-amd