Deepseek v4 Flash - Vulkan Performance Baseline?

[d-shehu]

Is anyone else seeing significantly slower performance with Deepseek v4 Flash and Vulkan based as compared to comparable systems posted in various forums. I'm seeing roughly 50% of the PP and TG I'd expect with either UD IQ4 or UD Q8.

All other models are performing as expected so I don't believe it's a problem with my drivers/configurations.

Hardware:
AM4 5950x
128 GB DDR4 3200
dual R9700
Ubuntu 24.04 Server (Minimized/Headless)

If someone is running either model with consumer grade hardware like a dual-channel AM4 or AM5 board with say 128GB of RAM either 24 - 64GB of VRAM and getting ~10-20 t/s using Vulkan would you kindly post your configs? Thanks in advance.

---

Addendums:

1. Upgrading llama.cpp from b10241 to b10269 reduced TG by 25%
2. Replacing Vulkan with ROCM improved TG by 25%
3. dspark reduces by performance by 30% - 50% depending
4. llama-cli crashes depending how up, down and gate layers offloaded.
   • Specific to DS4.

"Optimal" Tensor Offload

UD8 XL performance improved by offloading MOE by 50% by tweaking the offloads. But overall numbers are much worse than expected.

Key Params

-ngl all                 
--jinja                 
-sm layer                 
--fit off                 
 -fa on                 
 -ub 512 -b 2048 
 --cache-type-k q8_0 
 --cache-type-v q8_0
 --no-mmap                         
 --no-warmup

OT

blk\.([3-9]|1[0-1])\.ffn_(up|down)_exps=RPC1[192.168.5.2:26001],\
blk\.(1[6-9]|2[0-7])\.ffn_(up|down)_exps=RPC1[192.168.5.2:26001],\
blk\.(3[2-9]|4[0-3])\.ffn_(up|down)_exps=RPC1[192.168.5.2:26001]

TG Performance

Context: 2K - 64K
ROCm:  ~10-12 t/s 
Vulkan:  ~5-6 t/s