[Bug] Lens doesn't work on Vulkan

[wbruna]

Git commit

29ab511 (master-655-29ab511)

Operating System & Version

Debian 13, Mesa 25.2.6

GGML backends

Vulkan

Command-line arguments used

sd-cli --backend Vulkan1 --diffusion-model lens_turbo-Q8_0.gguf --llm gpt-oss-20b-mxfp4.gguf --tae taef2.safetensors -p flower --cfg-scale 1 --steps 4

Steps to reproduce

On Vulkan, the bf16 safetensors file just produce NaNs:

With a direct q8_0 quant:

The following change helps with the q8_0:

diff --git a/src/lens.hpp b/src/lens.hpp
index b5ff068..1cadd29 100644
--- a/src/lens.hpp
+++ b/src/lens.hpp
@@ -37,6 +37,10 @@ namespace Lens {
             auto w2 = std::dynamic_pointer_cast<Linear>(blocks["w2"]);
             auto w3 = std::dynamic_pointer_cast<Linear>(blocks["w3"]);
 
+            if (sd_backend_is(ctx->backend, "Vulkan")) {
+                w2->set_scale(1.f / 128.f);
+            }
+
             auto gate = ggml_silu(ctx->ggml_ctx, w1->forward(ctx, x));
             auto up   = w3->forward(ctx, x);
             x         = ggml_mul(ctx->ggml_ctx, gate, up);

I've tested all scales from 1/16 to 1/512, but quality doesn't seem to improve. It also doesn't help with the bf16.

What you expected to happen

The same gen with --backend ROCm,clip=Vulkan:

What actually happened

Bad images; low quality even with the scaling.

Logs / error messages / stack trace

No response

Additional context / environment details

Device is:

ggml_vulkan: 1 = AMD Radeon RX 7600 XT (RADV NAVI33) (radv) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat

[ZahyGabi]

I have the same problem with CPU-generation, and didn't help my naive modification of the above patch (s / Vulkan / CPU /