SD in kcpp (1.121) - how different is image gen backend in kcpp vs. stable diffusion cpp?

[alex-ie]

I'm trying to play with new SD UI that adds LoRAs. I have got some results in Z-Image-Turbo but got black square for Z-Image-Base (even without LoRA). But sd-cli from sd.cpp I have downloaded a week or two ago works fine with ZiB same input.

Is SD engine inside kcpp much different from sd.cpp?

Added:

Comparing the two, I have also noted another factor that hints me toward differences in backend engine: sd-cli is significantly faster than kcpp per one SD step on as far as I know same/similar settings/inputs. But could it be due to different default schedulers / samplers or else? I am new to SD and do not know how those affect.

[LostRuins]

No, it should be almost the same. Maybe the target architecture might be different but functionally all the code should be compatible.

I see you've previously mentioned the issue #1989

I think you tried

1. Running the oldpc.exe build
2. Running with Vulkan

and neither worked for your Pascal device?

Maybe we can try forcing a requantization, try running it with --sqquant 1 and see if that helps? Otherwise, @wbruna did mention leejet/stable-diffusion.cpp#1253 previously too, I dont think anything has changed there

[alex-ie]

Indeed I have faced "this" issue already. Upon reading the links you posted, I have found out only quantized versions produce black squares. And again today I have checked with sd-cli that. But original safetensors BF16 works with sd-cli (downloaded release vulkan build).

But I have just re-run kcpp 1.121 with --usevulkan and --noavx2 (is it same as old build?) and got black square with BF16 too. Hence this discussion is IMO useful.

P.S.
Comparing the two, I have also noted another factor that hints me toward differences in backend engine: sd-cli is significantly faster than kcpp per one SD step on as far as I know same/similar settings/inputs. But could it be due to different default schedulers / samplers or else? I am new to SD and do not know how those affect.

[wbruna]

I have got some results in Z-Image-Turbo but got black square for Z-Image-Base (even without LoRA). But sd-cli from sd.cpp I have downloaded a week or two ago works fine with ZiB same input.

The engine should be practically the same, but we have a bit of mismatch in code versions (the sd.cpp library is lagging a little bit; OTOH, our underlying ggml library is newer), and default parameter handling (e.g. we run the text model on CPU by default). Memory handling is also different from sd-cli, since as a server process we need to keep the model weights loaded between generations.

Could you send a detailed example, including the images (they have embedded metadata that could give us a clue) and loading parameters?

[alex-ie]

Beg your pardon, took a while for me to test and recheck.

In Summary:
kcpp 1.121 vs. sd.cpp d04e895, both run on Vulkan, 5 steps, seed=1, 256x256:

1. ZiB BF16 images attached - square on kcpp only (difference noted only on Ampere GPU of 3 NVIDIA generations tested).
2. On Pascal GPU Q4 ZiT on sd-cli is 2.5x slower than kcpp.
3. On Ampere GPU Q8 ZiT on sd-cli is 1.5x faster than kcpp.

P.S. Square instead of image on ZiB quantized in both kcpp and sd.cpp (IIRC) on Pascal and Ampere (but works on Maxwell).

I do not know what you will make of that. I have looked at exif data (via exiftool) and see nothing of interest. Or do images have some other metadata? Cheers.

[wbruna]

There is a known upstream issue that affects the VAE step when there is too little VRAM available: ggml-org#24943 . Koboldcpp hits it more frequently, due to the differences in memory allocation. Could you check if --sdoffloadcpu (--offload-to-cpu on sd-cli) helps?

Architecture / backend / quantization - specific blank results could be caused by precision issues. Please take a look at:
https://github.com/leejet/stable-diffusion.cpp/blob/master/docs/troubleshooting.md#completely-black-or-white-images-or-videos--nans
and see if the new parameters --linear-scale and --attn-scale (not supported by Koboldcpp yet) have any effect.