Non deterministic behaviour of deterministic samplers with IPEX

[mstuermer]

Hello,

I am running this fork under Gentoo Linux, kernel version 6.2.14, with Intel ARC 770 GPU and Intel ONE API successfully enabled. I am launching the script with "./webui.sh --use-ipex", Torch is detecting the GPU correctly and using it. Iteration speed is around 6 it/s. CPU is AMD Ryzen 7 3800X, 32 GB RAM, BAR is enabled, Python 3.10 is being used. No major errors/warnings on launch of the fork are given, so far I am using the default settings (aside theme). Image creation works, yeeaaaah!

So far, so good. Now to my question: I am experiencing non deterministic behaviour with deterministic samplers, meaning even when giving the exact same prompt, seed and CFG scale using the same seed, this always results in totally different pictures. So in short pressing the generate button twice without changing anything results in two very different pictures with UniPC.

Therefore my question: what might be the reason for that behaviour? Is it using IPEX, so the GPU acceleration, or might there be other switches enabled in the fork by default which could causes this? If the latter, which settings should I take a look at? Thanks in advance!

The pics below show the result of running exact the same parameters two times, one after the other.

[vladmandic]

two things come to mind:

• some random number generators can result in non-deterministic sequence even if its initialized using same rng-seed. that may be the case with ipex implementation, i don't know
• most cross-attention optimizations are non-deterministic by default as it provides a touch more performance. which are you using?

[mstuermer]

Thanks for your quick reply!

I am using the Scaled-Dot-Product cross-attention optimazation at the moment, no advanced options on that enabled.

[vladmandic]

SDP by default is non-deterministic, there is a sub-option just underneath to enable deterministic mode for it.

[mstuermer]

Do you mean the button marked in red, "SDP disable memory attention?" This lead to no change in the behaviour. Also when using other Cross-attention optimization methods or disable it the observed behaviour is still the same.

[vladmandic]

yes, that's the button i meant - that makes SDP deterministic. but that still leaves the first problem of how is random generator behaving.
only way to go around that is not to use random generator provided by gpu libraries and use cpu random generator, but i really don't want to go down that path.

[Mindset-Official]

I added this to devices.py get the seed to reproduce, it won't work on batches you have to set the random seed to .01,

def randn(seed, shape):

if shared.cmd_opts.use_ipex:
	torch.manual_seed(seed), torch.xpu.manual_seed_all(seed)
	
else:
	torch.manual_seed(seed)

if device.type == 'mps':
    return torch.randn(shape, device=cpu).to(device)
return torch.randn(shape, device=device)

I don't actually know how to code but I took a look at another ipex repo and tried different stuff, Not sure if this would interfere with other devices etc.

[vladmandic]

@Mindset-Official thanks, i'll something on those lines.