About Reproducibility

[CCRcmcpe]

"Reproduce" in this context refers to generating visually very close images with the same parameters.

Non-deterministic

WebUI

• WebUI generates noise on GPU.

stable-diffusion-webui/modules/processing.py

Lines 380 to 384 in 3715ece

	# randn results depend on device; gpu and cpu get different results for same seed;
	# the way I see it, it's better to do this on CPU, so that everyone gets same result;
	# but the original script had it like this, so I do not dare change it for now because
	# it will break everyone's seeds.
	noise = devices.randn(seed, noise_shape)

Notice the funny comments.

• WebUI does not implement all determinism-enhancing mechanics suggested by pytorch. See https://pytorch.org/docs/stable/notes/randomness.html

• Obviously WebUI is an actively developing project. All the codes are subject to change.

Hardware

• On GPUs that have different architectures, like 1070 and 2070 (Pascal vs Turing), the results will be not exactly the same, even deviates a lot, both on noise generations and subsequent computings.

• Not to mention results on the same architecture are not guaranteed to be the same, even you use the same GPU, you can still get a different picture, because the overall setup difference.

xformers

(If you use xformers) xformers is not deterministic, even when generating images in the same session, especially if CUTLASS backend is selected, which happens almost all the time. Ref facebookresearch/xformers#618

All of these does not mean you can't achieve determinism, but some condictions should be the same:

• Machine setup
• WebUI version, and all of its dependencies
• Obviously, parameters and options

And xformers should not be enabled for perfect determinism.

Personal Opinion

Some nerds cares about reproducibility. So do I. After those discovery I just bother less about determinism because it can be lost anytime. I enable xformers because perfect determinism is not needed for me.

[CCRcmcpe]

In case if you don't know, there is a breaking change list on wiki:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Seed-breaking-changes

[oakif]

Can they just take a deep breath and change the noise to be CPU-generated? Would help a lot with standardizing reproducibility.

[ClashSAN]

frog, seed 1, euler a, 20 steps. . can you make this? 3715ece
default settings on base model sd 1.4

[mezotaken]

WebUI does not implement all determinism-enhancing mechanics suggested by pytorch

On the pytorch page:

Note that deterministic operations tend to have worse performance than nondeterministic operations.

yeah, i wonder why...

[2u843yt385592yjh]

We were just discussing this here
#7799
My question is, what if we grab that noise value and add it to the metadata, so we force other people's noise into our prompts?

[OstCollector]

I am generating images by CPU in a VM. The host and guest got some huge updates these days, so I tried to reproduce with the same seed and noticed some difference. Now I can produce the same images after some reversion, but I have no time to further investigate into this.
I still found that the number of vcpus affect the output. I have a 1920x with 12c/24t but used to make a 2dies,12cores,1thread VM, which makes pytorch use all 24 cores. Reducing from 24 vcpu to 20 (I wanted to reserve some CPU for other VMs) generates slight difference, which looks like rounding errors. But setting HT on it (2dies/5cores/2threads), which uses 12 vcpus, makes greater and noticeable difference.

[2u843yt385592yjh]

@OstCollector You seem to be right about this. "Different number of cores" instead of "different gpu". This would explain why some people with 4060 get same results as 2060s

I added a print "device"... and after a generation, this is what was printed.