Current dev environment produce different pictures than before, could not use ddim and plms - noise output

[ArDiouscuros]

Describe your environment

• GPU: [cuda]
• VRAM: [6GB]
• CPU arch: [x86]
• OS: [Windows]
• Python: [miniconda]
• Branch: On branch upstream-development - Your branch is up to date with 'upstream/development'
• Commit: commit 4f247a3

Describe the bug
On development branch pictures generated with the same arguments now are very different.
Samplers ddim and plms produce noise instead of images

To Reproduce
Steps to reproduce the behavior:
Command: "brown corgi, clean background, flat design:1.0" -s 20 -S 1 -W 512 -H 512 -C 7.5 -A plms

Expected behavior
Expected picture the same as it was in main / development on 30 Sep

Screenshots

[metal3d]

I've got exactly the same problem in "development" branch, the output for others than KLMS sampler is the noise, not the output image.

[ArDiouscuros]

@lstein
Good news: commit 333219b restored plms/ddim samplers.
Still getting totally different results with k samplers.

[Any-Winter-4079]

@lstein I'm using an older commit, 7541c7c, from last night (where img2img is fixed for low strengths), and I'm seeing results have already changed by this commit for regular txt2img. It might even predate this commit.
"an anime girl" -s 50 -S 3031912972 -W 512 -H 512 -C 7.5 -A k_lms
7541c7c

Vs. older behavior

If there are changes to the behavior of the samplers, I (strongly) suggest adding it as an optional feature, as otherwise we are making all older generations not be reproducible, which is not good (reproducibility helps for testing purposes, among other things, such as re-generating after losing the image file, re-generating using more steps, re-generating with variations or new features introduced, etc.)

For example, it shouldn't be either use Karras o don't use Karras, but be able to choose between both options.
And same with any other change to the behavior of the samplers that affects results visually.

[Any-Winter-4079]

Btw, just to clarify, I don't think this change in behavior is introduced by karras. See how little the resulting image seems to change with Karras vs no Karras https://github.com/invoke-ai/InvokeAI/discussions/914#discussioncomment-3800884
I'm just using it as an example of something that results in a change of visual output.

[ArDiouscuros]

@Any-Winter-4079
With #977 I could reproduce my old images, but with the same prompt mentioned here #966 (comment) I could not get the same image. Now I am confused - are prompts and images are user/computer/etc. dependent?

[Any-Winter-4079]

@ArDiouscuros Results are Cuda / M1 specific.
Cuda machines can reproduce results from other Cuda machines but not M1 (Mac).
M1 machines can reproduce results from other M1 machines but not Cuda.
(Until a fix is found)

[ArDiouscuros]

@Any-Winter-4079

M1 machines can reproduce results from other M1 machines but not Cuda.

Well, it was interesting exercise.
Just generated following image on my Windows CUDA machine:
prompt: "an anime girl" -s 50 -S 3031912972 -W 512 -H 512 -C 7.5 -A k_lms
result:

prompt: "an anime girl" -s 25 -S 3031912972 -W 512 -H 512 -C 7.5 -A k_lms --use_mps_noise
result:

[lstein]

We do not have consistency between CUDA and CPU-based rendering when using the same seed, and never have. I think the random number generator is different on the two platforms, and will come up with a test for that bit of speculation. Lincoln

On Fri, Oct 7, 2022 at 4:54 PM ArDiouscuros ***@***.***> wrote: M1 machines can reproduce results from other M1 machines but not Cuda. Just generated following image on my Windows CUDA machine: prompt: "an anime girl" -s 50 -S 3031912972 -W 512 -H 512 -C 7.5 -A k_lms result: [image: image] <https://user-images.githubusercontent.com/72071512/194650727-c8632c06-03ed-4bd3-a9cb-e17118a415e6.png> prompt: "an anime girl" -s 25 -S 3031912972 -W 512 -H 512 -C 7.5 -A k_lms --use_mps_noise result: [image: image] <https://user-images.githubusercontent.com/72071512/194650838-5d45001b-acd4-4588-a9f4-3daea208f5d4.png> — Reply to this email directly, view it on GitHub <#966 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3EVM6BCWFL7ZNDBAY663WCCEY3ANCNFSM6AAAAAAQ7KRNJ4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Written on my cell phone. Anything that seems odd is the fault of auto-correct.

[ArDiouscuros]

@lstein
I have made a quick test here: #981 (comment)
As I have no access to M1 it is difficult to test and compare, but on a first glance I am getting the same results as in
https://github.com/invoke-ai/InvokeAI/blob/development/docs/help/SAMPLER_CONVERGENCE.md

[Any-Winter-4079]

@lstein he's done it though. He has recreated on cuda an mps image! Awesome :)

[Any-Winter-4079]

@ArDiouscuros yes, you are. It'd be an incredible coincidence otherwise.
Could the opposite be done? So M1 can reproduce cuda images.

[Any-Winter-4079]

This is fixed now and the ability to reproduce mps images on cuda is already on a PR, so I'm closing this issue.