Switching to DPM++ SDE Karras sampling method

[gameveloster]

How can I change the scheduler in StableDiffusionPipeline to use what AUTOMATIC111's gui calls the "DPM++ SDE Karras" sampling method?

Also, whats the default sampling method being used by StableDiffusionPipeline?

[patrickvonplaten]

This might be the only scheduler method we don't have yet - I assume you referring to: https://github.com/crowsonkb/k-diffusion/blob/41b4cb6df0506694a7776af31349acf082bf6091/k_diffusion/sampling.py#L543

Think this is already the second issue about it, would be extremely nice to add it then. Maybe someone from the community has time to do it ? :-) Will try to find some time otherwise.

Here a nice doc on how to change schedulers in general: https://huggingface.co/docs/diffusers/main/en/using-diffusers/schedulers

[A0158]

You can use this to find alternative of DPM++ SDE Karras in StableDiffusionPipeline :
https://huggingface.co/docs/diffusers/v0.11.0/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler

You can also try:
https://huggingface.co/docs/diffusers/api/schedulers/dpm_discrete

The default sampling method used by StableDiffusionPipeline is PNDMScheduler.

I hope this helps :)

[A0158]

Hi @patrickvonplaten

I am interested to add it.

[patrickvonplaten]

@A0158,

This would be great :-) Do you want to open a PR for it?

It's quite useful to compare your implementation with https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_k_diffusion.py to make sure results are the same

[A0158]

Yeah ! I will open a PR for it.

Thanks for letting me know. I will compare the implementation to verify the results.

[jslegers]

Has output for the different schedulers in diffusers been compared with output in the AUTOMATIC1111 gui?

When testing Epic Diffusion, I was unable to reproduce the output of a basic diffusers script with that of any of the schedulers in the AUTOMATIC1111 GUI, in spite of using the same seed, CFG scale & other parameters.

I initially suspected this may be related to how the AUTOMATIC1111 GUI overcomes the 77 token limit of CLIP, but after reading the description of how they achieve this I'm less convinced...

See also this issue.

[patrickvonplaten]

Hey @jslegers,

For the future, it would be great if you could open a new issue. I've answered here in more detail: https://huggingface.co/spaces/anzorq/finetuned_diffusion/discussions/64

[jslegers]

For the future, it would be great if you could open a new issue.

Personally, I'm not a fan of raising issues for just asking a question. In my experience, as a senior software developer, this easily leads to issue-spam and makes it difficult to keep an overview of actual bugs & feature requests. Also, I felt this issue was related enough to not warrant raising a new issue.

But I'll try to keep your different preference in mind for the future whenever I have similar questions.

https://huggingface.co/spaces/anzorq/finetuned_diffusion/discussions/64

Thanks for the link.

[patrickvonplaten]

It's a bit easier to find issues according to the issue title later on, so we usually prefer to open new issues when the topic is slightly different.

[nipunjindal]

Hello @patrickvonplaten,

I'd like to pick this ticket and try to raise a PR.
Let me know if that sounds ok?

[patrickvonplaten]

Hey @nipunjindal,

This sounds great! Let me know if you need help

[nipunjindal]

Hello @patrickvonplaten,

Happy to get some help here. I am unable to find text related to this scheduler, do you mind sharing the way to go about it?
Thanks! Should be able to complete it this week, once I get jump-started.

[patrickvonplaten]

Hey @nipunjindal,

Would this feature request maybe help: #2905

Essentially, it's not really a new scheduler, it just uses different sigmas. We could quite easily support this with a use_karras_sigmas flag I think :-)

[patrickvonplaten]

Hey @nipunjindal,

Would this feature request maybe help: #2905

Essentially, it's not really a new scheduler, it just uses different sigmas. We could quite easily support this with a use_karras_sigmas flag I think :-)

[nipunjindal]

Hello @patrickvonplaten,

Thanks definitely this is helpful (simplifies it all :)), to me the sample_dpmpp_sde code din't appear same as dpm++ multi step. But I can always compare with the kdiffusion pipeline.
Nonetheless, on this journey I went through the issue of Scheduler Design and it was super helpful for me to be get more accustomed to the system. Will report back tomorrow.

[LuChengTHU]

I've added the "correct" version of the SDE-type DPM-Solver++: #3344

[sayakpaul]

Closing this with #3344