[docs] Clean scheduler api #4204
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
stevhliu
force-pushed
the
clean-scheduler-api
branch
from
293d82a
to
bee6717
Compare
| ## What is a scheduler? | ||
|
|
||
| The schedule functions, denoted *Schedulers* in the library take in the output of a trained model, a sample which the diffusion process is iterating on, and a timestep to return a denoised sample. That's why schedulers may also be called *Samplers* in other diffusion models implementations. | ||
|
|
||
| - Schedulers define the methodology for iteratively adding noise to an image or for updating a sample based on model outputs. | ||
| - adding noise in different manners represent the algorithmic processes to train a diffusion model by adding noise to images. | ||
| - for inference, the scheduler defines how to update a sample based on an output from a pretrained model. | ||
| - Schedulers are often defined by a *noise schedule* and an *update rule* to solve the differential equation solution. | ||
|
|
||
| ### Discrete versus continuous schedulers | ||
|
|
||
| All schedulers take in a timestep to predict the updated version of the sample being diffused. | ||
| The timesteps dictate where in the diffusion process the step is, where data is generated by iterating forward in time and inference is executed by propagating backwards through timesteps. | ||
| Different algorithms use timesteps that can be discrete (accepting `int` inputs), such as the [`DDPMScheduler`] or [`PNDMScheduler`], or continuous (accepting `float` inputs), such as the score-based schedulers [`ScoreSdeVeScheduler`] or [`ScoreSdeVpScheduler`]. |
There was a problem hiding this comment.
I'd be in favor of keeping this information. Any reason not to?
There was a problem hiding this comment.
Most of the information is still there, I just made it less wordy and more concise. The only information I removed is how schedulers are designed (this can be found in the Philosophy).
There was a problem hiding this comment.
From the commits, it felt like the sections (that should be present like this one) are entirely gone.
There was a problem hiding this comment.
Hmm that's strange, I can still see all the info there:
Any idea how to get this test to pass? |
|
Hmm the failed test is very new to me. @patrickvonplaten any idea? |
|
@sayakpaul @patrickvonplaten @stevhliu about the failing test, I think you need to manually update this reference code |
|
Thank you so much @yiyixuxu, that did the trick! ❤️ |
* clean scheduler mixin * up to dpmsolvermultistep * finish cleaning * first draft * fix overview table * apply feedback * update reference code
* clean scheduler mixin * up to dpmsolvermultistep * finish cleaning * first draft * fix overview table * apply feedback * update reference code
This PR focuses on tidying up the Scheduler API section:
[[autodoc]]forKarrasDiffusionSchedulerssince it seems like it is more of a mapping?toctreefor consistency (important to use the same language everywhere to avoid confusion, for example DPM Discrete Scheduler -->KDPM2DiscreteScheduler)SchedulerOutputfor each scheduler