The schedule of beta when T is changing

[Guo-Stone]

When changing diffusion time T, why is the beta scaled? In DDIM, when shortening the diffusion time T, they don't scale the beta. Your code is as following：
-----------------------------------Python Code----------------------------------

Adjust b0 and bT if T is not 200

This is a good approximation, with the beta correction below, unless T is very small

assert T >= 15, "With discrete time and T < 15, the schedule is badly approximated"
b0 *= 200 / T
bT *= 200 / T

[blt2114]

Hi @Guo-Stone,

Thanks for the good question. We did not properly explain this in either the code or in the paper supplement.

In our initial implementation assumed a fixed number of time steps T=200, and set the sequence of beta's as in DDPM [1]. However, this can be adapted for inference with a smaller number of time steps by appealing to the continuous-time SDE perspective outlined in [2].

Specifically, schedule, as computed here (

RFdiffusion/rfdiffusion/diffusion.py

Line 34 in 820bfdf

schedule = torch.linspace(b0, bT, T)

) agrees with the variables $\beta_i$ in appendix C of [2], when you take $\bar \beta_{min}=b_0 * 200$ and $\beta_{max}=b_T*200$.

As described in appendix C of [2], when the number of steps (T) is large, this schedule agrees with the variance preserving SDE with $\beta(t)=\bar \beta_{min} + t (\bar \beta_{max}- \bar \beta_{min}).$

But when T< 15, we find that this approximation is inaccurate, and empirically find more steps are needed. I suspect, however, that other more clever scalings of intermediate steps could allow good inferences with perhaps still significantly fewer steps.

Does this help?

References:
[1] Denoising Diffusion Probabilistic Models https://arxiv.org/abs/2006.11239
[2] Score-Based Generative Modeling through Stochastic Differential Equations https://arxiv.org/abs/2011.13456

[Guo-Stone]

Thanks for your excellent reply! I am sorry that I couldn't explain my question and misled you. I just confuse about when we changes the number of steps (T), why we need to scale b0 and bT. As in your code:
b0 *= 200 / T
bT *= 200 / T
And I suspect why you do that is to accelerate generation process as DDIM [1] does. I am not sure and looking forward to your response!
Reference:
[1] DENOISING DIFFUSION IMPLICIT MODELS https://arxiv.org/abs/2010.02502

[sokrypton]

Easiest way to explain this:
The model was trained with 200 steps. If you want to take less than 200 steps, then you need to take larger steps to eventually end up at the same point.

So if you want to take 100 steps, then your step size should be 200/100 = 2.0

[Guo-Stone]

Thanks for your easy explanation! I see~ orz