The number of function evaluation (NFE) in ODE mode

[LucasYFL]

Hi, I realized that there is no way to specify NFE in the code. The num-sampling-steps argument is passed to odeint to create the estimated path, per torchdiffeq: https://github.com/rtqichen/torchdiffeq/blob/cae73789b929d4dbe8ce955dace0089cf981c1a0/FAQ.md , which has no effect on how the ODE is solved.

[minghuix98]

Yes. I agree. When using ODE, it's also extremly fast, which could be considered as a very powerful method for fast sampling.
Do you notice that in your implementation?
Do you have the way to know the resulted NFE using ODE even if we can not specify the NFE for using ODE?

[minghuix98]

Hi, I realized that there is no way to specify NFE in the code. The num-sampling-steps argument is passed to odeint to create the estimated path, per torchdiffeq: https://github.com/rtqichen/torchdiffeq/blob/cae73789b929d4dbe8ce955dace0089cf981c1a0/FAQ.md , which has no effect on how the ODE is solved.

I'm guessing using non-adaptive solvers will make specifying the NFE work?
But I haven't tested it.

[minghuix98]

By the way, when i using SDE solver, the output array fulls of nan value. In contrast, the results using ODE is reasonable.

[LucasYFL]

To know the exact NFE, you can add a counter to forward function. I guess using a fix grid solver could specify NFE. Or, a simple DDIM sampler could work as well. Still, I would love to see the author's reply.

[minghuix98]

To know the exact NFE, you can add a counter to forward function. I guess using a fix grid solver could specify NFE. Or, a simple DDIM sampler could work as well. Still, I would love to see the author's reply.

I agree. Seems like SiT didn't get too much attention compared with DiT. But SiT works better compared with DiT on my own data.

[willisma]

Thanks for the discussion. We used a self-implemented second-order Heun ODE solver for the numbers reported in the paper, which is fixed to be 125 steps (so 250 NFEs) exactly. We observed similar performance with a vanilla Euler (DDIM) sampler and a black-box dopri5.

[ShoufaChen]

Hi @willisma,

Thank you very much for your clarification.

May I kindly ask if you have any plans to release your self-implemented second-order Heun ODE solver?

Additionally, do you recall how similar the performances are between DDIM and the black-box Dropi5?

Thank you in advance for your response.

[willisma]

Hi @ShoufaChen,

We'll release the Heun solver soon. Regarding the performance difference, I believe the FID difference on 400K B/2 model is < 0.1

[joh-schb]

Are there any updates on when you plan to release the self-implemented second-order Heun ODE solver? Would it be equivalent to using the SDE Heun version and setting the diffusion norm to zero?