Could not reproduce the results on CIFAR-10

[zzw-zwzhang]

I used your provided model-790000.ckpt on DDIM codebase to generate cifar10 images. But I got the FID is 4.818.

Is there any difference?

[jS5t3r]

how many samples did you generate? FID depends on the number of samples.

[zzw-zwzhang]

how many samples did you generate? FID depends on the number of samples.

Hi, 50000 samples are generated for evaluation.

[jS5t3r]

well, that ought to be fine.... I know on that on the paper there was a FID something around 3...
DDIM is not as good as ddpm

[jS5t3r]

Did you use the "ema_cifar10"?

[zzw-zwzhang]

Did you use the "ema_cifar10"?

It's been a long time and I forgot which model I used. But, if the diffusion model is not trained with ema, its FID should be 12.

After that, I trained a DDPM model from scratch, and the fid is close to that in the paper.

[jS5t3r]

Would you recommend another repository (maybe with pretrained weights on cifar10 ema)?

[zzw-zwzhang]

Would you recommend another repository (maybe with pretrained weights on cifar10 ema)?

I forget which codebase provides reliable pre-trained DDPM models. You can try PNDM first. If you cannot reproduce the results, you can train a DDPM model from scratch.

https://github.com/luping-liu/PNDM
https://github.com/zzw-zwzhang/pytorch-ddpim

[jS5t3r]

Thanks. Here comes an update. Seen on huggingface.

https://huggingface.co/google/ddpm-cifar10-32/discussions/10#646a1aec96cfe72aef787baf

There is also claimed to use ddim with 100 steps and fid of 4.5.