Reproducing FID

[Godkimchiy]

Hi authors,

Thanks for sharing this great work:)
I cannot reproduce 6.11(t=.9) FID on FFHQ in Table 2.
I used pretrained VQGAN and trained Absorbing diffusion sampler for 2M steps
under the same conditons, but the FID was only 8.10.

Is there sth important I need to check?

[peterhessey]

Hi, we're glad you're managing to successfully use the code!

How many samples are you using to evaluate the FID (--n_samples flag?) I believe we used 50k for those experiments so if you're not using that many that will definitely give an improved FID

[Godkimchiy]

Thank you for reply!
I generated 50,000 samples following README.
Although I still didn't get the reported value, I found that using the trained model
for shorter steps than 2M shows better FID.

and I have one more question.
Is the pretrained VQGAN model you provided trained the same as the one provided by Taming Transformers?

[peterhessey]

Interesting 🤔 What value are you getting for the <2M steps training? We'll take a look into our training logs to get the exact details of the training run used for our 6.11 value.

Is the pretrained VQGAN model you provided trained the same as the one provided by Taming Transformers?

Nope! We trained our own VQGANs. The architecture is almost identical to Taming Transformers' but adapted to allow us to use DiffAug and control other aspects of the model :)

[Godkimchiy]

The only difference is that training was conducted at NVIDIA GeForce RTX 3090, and
other than that, the same commands in README.md were used.

Thanks a lot :)