Some questions

[TheshowN]

adjust = np.sign(sign(Discriminator(real_images)) - ada_target) * C # C = (batch_size * ada_interval) / (ada_kimg * 1000)
or
adjust = np.sign(sign(Discriminator(real_images) - 0.5) - ada_target) * C # C = (batch_size * ada_interval) / (ada_kimg * 1000) according to the paper?

1. Which one in correct?
2. Any suggestions for the setting of 'ada_kimg'？
3. Are these lines of code suitable for multi-gpu training? 'Diffusion' will get different updates on different ranks? And how to set the 'batch_size'?

I think your work is wonderful! Thank you for your reply in advance!

[Zhendong-Wang]

Hi there,

Thanks for you interest in our work. Sorry to make the confusion there.

1. The answer is that they are both correct. In paper, our narative is based on the original GAN, whose discriminator outputs the probability that the given image is real, so 0.5 is the middle point. However, in practice, people use different loss functions and the probablity might not be actually trained. Here, we built our code based on the StyleGAN2 and the Discriminator(real_images) will output the logits not the probability so you don't need - 0.5 here. The middle point of the logits for GAN is 0.0.
2. We set ada_kimg=100 for all experiments, which is also the default value for ada_kimg.
3. Yes, the code is suitable for multi-gpu training by just including --gpus=4 in the command. Yes, 'Diffusion' gets different updates on different ranks. batch_size is the batch_size used for training. You could see details in the diffusion-stylegan2/train.py dictionay cfg_specs, which is inherited from StyleGAN2 paper.