-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem in reproducing fid score #22
Comments
This one I am unsure as I have never tried the link you referred to before.
This is weird. Do you mind sharing the hardware setup? E.g., the number of GPUs, etc. For GAN's training, the batch size matters a lot. If possible, please use the same batch size as specified in the paper.
I am sorry and I definitely understand that this is a headache. BTW, I recently encountered a tool, i.e., rclone, and it can interact with GDrive well (on the remote server, etc). Please refer to the documentation for how to set it up. The only thing that needs some effort is to set up a Google Cloud API, which is not hard by following its instruction. Hope these help. |
Hi, @Xiaoming-Zhao! I'm facing a similar issue. I trained my model with the following configuration: In the original paper, the model was trained with a batch size of 64, so I also tried that, but the FID score still increased. Previously, when I trained with a batch size of 8, the FID score was around 20 and remained stable until the end of 5000 steps. However, this time, even though I started with a lower FID score of 18, it kept increasing after around 1000 steps. I'm wondering if using ffhq256x256 data from Kaggle instead of real 256-sized data could be causing overfitting. Because I think the size of the Kaggle set is a little smaller. Are there any other possible reasons for this behavior? Thanks! |
Hi @parkjh688, I need several more information if possible.
How many GPUs did you use to train GMPI? The reason I am asking is that the
What is the dataset you used for this training? And how many GPUs were you using?
One caveat I could see is that the Kaggle dataset uses a different resizing method from the one the pre-trained StyleGAN2 uses. Specifically: So maybe you want to process the dataset following the official instruction to have a double check.
I noticed that you have Hope these help. |
Oh, I didn't know that. I used 6 GPUs and my configuration in curriculums.py was 256: {'batch_size': 64, 'num_steps': 32, 'img_size': 256, 'tex_size': 256, 'batch_split': 16, 'gen_lr': 0.002, 'disc_lr': 0.002} this. Then if I want to train the model with 64 batch size I have to train it with 4 GPUs and the configuration will be 256: {'batch_size': 16, 'num_steps': 32, 'img_size': 256, 'tex_size': 256, 'batch_split': 1, 'gen_lr': 0.002, 'disc_lr': 0.002} like this. 16 (per GPU) x 4 (#GPUs) = 64
I use this kaggle dataset.. Thanks! |
Got you. So the Kaggle dataset is indeed able to reproduce FID based on your previous statement
Then I would recommend reducing the Hope this helps. |
Hello, thank you for your detailed help. "res_dict": {
But the fid goes up and the result was like this. |
Do you mind trying the default configuration: Lines 90 to 94 in 672294b
See the discussion above in this issue. Essentially:
Hope these help. |
Hello! Thank you for your kind help.
The fid score with 256 was 18.92-21.52. (with one peak pertubation) |
This curve looks reasonable to me. I am not sure about the peak but I guess it may due to some randomness. Regarding the FID: FID score largely depends on the number of images used to compute the score. The more images you use, the large probability you will obtain a lower score. However, FID with plenty of images is costly to compute. Therefore, during training, we use a small number of images to get a sense of the FID trend:
During the full evaluation, we use 50k fake and real images as stated in the paper and this follows StyleGAN's papers: Line 17 in 672294b
Hope this resolves your confusion. |
Thank you for amazing work again.
I have some trouble to reproducing fid score..
(https://www.kaggle.com/datasets/denislukovnikov/ffhq256-images-only)
Could you give me some advice about this situation?
(I will try again with 1025 90G version with your advice..
Also,, if you give me some personal help please email me.. I will so appreciate your help. howtowhy@gmail.com
I really want to refer your research but.. dataset problem is not easy for me... )
Thank you so much.
The text was updated successfully, but these errors were encountered: