Problems encountered in training

[1hexf1]

Hello, author. At the beginning of the training, the system asked me for the netG model of the 600th training. After I downloaded the model you gave me, the program continued to run, but stopped after only a few minutes. Ask why you need the model for the 600th run to start training and why the program stops after a few minutes. Thanks to the author.
我的英文不太好，加一个我原本的意思。
作者你好，在开始训练时系统向我索要第600次训练的netG模型。我在您所给的模型下载下来后，程序可以继续运行，但只运行了几分钟就会停下了。请问为什么开始训练需要第600次的模型，还有为什么程序运行几分钟就会停下来。感谢作者。

[wtliao]

@1hexf1 那一行直接输入第600次的次训练的netG模型是我为了直接看600次的次训练的netG模型生成图片的结果而设置的。你同步一下我现在的repo，我把这些删了，加了些注释。运行几分钟就停下来是因为现在是测试生成图片的结果，我忘记改回train模式了，你可以看一下代码。

[1hexf1]

@wtliao 作者你好，请问一下，我已经训练好模型，我应该怎么使用程序文本生成图片，谢谢作者。

[priyankaupadhyay090]

Hello @wtliao author, may I ask, I have trained the model, how should I use procedural text to generate pictures, thank the author.

Hey, I needed some help in training using main.py. I am getting some error and would much appreciate your help.

[priyankaupadhyay090]

@wtliao and @BossaMelon Thank you for the great work. Your paper is really helpful to understand the topic.

can I please ask what results I should expect after training ? either I run main.py or main_finetune.py (by changing the path into bird.yml) I dont see any results, or no generated fake images. No results inside caps, real, test_every, and valid folder.

this is the output after running "main_finetune.py"

python main_finetune.py
Using config:
{'B_VALIDATION': True,
'CONFIG_NAME': 'bird_sloss01',
'CUDA': True,
'DATASET_NAME': 'birds',
'DATA_DIR': 'data/birds',
'GAN': {'B_ATTENTION': True,
'B_DCGAN': True,
'CONDITION_DIM': 100,
'DF_DIM': 64,
'GF_DIM': 128,
'R_NUM': 2,
'Z_DIM': 100},
'GPU_ID': 0,
'RESTORE': False,
'RNN_TYPE': 'LSTM',
'TEXT': {'CAPTIONS_PER_IMAGE': 10,
'DAMSM_NAME': 'DAMSMencoders/bird/text_encoder200.pth',
'EMBEDDING_DIM': 256,
'WORDS_NUM': 18},
'TRAIN': {'BATCH_SIZE': 24,
'B_NET_D': True,
'DISCRIMINATOR_LR': 0.0002,
'DSAVE_INTERVAL': 10,
'ENCODER_LR': 0.0002,
'FLAG': True,
'GENERATOR_LR': 0.0002,
'GSAVE_INTERVAL': 10,
'MAX_EPOCH': 600,
'NET_E': '',
'NET_G': 'trained_model/finetune/cub/netG_550.pth',
'NF': 64,
'RNN_GRAD_CLIP': 0.25,
'SMOOTH': {'GAMMA1': 5.0,
'GAMMA2': 5.0,
'GAMMA3': 10.0,
'LAMBDA': 1.0},
'SNAPSHOT_INTERVAL': 2000,
'WARMUP_EPOCHS': 100},
'TREE': {'BASE_SIZE': 256, 'BRANCH_NUM': 1},
'USE_SN': False,
'WORKERS': 1,
'loss': 'hinge'}
seed now is : 100
Total filenames: 11788 001.Black_footed_Albatross/Black_Footed_Albatross_0046_18.jpg
Load filenames from: data/birds/train/filenames.pickle (8855)
Load filenames from: data/birds/test/filenames.pickle (2933)
Load from: data/birds/captions.pickle
5450 10
/opt/conda/lib/python3.6/site-packages/torch/nn/modules/rnn.py:61: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.5 and num_layers=1
"num_layers={}".format(dropout, num_layers))
Load pretrained inception v3 model
step: 0
step: 100
state_epoch: 0

and after this nothing happens, execution is finished now. in the code we are creating many new folders and saving the results but I dont find any results inside caps, real, test_every, and valid folder. Please let me know if I am missing something. Looking forward to hear from you soon :)

[wtliao]

@wtliao 作者你好，请问一下，我已经训练好模型，我应该怎么使用程序文本生成图片，谢谢作者。

sorry for my very late reply. In the main.py you can find the generate_sample function, from which you can generate images with your captions.

[wtliao]

@wtliao and @BossaMelon Thank you for the great work. Your paper is really helpful to understand the topic.

can I please ask what results I should expect after training ? either I run main.py or main_finetune.py (by changing the path into bird.yml) I dont see any results, or no generated fake images. No results inside caps, real, test_every, and valid folder.

this is the output after running "main_finetune.py"

python main_finetune.py Using config: {'B_VALIDATION': True, 'CONFIG_NAME': 'bird_sloss01', 'CUDA': True, 'DATASET_NAME': 'birds', 'DATA_DIR': 'data/birds', 'GAN': {'B_ATTENTION': True, 'B_DCGAN': True, 'CONDITION_DIM': 100, 'DF_DIM': 64, 'GF_DIM': 128, 'R_NUM': 2, 'Z_DIM': 100}, 'GPU_ID': 0, 'RESTORE': False, 'RNN_TYPE': 'LSTM', 'TEXT': {'CAPTIONS_PER_IMAGE': 10, 'DAMSM_NAME': 'DAMSMencoders/bird/text_encoder200.pth', 'EMBEDDING_DIM': 256, 'WORDS_NUM': 18}, 'TRAIN': {'BATCH_SIZE': 24, 'B_NET_D': True, 'DISCRIMINATOR_LR': 0.0002, 'DSAVE_INTERVAL': 10, 'ENCODER_LR': 0.0002, 'FLAG': True, 'GENERATOR_LR': 0.0002, 'GSAVE_INTERVAL': 10, 'MAX_EPOCH': 600, 'NET_E': '', 'NET_G': 'trained_model/finetune/cub/netG_550.pth', 'NF': 64, 'RNN_GRAD_CLIP': 0.25, 'SMOOTH': {'GAMMA1': 5.0, 'GAMMA2': 5.0, 'GAMMA3': 10.0, 'LAMBDA': 1.0}, 'SNAPSHOT_INTERVAL': 2000, 'WARMUP_EPOCHS': 100}, 'TREE': {'BASE_SIZE': 256, 'BRANCH_NUM': 1}, 'USE_SN': False, 'WORKERS': 1, 'loss': 'hinge'} seed now is : 100 Total filenames: 11788 001.Black_footed_Albatross/Black_Footed_Albatross_0046_18.jpg Load filenames from: data/birds/train/filenames.pickle (8855) Load filenames from: data/birds/test/filenames.pickle (2933) Load from: data/birds/captions.pickle 5450 10 /opt/conda/lib/python3.6/site-packages/torch/nn/modules/rnn.py:61: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.5 and num_layers=1 "num_layers={}".format(dropout, num_layers)) Load pretrained inception v3 model step: 0 step: 100 state_epoch: 0

and after this nothing happens, execution is finished now. in the code we are creating many new folders and saving the results but I dont find any results inside caps, real, test_every, and valid folder. Please let me know if I am missing something. Looking forward to hear from you soon :)

Hi, thanks for playing my work. Every thing is in main.py: training, fitune and inference. You can start to generate some images with my trained model to get familiar with the work. About the inference, I have answered the similar question in #12 (comment)

[priyankaupadhyay090]

@wtliao and @BossaMelon Thank you for the great work. Your paper is really helpful to understand the topic.
can I please ask what results I should expect after training ? either I run main.py or main_finetune.py (by changing the path into bird.yml) I dont see any results, or no generated fake images. No results inside caps, real, test_every, and valid folder.
this is the output after running "main_finetune.py"
python main_finetune.py Using config: {'B_VALIDATION': True, 'CONFIG_NAME': 'bird_sloss01', 'CUDA': True, 'DATASET_NAME': 'birds', 'DATA_DIR': 'data/birds', 'GAN': {'B_ATTENTION': True, 'B_DCGAN': True, 'CONDITION_DIM': 100, 'DF_DIM': 64, 'GF_DIM': 128, 'R_NUM': 2, 'Z_DIM': 100}, 'GPU_ID': 0, 'RESTORE': False, 'RNN_TYPE': 'LSTM', 'TEXT': {'CAPTIONS_PER_IMAGE': 10, 'DAMSM_NAME': 'DAMSMencoders/bird/text_encoder200.pth', 'EMBEDDING_DIM': 256, 'WORDS_NUM': 18}, 'TRAIN': {'BATCH_SIZE': 24, 'B_NET_D': True, 'DISCRIMINATOR_LR': 0.0002, 'DSAVE_INTERVAL': 10, 'ENCODER_LR': 0.0002, 'FLAG': True,'GENERATOR_LR': 0.0002, 'GSAVE_INTERVAL': 10, 'MAX_EPOCH': 600, 'NET_E': '', 'NET_G': 'trained_model/finetune/cub/netG_550.pth', 'NF': 64, 'RNN_GRAD_CLIP': 0.25, 'SMOOTH': {'GAMMA1': 5.0, 'GAMMA2': 5.0, 'GAMMA3': 10.0, 'LAMBDA': 1.0}, 'SNAPSHOT_INTERVAL': 2000, 'WARMUP_EPOCHS': 100}, 'TREE': {'BASE_SIZE': 256, 'BRANCH_NUM': 1}, 'USE_SN': False, 'WORKERS': 1, 'loss': 'hinge'} seed now is : 100 Total filenames: 11788 001.Black_footed_Albatross/Black_Footed_Albatross_0046_18.jpg Load filenames from: data/birds/train/filenames.pickle (8855) Load filenames from: data/birds/test/filenames.pickle (2933) Load from: data/birds/captions.pickle 5450 10 /opt/conda/lib/python3.6/site-packages/torch/nn/modules/rnn.py:61: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.5 and num_layers=1 "num_layers={}".format(dropout, num_layers)) Load pretrained inception v3 model step: 0 step: 100 state_epoch: 0
and after this nothing happens, execution is finished now. in the code we are creating many new folders and saving the results but I dont find any results inside caps, real, test_every, and valid folder. Please let me know if I am missing something. Looking forward to hear from you soon :)

Hi, thanks for playing my work. Every thing is in main.py: training, fitune and inference. You can start to generate some images with my trained model to get familiar with the work. About the inference, I have answered the similar question in #12 (comment)

Thank you, I am working on it right now :)