Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve issue mentioned in #242 #246

Merged
merged 3 commits into from
Nov 19, 2019
Merged

Conversation

gpengzhi
Copy link
Collaborator

Resolve #242
Fix variable scope issue in dynamic decode function.

Copy link
Member

@ZhitingHu ZhitingHu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The beam_search_decode uses tf.contrib.seq2seq.dynamic_decode. Why the dynamic_decode inside texar would affect beam-search?

Please replicate seq2seq_attn, transformer, and text_style_transfer results first before merging

@gpengzhi
Copy link
Collaborator Author

The beam_search_decode uses tf.contrib.seq2seq.dynamic_decode. Why the dynamic_decode inside texar would affect beam-search?

Please replicate seq2seq_attn, transformer, and text_style_transfer results first before merging

We here make texar.tf.modules.decoders.dynamic_decode has the same variable scope as that in tensorflow.contrib.seq2seq.dynamic_decode so that everything learnt in the training procedure with texar.tf.modules.decoders.dynamic_decode can be reused in the inferencing procedure with tensorflow.contrib.seq2seq.dynamic_decode.

@gpengzhi
Copy link
Collaborator Author

python seq2seq_attn.py --config_model config_model --config_data config_toy_copy
step=0, loss=31.1855
step=50, loss=18.9741
step=100, loss=21.2726
step=150, loss=3.9846
step=200, loss=0.4235
step=250, loss=0.2036
step=300, loss=0.1700
val epoch=0, BLEU=91.9000; best-ever=91.9000
test epoch=0, BLEU=93.1400
==================================================
step=0, loss=1.2390
step=50, loss=7.9516
step=100, loss=0.6531
step=150, loss=1.1278
step=200, loss=0.4025
step=250, loss=0.2532
step=300, loss=0.2689
val epoch=1, BLEU=99.3900; best-ever=99.3900
test epoch=1, BLEU=99.5000
==================================================
step=0, loss=0.3637
step=50, loss=0.1413
step=100, loss=0.1402
step=150, loss=0.2444
step=200, loss=0.0238
step=250, loss=0.4120
step=300, loss=0.2562
val epoch=2, BLEU=98.8400; best-ever=99.3900
test epoch=2, BLEU=98.7600
==================================================
step=0, loss=0.2224
step=50, loss=0.3302
step=100, loss=0.0373
step=150, loss=0.0894
step=200, loss=0.0326
step=250, loss=1.4304
step=300, loss=0.4831
val epoch=3, BLEU=99.6100; best-ever=99.6100
test epoch=3, BLEU=99.2100

@gpengzhi
Copy link
Collaborator Author

python transformer_main.py --run_mode=train_and_evaluate --config_model=config_model --config_data=config_iwslt15
I1118 16:11:46.697821 140642838472448 transformer_main.py:269] step: 500, loss: 7.1598
step: 500, loss: 7.1598
I1118 16:37:26.853804 140642838472448 transformer_main.py:269] step: 1000, loss: 6.2243
step: 1000, loss: 6.2243
I1118 17:02:50.844758 140642838472448 transformer_main.py:269] step: 1500, loss: 5.5672
step: 1500, loss: 5.5672
I1118 17:28:42.947324 140642838472448 transformer_main.py:269] step: 2000, loss: 5.4850
step: 2000, loss: 5.4850
I1118 18:18:29.563245 140642838472448 transformer_main.py:213] epoch: 0, eval_bleu 4.4055
epoch: 0, eval_bleu 4.4055
I1118 18:18:29.563424 140642838472448 transformer_main.py:217] epoch: 0, best bleu: 4.4055
I1118 18:18:29.563504 140642838472448 transformer_main.py:221] saving model to ./outputs/best-model.ckpt
saving model to ./outputs/best-model.ckpt

@gpengzhi
Copy link
Collaborator Author

python main.py --config config
gamma: 1.0, lambda_g: 0.0
step: 1, loss_d: 0.6908 accu_d: 0.6250
step: 1, loss_g: 9.1450 loss_g_ae: 9.1450 loss_g_clas: 0.6957 accu_g: 0.4062 accu_g_gdy: 0.4531
step: 500, loss_d: 0.1236 accu_d: 0.9625
step: 500, loss_g: 4.4774 loss_g_ae: 4.4774 loss_g_clas: 0.1794 accu_g: 0.9406 accu_g_gdy: 0.8531
step: 1000, loss_d: 0.0985 accu_d: 0.9688
step: 1000, loss_g: 1.5501 loss_g_ae: 1.5501 loss_g_clas: 1.4881 accu_g: 0.5281 accu_g_gdy: 0.5172
step: 1500, loss_d: 0.0933 accu_d: 0.9672
step: 1500, loss_g: 0.9056 loss_g_ae: 0.9056 loss_g_clas: 2.9621 accu_g: 0.2875 accu_g_gdy: 0.3063
step: 2000, loss_d: 0.0964 accu_d: 0.9641
step: 2000, loss_g: 0.6762 loss_g_ae: 0.6762 loss_g_clas: 3.6231 accu_g: 0.2344 accu_g_gdy: 0.2141
step: 2500, loss_d: 0.0729 accu_d: 0.9766
step: 2500, loss_g: 0.5492 loss_g_ae: 0.5492 loss_g_clas: 4.2097 accu_g: 0.1797 accu_g_gdy: 0.1672
step: 3000, loss_d: 0.0684 accu_d: 0.9656
step: 3000, loss_g: 0.4170 loss_g_ae: 0.4170 loss_g_clas: 4.6914 accu_g: 0.1547 accu_g_gdy: 0.1375
step: 3500, loss_d: 0.0801 accu_d: 0.9781
step: 3500, loss_g: 0.4192 loss_g_ae: 0.4192 loss_g_clas: 4.9181 accu_g: 0.1625 accu_g_gdy: 0.1594
step: 4000, loss_d: 0.0625 accu_d: 0.9703
step: 4000, loss_g: 0.3330 loss_g_ae: 0.3330 loss_g_clas: 5.1321 accu_g: 0.1219 accu_g_gdy: 0.1047
step: 4500, loss_d: 0.0643 accu_d: 0.9734
step: 4500, loss_g: 0.2973 loss_g_ae: 0.2973 loss_g_clas: 5.0299 accu_g: 0.1156 accu_g_gdy: 0.1031
step: 5000, loss_d: 0.0873 accu_d: 0.9656
step: 5000, loss_g: 0.2669 loss_g_ae: 0.2669 loss_g_clas: 5.7881 accu_g: 0.1031 accu_g_gdy: 0.0844
step: 5500, loss_d: 0.0612 accu_d: 0.9766
step: 5500, loss_g: 0.2889 loss_g_ae: 0.2889 loss_g_clas: 5.2957 accu_g: 0.0922 accu_g_gdy: 0.1062
step: 6000, loss_d: 0.0820 accu_d: 0.9719
step: 6000, loss_g: 0.2769 loss_g_ae: 0.2769 loss_g_clas: 5.6069 accu_g: 0.1031 accu_g_gdy: 0.1156
step: 6500, loss_d: 0.0933 accu_d: 0.9672
step: 6500, loss_g: 0.2454 loss_g_ae: 0.2454 loss_g_clas: 6.1655 accu_g: 0.0750 accu_g_gdy: 0.0719
epoch: 1, loss_d: 0.0675 accu_d: 0.9859
epoch: 1, loss_g: 0.2187 loss_g_ae: 0.2187 loss_g_clas: 5.9780 accu_g: 0.0924 accu_g_gdy: 0.0924
val: loss_g: 0.1296 loss_g_ae: 0.1296 loss_g_clas: 6.4902 loss_d: 0.0660 accu_d: 0.9767 accu_g: 0.0523 accu_g_gdy: 0.0529 bleu: 90.5210

@ZhitingHu
Copy link
Member

let's merge, and make a PR to update to v0.2.4

@gpengzhi gpengzhi merged commit 8553974 into asyml:master Nov 19, 2019
@gpengzhi gpengzhi deleted the seq2seq_attn branch February 12, 2020 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible bug in seq2seq_attn
2 participants