Error during training

[walidar]

Thank you for sharing the code.. during training I got the following error.. do you know the reason or how to fix it?

Epoch 0 Update 5000 Cost 118.37625885 NaN_in_grad 0 NaN_in_cost 0 Gradient_clipped 5000 UD 1436.17706203 33.74 sentence/s

Source 0 : I a m h a p p y I m a d e t h i s d e c i s i o n .
Truth 0 : Traceback (most recent call last):
File "/lium/buster1/aransa/workspace/dl4mt-c2c/bpe2char/train_bi_bpe2char.py", line 145, in
main(0, args)
File "/lium/buster1/aransa/workspace/dl4mt-c2c/bpe2char/train_bi_bpe2char.py", line 82, in main
gen_sample=gen_sample,
File "/lium/buster1/aransa/workspace/dl4mt-c2c/bpe2char/nmt.py", line 459, in train
print "".join(truth_)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 24: ordinal not in range(128)

[jaseleephd]

Hi @walidar, what's your target language? If you're training a model for EN-X rather than X-EN (as was the original setup), you will have to switch the source & target vocabularies accordingly (for example, https://github.com/nyu-dl/dl4mt-c2c/blob/master/bpe2char/train_bi_bpe2char.py#L22, https://github.com/nyu-dl/dl4mt-c2c/blob/master/bpe2char/train_bi_bpe2char.py#L23). Take a look at wmt_path.py and make sure you're using the right source & target vocabularies.

Let me know if this helped!

[jaseleephd]

Also, if you're using your own corpora to train your models (and not WMT'15 datasets), you first have to learn your dictionaries (& BPE rules for bpe2char models) using Rico Sennrich's Subword-NMT repo and then executing build_dictionary_word.py

[jaseleephd]

Hi @walidar, have you managed to resolve this?