tiny weights after tuning

[glample]

Hi,

When I tune a Moses model, at the end I sometimes get tiny weights in the final moses.ini. For instance:

[weight]

LexicalReordering0= 1 1.99141e-39 1.59568e-39 1.63642e-39 2.05499e-39 2.16443e-39
Distortion0= 2.88571e-39
LM0= 3.74528e-39
WordPenalty0= 2.5908e-39
PhrasePenalty0= 1.57735e-39
TranslationModel0= 3.49715e-39 6.27445e-40
UnknownWordPenalty0= 1

At the top of the file, I also see BLEU 0.22197 which means that the model has actually learned something. The very tiny weights might be okay if they were all renormalized at test time, however, when I use that moses.ini file to translate something, the model totally fails.

The issue occurs quite randomly, sometimes it works (weights look normal, uniform between 0 and 1), sometimes I obtain what I have above, and the model is not usable.

Any ideas what might be the issue? Note that I disabled the reordering model (by creating an empty reordering-table.wbe-msd-bidirectional-fe.gz file), and I don't know if this can be related to the problem I have (even though I think this should be fine, as the tuning is still working once in a while).

Thanks!

[hieuhoang]

the weights looks wrong tbh, basically all zeros except 1 for the unknown word penalty.

1. if you want to disable the reordering model, just delete the lines in the [feature] and [weight] sections before tuning. I don't know the values the model give for UNK, it may not be 'neutral'.
2. Check that your tuning set is correct, and the output is as expected. eg. did you accidently swap source and target files.

[hieuhoang]

Closing. Reopen if you still have problems