Problems with the next word predictions

[ishpiki]

Hi,
I have one issue related to the next word prediction, because by given word and previous hidden states we could try to predict the next most probable word according to softmax probability distribution. The same problem was discussed here #96.

When I did it with the trained model (default settings, wiki-2 dataset) the result was not so good:

original: an English film , television and theatre actor . He had a guest @-@ starring role on the television series The Bill in 2000 . This was followed by a starring role in the play Herons written by Simon Stephens , which was performed in 2001 at the Royal Court Theatre . He had a guest role in the television
predicted: @-@ , and , . , also a appearance seller in the series , and the , was the by a in in the film , . by . who was released by the . the Academy of in was previously appearance in the film series

When I used another frameworks for language modelling (e.g. TensorFlow, Keras) and the same function to predict the next words, it gives me more meaningful results.

Maybe I should modify your code in some way for this task?

Thanks.