Are RNNs thread-safe?

[FilippoC]

Hi,

as far as I understand, we are allowed to create one computational graph per thread (or we can use the dynet multi-processing API).

Of course, parameters are shared (but not expressions).
However, the RNN API, using builder.new_graph(cg), does not seems thread-safe (ie. states are shared between threads).

In the example provided:

1. this function is called for each thread https://github.com/clab/dynet/blob/master/examples/multiprocessing/rnnlm.h#L35
2. one expression is created in each thread for each parameter, for example : https://github.com/clab/dynet/blob/master/examples/multiprocessing/rnnlm.h#L39
3. however, the RNN internal expressions are shared https://github.com/clab/dynet/blob/master/examples/multiprocessing/rnnlm.h#L37

[neubig]

DyNet is not threadsafe yet, but this PR is aiming to make it so: #1064

Once this is merged, we can examine this issue further.