translation with language models

[VictorChen2012]

I'm wondering if it is possible to combine an LM with a seq2seq model under OpenNMT-tf, e.g. shallow fusion, deep fusion or cold fusion.

Currently, vars and ops of LM decoder and seq2seq decoder are in different name scope. It's too complicated to directly load and merge two pretrained models, i.e. LM and the seq2seq model under the same name scope.

Any suggestions to the goal above?

[guillaumekln]

Shallow fusion should be the most accessible but it may not be easy to integrate at this time. However, there are some incoming changes that should facilitate such combinations.

I'm interested in supporting shallow fusion in the near future.

[VictorChen2012]

Shallow fusion should be the most accessible but it may not be easy to integrate at this time. However, there are some incoming changes that should facilitate such combinations.

I'm interested in supporting shallow fusion in the near future.

Thanks for your quick reply! I'll see if I can contribute then.

[lkluo]

Any update?

[guillaumekln]

No one is currently working on this as far as I know.