2.0 Reference Models: NMT Model (TPU with dist strat and Keras)

[dynamicwebpaige]

Sequence-to-sequence (seq2seq) models (Sutskever et al., 2014, Cho et al., 2014) have enjoyed great success in a variety of tasks such as machine translation, speech recognition, and text summarization.

This model will give TF 2.0 end users a full understanding of seq2seq models and show them how to build a competitive seq2seq model from scratch. We focus on the task of Neural Machine Translation (NMT), which was the very first testbed for seq2seq models.

For the TensorFlow 1.x equivalent, please refer here.

The purpose of this issue is to migrate models into the TF 2.0 Keras API. Each migrated model must be eager and distribution compatible, with tests, and all associated engineering artifacts.

[JACKHAHA363]

Any updates? I find that the keras high level API is not flexible enough compared to the old contrib.seq2seq.

[npuichigo]

Any updates? +1

[rachellj218]

We are revamping the official model garden: more details can be viewed in this RFC tensorflow/community#130. We focus on supporting BERT and its variants for NLP tasks this quarter. We'd like to have NMT but don't have cycles now. Will try it include it in Q4.

[gadagashwini]

Tensorflow v2.x supports Sequence-to-Sequence NMT with Attention Mechanism, for more information refer these links ref1 and ref2. Thank you!

[mohantym]

Hi @dynamicwebpaige !
We are checking to see whether you still need help in this issue .

I think above add on documentation are still not compatible with tpu /dist-strat/eager mode.

Tested with nmt_with_attention from Tensorflow/text.
Results are not convincing in 2.9 too.

Attached gist for reference.

Thank you!

[tilakrayal]

Hi,

Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.

The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.

Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.

[github-actions]

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.