Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request for examples: Seq2seq #335

Open
seanpmorgan opened this issue Jul 6, 2019 · 8 comments

Comments

@seanpmorgan
Copy link
Member

@seanpmorgan seanpmorgan commented Jul 6, 2019

I believe there are many seq2seq examples already published, but it makes sense that we at least have one or two in our examples section for user experience.

Related:
https://google.github.io/seq2seq/
https://github.com/tensorflow/addons/blob/master/examples/README.md

@kazemnejad

This comment has been minimized.

Copy link
Contributor

@kazemnejad kazemnejad commented Jul 20, 2019

I can pick this up if it is ok.

Some ideas to start the example:

  • For machine translation:
    • On toy English-to-German dataset (I think German-to-English has better user experience since we can demonstrate the outputs better) from http://opennmt.net/OpenNMT-tf/quickstart.html which contains 10k tokenized sentences.
    • Trivial seq2seq LSTM architecture with Attention Mechanism
    • Use beam decoding to generate outputs
    • Maybe BLEU calculation
@seanpmorgan

This comment has been minimized.

Copy link
Member Author

@seanpmorgan seanpmorgan commented Jul 20, 2019

I can pick this up if it is ok.

Some ideas to start the example:

  • For machine translation:

    • On toy English-to-German dataset (I think German-to-English has better user experience since we can demonstrate the outputs better) from http://opennmt.net/OpenNMT-tf/quickstart.html which contains 10k tokenized sentences.
    • Trivial seq2seq LSTM architecture with Attention Mechanism
    • Use beam decoding to generate outputs
    • Maybe BLEU calculation

That'd be great! Thanks and welcome to Addons!

@PyExtreme

This comment has been minimized.

Copy link
Contributor

@PyExtreme PyExtreme commented Sep 14, 2019

Hi @seanpmorgan @kazemnejad , Is it still an Open Issue?
Asking because I am interested to work on it.

@kazemnejad

This comment has been minimized.

Copy link
Contributor

@kazemnejad kazemnejad commented Sep 14, 2019

Hi @PyExtreme, Thank you for showing your interest.
Yes, actually I'm currently working on it. I was waiting for the #375 to be fixed and it got fixed 3 days ago in #503. I think I can submit a PR in the next few days. Thus you can work on that PR if you want.

@matthen

This comment has been minimized.

Copy link

@matthen matthen commented Nov 4, 2019

+1 any progress on this?
I'd appreciate an example of how to create and train a keras decoder with attention. I can't figure out how I am supposed to set up AttentionWrapper in a model without yet knowing the memory tensor

@kazemnejad

This comment has been minimized.

Copy link
Contributor

@kazemnejad kazemnejad commented Nov 4, 2019

Hi @Mainak431, Sorry for the inconvenience.

Actually, the draft of this example is present at my fork. However, in the default Keras training mode (graph mode) + tf.data.dataset (where the default mode is eager), there is a bug related to caching of tensor dimension which I guess is from the AttentionWrapper. Unfortunately, I'm busy with my university's works at this moment and I could not work on that bug, so I would be happy if someone could find the source of that bug and help the progress.

@matthen

This comment has been minimized.

Copy link

@matthen matthen commented Nov 4, 2019

thanks for sharing @kazemnejad! Where do you get the bug? I was getting an error at some point from the rnn cell being wrapped by AttentionWrapper that it was being passed a rank 1 tensor when it was expecting a rank 2

@guillaumekln guillaumekln added the seq2seq label Nov 5, 2019
@zhedongzheng

This comment has been minimized.

Copy link

@zhedongzheng zhedongzheng commented Nov 13, 2019

Thanks @kazemnejad for submitting the fix. Based on his contribution, I have successfully get the seq2seq code running for a semantic parsing task.

For anyone who is interested, the notebook is here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.