-
-
Notifications
You must be signed in to change notification settings - Fork 676
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is this Reshape step redundant? #3
Comments
Also you don't need this line of code: you can pass name = 'name' in any layer |
@hamelsmu yes the Reshape layer is redundant and does not add any value to the model (Everything is done by the Permute layer). It's more to enforce the correct shape. The output of the Permute layer is (?, ?) and by adding this Reshape layer, we make it more clear about the real shapes (they are static and known at compilation time). So I wanted to reflect this idea of static shapes (vs dynamic shapes). |
Thanks for your feedback! Highly appreciated!
Is this what you meant? Removing the Else clause and adding |
Yeah thats right |
It would not work here because we define different layers with the same names twice.
|
@philipperemy right. However I suppose you can say that the attention layer is
|
Ok seems good for me! The only thing is that
|
Let me know if it seems good for you: PR: #4 |
Thanks! |
See this line of code: https://github.com/philipperemy/keras-attention-mechanism/blob/master/attention_lstm.py#L19
Isnt this redundant? Because the Permute layer right before it will reshape the Tensor.
Let me know if I'm missing something. I am trying to understand attention and thus far your writeup is helping
The text was updated successfully, but these errors were encountered: