-
Notifications
You must be signed in to change notification settings - Fork 229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About LSTM Layer in GroundHog #23
Comments
@kyunghyuncho , you wrote the LSTM layer. |
Sorry about the late reply! As @gaoyuankidult correctly noticed, the implementation in GoundHog lacks the bias terms for the gaters, which I believe wouldn't make much difference. Also, note that there are a number of variants of LSTMs (see, e.g., http://arxiv.org/pdf/1409.2329.pdf). You can train a neural machine translation model using LSTM by
Though, this feature has been tested only internally quite some time ago. If you run into any issues with this, please, leave a comment here about the details and I'll take a look into it. |
Hello @kyunghyuncho When I am trying to train a model using LSTM, I run into this error:
Do you have any suggestions? Thanks! |
Can you provide your state variables? On Thu, Jan 29, 2015 at 1:36 PM, infinitezxc notifications@github.com
|
Thanks for the quick reply :) @kyunghyuncho I am using the default states in |
@infinitezxc Have you solved the LSTM error? I met the error too. |
Hello
I have checked the LSTM Layer in GroundHog / groundhog / layers / rec_layers.py. I wonder is this a complete standard LSTM layer(e.g. described in Alex Grave's paper, http://arxiv.org/pdf/1308.0850v5.pdf ) or a prototype for now?
I didn't see bias term in # input/output gate update. did I miss it? btw. do you have an example using this layer?
thanks.
btw. I can write a wiki(tutorial) about it, If there is some example.
formulas described in paper:
code of LSTM layer:
The text was updated successfully, but these errors were encountered: