Hope intermediate state in dynamic_rnn

[Sraw]

To implement a attention machine, I need the intermediate state to calculate the attention, but dynamic_rnn doesn't return the intermediate states but only final state.
Hope the dynamic_rnn could return intermediate states so that we can implement a attention machine with dynamic network.

[ebrevdo]

You can write a variant of the LSTMCell that returns both state tensors as part of the output, if you need both c and h state for each time step. If you just need the h state, that's the output of reach time step.

[Sraw]

@ebrevdo Thanks for reply. Really helpful.

[WolfNiu]

@ebrevdo May I ask a question? For each time step of LSTM there is a memory cell value c, a hidden state h, and an output y. So what you were suggesting is that the outputs returned by dynamic RNN contain h rather than y for each step? Would that also mean the returned output_states is actually outputs[-1]? Thank you.

[Sraw]

@WolfNiu I can answer your question.
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
if you look into the structure of LSTM, you will see that the hidden state h and the output y is the same, they all come from the same line.
So actually, there are only two values at each step, h or y and c.

[evanthebouncy]

@ebrevdo

I tried your suggestion but I ran into such trouble doing it, can you take a look at my stackoverflow post?

https://stackoverflow.com/questions/45528146/tensorflow-create-a-custom-sub-class-of-lstm-cell-with-a-dfferent-call-functi

Thanks

[ghost]

@Sraw am I able to assign specific values for lstm gates? for instance always 1 for input and....