Tensorflow Implementation of Nested LSTM Cell
Here is a tensorflow implementation of Nested LSTM cell.
|Nested LSTM Architecture. Courtesy of Moniz et al.|
NLSTM cell is basically a LSTM-like cell that uses the cell memory to control the state of the inner LSTM, and as such, the architecture can be generalized to multiple layers. For a comparison between LSTM and NLSTM,
|LSTM and stacked LSTM, versus nested LSTM. Courtesy of Moniz et al.|
The implementation here is compatible with the tensorflow rnn API.
from rnn_cell import NLSTMCell cell = NLSTMCell(num_units=3, depth=2) init_state = cell.zero_state(batch_size, dtype=tf.float32) output, new_state = cell(inputs, state=init_state) ...
- Moniz et al, "Nested LSTMs." https://arxiv.org/abs/1801.10308