tensorflow_rnn_cell.txt

# Tensorflow LSTM cell requires 2x n_hidden length (state & cell)
init_state = tf.placeholder("float", [None, 2*lstm_size])


https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/rnn_cell.py
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/rnn.py



//=== https://www.tensorflow.org/versions/r0.9/api_docs/python/rnn_cell.html
Neural Network RNN Cells
Base interface for all RNN Cells
class tf.nn.rnn_cell.RNNCell
RNN Cells for use with TensorFlow's core RNN methods
class tf.nn.rnn_cell.BasicRNNCell
class tf.nn.rnn_cell.BasicLSTMCell
class tf.nn.rnn_cell.GRUCell
class tf.nn.rnn_cell.LSTMCell
Classes storing split RNNCell state
class tf.nn.rnn_cell.LSTMStateTuple
RNN Cell wrappers (RNNCells that wrap other RNNCells)
class tf.nn.rnn_cell.MultiRNNCell
class tf.nn.rnn_cell.DropoutWrapper
class tf.nn.rnn_cell.EmbeddingWrapper
class tf.nn.rnn_cell.InputProjectionWrapper
class tf.nn.rnn_cell.OutputProjectionWrapper
Module for constructing RNN Cells.

//=== https://www.tensorflow.org/versions/r0.10/api_docs/python/rnn_cell.html
Neural Network RNN Cells
Base interface for all RNN Cells
class tf.nn.rnn_cell.RNNCell
RNN Cells for use with TensorFlow's core RNN methods
class tf.nn.rnn_cell.BasicRNNCell
class tf.nn.rnn_cell.BasicLSTMCell
class tf.nn.rnn_cell.GRUCell
class tf.nn.rnn_cell.LSTMCell
Classes storing split RNNCell state
class tf.nn.rnn_cell.LSTMStateTuple
RNN Cell wrappers (RNNCells that wrap other RNNCells)
class tf.nn.rnn_cell.MultiRNNCell
class tf.nn.rnn_cell.DropoutWrapper
class tf.nn.rnn_cell.EmbeddingWrapper
class tf.nn.rnn_cell.InputProjectionWrapper
class tf.nn.rnn_cell.OutputProjectionWrapper


//===
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/rnn_cell.py
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/api_docs/python/functions_and_classes/shard7/tf.nn.rnn_cell.BasicLSTMCell.md

//=== Basic LSTM recurrent network cell.

The implementation is based on: http://arxiv.org/abs/1409.2329.

We add forget_bias (default: 1) to the biases of the forget gate in order to reduce the scale of forgetting in the beginning of the training.

It does not allow cell clipping, a projection layer, and does not use peep-hole connections: it is the basic baseline.

For advanced models, please use the full LSTMCell that follows.

tf.nn.rnn_cell.BasicLSTMCell.__call__(inputs, state, scope=None) {#BasicLSTMCell.call}

Long short-term memory cell (LSTM).

tf.nn.rnn_cell.BasicLSTMCell.__init__(num_units, forget_bias=1.0, input_size=None, state_is_tuple=True, activation=tanh) {#BasicLSTMCell.init}

Initialize the basic LSTM cell.

Args:

num_units: int, The number of units in the LSTM cell.
forget_bias: float, The bias added to forget gates (see above).
input_size: Deprecated and unused.
state_is_tuple: If True, accepted and returned states are 2-tuples of the c_state and m_state. If False, they are concatenated along the column axis. The latter behavior will soon be deprecated.
activation: Activation function of the inner states.
tf.nn.rnn_cell.BasicLSTMCell.output_size {#BasicLSTMCell.output_size}

tf.nn.rnn_cell.BasicLSTMCell.state_size {#BasicLSTMCell.state_size}

tf.nn.rnn_cell.BasicLSTMCell.zero_state(batch_size, dtype) {#BasicLSTMCell.zero_state}

Return zero-filled state tensor(s).

Args:

batch_size: int, float, or unit Tensor representing the batch size.
dtype: the data type to use for the state.
Returns:

If state_size is an int or TensorShape, then the return value is a N-D tensor of shape [batch_size x state_size] filled with zeros.

If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size x s] for each s in state_size.



//=== https://www.tensorflow.org/versions/r0.10/api_docs/python/nn.html#rnn

tf.nn.rnn(cell, inputs, initial_state=None, dtype=None, sequence_length=None, scope=None)

Creates a recurrent neural network specified by RNNCell cell.

The simplest form of RNN network generated is: py state = cell.zero_state(...) outputs = [] for input_ in inputs: output, state = cell(input_, state) outputs.append(output) return (outputs, state) However, a few other options are available:

An initial state can be provided. If the sequence_length vector is provided, dynamic calculation is performed. This method of calculation does not compute the RNN steps past the maximum sequence length of the minibatch (thus saving computational time), and properly propagates the state at an example's sequence length to the final state output.

The dynamic calculation performed is, at time t for batch row b, (output, state)(b, t) = (t >= sequence_length(b)) ? (zeros(cell.output_size), states(b, sequence_length(b) - 1)) : cell(input(b, t), state(b, t - 1))

Args:

cell: An instance of RNNCell.
inputs: A length T list of inputs, each a Tensor of shape [batch_size, input_size], or a nested tuple of such elements.
initial_state: (optional) An initial state for the RNN. If cell.state_size is an integer, this must be a Tensor of appropriate type and shape [batch_size, cell.state_size]. If cell.state_size is a tuple, this should be a tuple of tensors having shapes [batch_size, s] for s in cell.state_size.
dtype: (optional) The data type for the initial state and expected output. Required if initial_state is not provided or RNN state has a heterogeneous dtype.
sequence_length: Specifies the length of each sequence in inputs. An int32 or int64 vector (tensor) size [batch_size], values in [0, T).
scope: VariableScope for the created subgraph; defaults to "RNN".

Returns:
A pair (outputs, state) where: 
- outputs is a length T list of outputs (one for each input), 
  or a nested tuple of such elements. 
- state is the final state