A bug when applying MultiRNNCell?

[BruceChen034020]

---

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): This code is very similar to an official example
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
• TensorFlow installed from (source or binary): pip install tensorflow
• TensorFlow version (use command below): b'unknown' 1.4.0
• Python version: Python 3.5.2 :: Anaconda 4.2.0 (64-bit)
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:
• Exact command to reproduce:

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

tf.nn.MultiRNNCell sometimes doesn't work.

It raises an issue like this:
ValueError: Dimensions must be equal, but are 64 and 96 for 'lstm/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/basic_lstm_cell/MatMul_1' (op: 'MatMul') with input shapes: [128,64], [96,128].

Source code / logs

import tensorflow as tf
import numpy as np

hidden_layer_size = 32
embed = tf.zeros((128, 6, 64), dtype=tf.float32)

num_LSTM_layers = 2
with tf.variable_scope("lstm"):

lstm_cell = tf.contrib.rnn.BasicLSTMCell(hidden_layer_size, forget_bias=1.0)
cell = tf.contrib.rnn.MultiRNNCell(cells=[lstm_cell]*num_LSTM_layers, state_is_tuple=True)
outputs, states = tf.nn.dynamic_rnn(cell, embed, dtype=tf.float32)

Error:
ValueError: Dimensions must be equal, but are 64 and 96 for 'lstm/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/basic_lstm_cell/MatMul_1' (op: 'MatMul') with input shapes: [128,64], [96,128].

[peytonhong]

I had the same problem and found how to solve it.
You may change the highlighted code as below.

def lstm_cell():
   lstm = tf.contrib.rnn.BasicLSTMCell(hidden_layer_size, forget_bias=1.0)
   return lstm
cell = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(num_LSTM_layers)])
outputs, states = tf.nn.dynamic_rnn(cell, embed, dtype=tf.float32)

[facaiy]

As said by @peytonhong ,[lstm_cell] * num_LSTM_layers all refer to the same object lstm_cell. The multiplication operator on list doesn't copy its elements. So it's not a bug, right?

[peytonhong]

No, I don't think it is a bug. The problem was came from a misunderstanding about the structure of TensorFlow.

[renewang]

Hi, I encountered the same problem. After searching issues, I found many users have the same difficulties to construct MultiCellRNN correctly. So I did some code trace and explained what I understood from code in this Stackflow post https://stackoverflow.com/questions/48865554/using-dynamic-rnn-with-multirnn-gives-error/49066981#49066981
I hope this might help future users have the same problem with more clear understanding and also would like invite developers to correct me if there are something wrong in my explanation. Thanks!