Interaction between tf.map_fn and tf.gradients

[daniellevy]

Hi,

I am using Tensorflow v0.11 and I have tried on Mac OS X and Centos 6

I am running into an error when running the following code:

W = tf.get_variable('W', (5, 3))

x = tf.placeholder(tf.float32, shape=(None, 5))

h = tf.matmul(x, W)

grads = tf.map_fn(lambda x: tf.gradients(x, W)[0], h)

I basically want to have the following but without a fixed batch size:
grads = [tf.gradients(h[t], W)[0] for t in range(batch_size)]

My error is:

Invalid argument: TensorArray map/TensorArray_1@map/while/gradients: Could not write to TensorArray index 3 because it has already been read.
[...]
tensorflow.python.framework.errors.InvalidArgumentError: TensorArray map/TensorArray_1@map/while/gradients: Could not write to TensorArray index 3 because it has already been read.
	 [[Node: map/while/gradients/map/while/TensorArrayRead_grad/TensorArrayWrite = TensorArrayWrite[T=DT_FLOAT, _class=["loc:@map/TensorArray"], _device="/job:localhost/replica:0/task:0/cpu:0"](map/while/gradients/map/while/TensorArrayRead_grad/TensorArrayGrad/TensorArrayGrad, map/while/Identity, map/while/gradients/Fill, map/while/gradients/map/while/TensorArrayRead_grad/TensorArrayGrad/gradient_flow)]]

I have tried the following workaround using scan instead of map_fn with a zero initializer but to no avail:

initializer = np.zeros((5, 3)).astype('float32')
grads = tf.scan(
	lambda a, x: tf.gradients(x, W)[0],
	h,
	initializer)

Is this a know issue?

[aselle]

@yuanbyu, do you have any ideas? @daniellevy , could you try 1.0, please and see if this is still a problem.

[daniellevy]

Yes the problem still occurs with v1.0 with the same error message.

[aselle]

It looks like this is a known issue and this issue is probably a duplicate of #3972. There @yuanbyu suggests using tf.while_loop instead of tf.map_fn. Give that a try and let us know if that works. Thanks!

[dillonalaird]

I think this works

max_seq_len = 10
x = tf.placeholder(tf.float32, [None, 5])
W = tf.get_variable("W", [5, 3])
h = tf.matmul(x, W)

def body(old_g, t):
    g = tf.gradients([h[t]], [W])[0]
    new_g = tuple(tf.select(tf.equal(ti, t), g, old_g[ti]) for ti in range(len(old_g)))
    return new_g, t + 1

def cond(_, t):
    return tf.less(t, tf.shape(h)[0])

grads = tf.while_loop(cond, body, [(tf.zeros_like(W),)*max_seq_len, tf.constant(0)])

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    grads_out = sess.run(grads, feed_dict={x: np.random.randn(2*5).reshape(2, 5)})

You basically just build a giant empty tuple and fill it in as you go. dynmaic_rnn does something similar to this.

[aselle]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!