tf.copy() as alternative to tf.identity()

[danijar]

tf.identity(tensor) does or does not create a copy of the tensor based on whether it's on the same device. This can lead to bugs that are hard to find. The current way of ensuring a copy is to perform an arithmetic/logic operation that doesn't change the value, such as tensor + 0, 1 * tensor, or tf.equal(tensor, True). Needless to say, this makes code hard to read. Moreover, different treatment is needed for different tensor types. Can we have a tf.copy(tensor) that does this for us?

[ppwwyyxx]

Just curious, could you elaborate what kind of bugs it could lead to?

[jyegerlehner]

The current way of ensuring a copy is to perform an arithmetic/logic operation that doesn't change the value

Couldn't we use tf.assign for that?

[danijar]

@ppwwyyxx For example, I was trying to feed the transition tuple of state, action, reward, and successor state to a reinforcement learning agent. Here, env.state is a variable holding the current state of the environment that gets updated when calling env.step(action):

# NOTE: old_state actually refers to the new state after stepping the env. Need env.state * 1 instead.
old_state = tf.identity(env.state)
with tf.control_dependency([old_state]):
    action = agent.act(env.state)
    reward = env.step(action)
with tf.control_dependency([reward]):
    experience_op = agent.experience(old_state, action, reward, env.state)

@jyegerlehner That would require creating a second variable to hold the old state of the variable. That's wasteful if the value is only needed inside each sess.run() call separately. Instead, the value should be hold by a tensor.

[ppwwyyxx]

I believe your issue can be described in this small repro:

ones = np.ones((10,10))
v = tf.get_variable('asdf', shape=[10,10], dtype=tf.float32)
old_state = v.read_value()   # equivalent to tf.identity
with tf.control_dependencies([old_state]):
    reward = tf.assign_add(v, ones)
with tf.Session() as sess:
    tf.global_variables_initializer().run()
    a, b = sess.run([reward, old_state])
    print(a.sum(), b.sum())

The above code, though appears to be deterministic (due to the control dependency), actually is not. It sometimes prints two equal results, sometimes not.

Using get_variable(..., use_resource=True) (will become default in the future) seems to fix the problem. Could you try this? @danijar

[jiagengliu]

I also wonder if we could presumably apply tf.copy to tf.Variable, tf.placeholder and so on.
I am referring to the situation described here: https://stackoverflow.com/questions/45947973/how-to-create-copies-of-variables-constants-placeholders-in-tensorflow

[thjashin]

@ppwwyyxx I have the same problem. I post some snippets in #4663 .

[ppwwyyxx]

@thjashin And my same solution seems applicable.

[thjashin]

@ppwwyyxx Yeah using ResourceVariable actually works but just wonder why this happens. It's pretty strange that control_dependencies doesn't work with plain Variable.

[Emerald01]

@danijar
So if I understand this issue correctly, does this mean that:
for identity:
var_identity = tf.identity(var)

if var and var_identity are in the same device, var_identity would be similar to a C++ reference to var (basically they are the same thing in the same memory, var_identity is just an alias for var), any change in var would be represented equally in var_identity. In this sense, I think implementation-wise, it is similar to that in get_variable() function set var_identity a reusable variable of var.

if var_identity is transferred to another device, which I think it is pretty typical when training multiple workers and they are shared by one param server, the var_indentity is essentially a copy of var. This makes sense, because, between different devices, nothing can be really shared but a copy would be the best solution.

[chuchienshu]

@danijar

import tensorflow as tf
# Create a variable.
w = tf.Variable(3.)
copyw = tf.identity(w)
w = w+1
# Use the variable in the graph like any Tensor.
y = tf.add(w, w)
# The overloaded operators are available too.
z = tf.sigmoid(w + y)
init_op = tf.global_variables_initializer()
# Launch the graph in a session.
with tf.Session() as sess:
    # Run the Op that initializes global variables.
    sess.run(init_op)
    w_, cw_ = sess.run([ w, copyw])
    print(w_, cw_)  # output 4.0 3.0

Based on above snippet, I think tf.identity() works as deep copy. as least in tensorflow 1.11.0.

[dynamicwebpaige]

As demonstrated by @chuchienshu above, tf.identity() now works as deep copy. Closing this issue.

[kartynnik]

@dynamicwebpaige This issue was not about whether or not tf.identity was working, but rather about creating a tf.copy alias for this behavior that would clearly reflect the intent of a deep copy. So closing it seems to be premature.

[htanwar922]

tf.identity raises errors when GPU is involved. Using tf.device('CPU') will prevent the error:

with tf.device('CPU'):
    tensor2 = tf.identity(tensor1)         # .take(-1)

Remember tensor2 is a Variant. You may use take(-1) in case tensor1 is a Dataset and you want tensor2 to be one of the Dataset family only.
Actually the error is generated for the Datasets in first place.