Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce
Mobile device

According to its API

t * clip_norm / l2norm(t)

The gradient seems t / t * clip_norm, right ? I think the result is undefined when t == 0.

@alextp Could you please comment or reassign?

No, this rescaling is only valid "If the L2-norm is greater than clip_norm" (see API), otherwise t should remain unchanged, so gradient should be all ones.
"if the L2-norm of t is already less than or equal to clip_norm, then t is not modified."

I think you're right.

This is very similar with #20091 and it's essentially caused by the fact that the implementation of clip_by_norm requires computing the gradient w.r.t. tf.norm(t) even when |t| < clip_norm. I am not sure how this can be solved since TF uses static graphs. Would have been trivial in a framework that uses dynamic graphs ...

In [20]: a = tf.constant(1.0)
In [21]: b = tf.constant(2.0)
In [22]: c = tf.maximum(a, b)
In [23]: sess.run(tf.gradients(c, [a, b]))
Out[23]: [0.0, 1.0]

Because tf.maximum cannot block the gradient propagation for the smaller value (note that gradient w..rt. a is 0.0, rather than None), I have no idea of how to solve the problem. Let's wait for reply from @alextp .

Yes, and similar:

a = tf.zeros([3], dtype=tf.float32)
b = tf.maximum(5., tf.norm(a))
c = tf.gradients(b, [a])
s = tf.Session()
s.run(c)

Gives
[array([nan, nan, nan], dtype=float32)]

I am not sure this solves the issue. The code still fails for very small vectors, e.g.:
a = tf.ones([3], dtype=tf.float32) * 1e-20

Yes, I think it's called "dynamic graphs" :)

I think the root is that tensorflow, in graph mode, cannot block the gradient backpropagation of a slice of Tensor, say tf.maximum op here. NaN propagation is really annoying, and two "where" trick hurts performance. Is there any solution to solve the problem totally?

It is especially annoying since it is usually very hard to debug and doesn't have a "mathematical" cause. It would be nice to have a tool that would help TF users to understand when this issue is generating
their "NaN problem".

Any updates or suggestions here? I am running into this and do not know how to solve it.