Adding @yuanbyu@, @ebrevdo.

We don't support taking gradients of nonscalars, so I wouldn't expect this to work. However, this particular error message is pretty confusing. @yuanbyu: Is there a way we could improve this error message if someone tries for a Hessian in the naive way and control flow is involved?

Yes, the error message should be better. Let me see what I can do.

@girving Here theta is just a scalar value though. What would be the work around here?

@dementrock: That's true in this case, but we don't want to do extra work on the control flow ops if all it provides is higher order derivatives w.r.t. scalars. What is your intended use case?

@girving I was doing some hessian vector product computations and had the same error, but the code snippet above is simpler and highlights the issue.

So is there no way to get higher order derivatives w.r.t. scan right now?

@dementrock: We don't support higher order gradients even ignoring control flow.

@girving No support as in no official support or it won't work at all? Seems like the following code at least compiles:

import tensorflow as tf

theta = tf.Variable(initial_value=1.)


def fn(x, prev):
    return prev + x * theta

result = fn(fn(1.0, 2.0), 3.0)

grad_theta = tf.gradients(result, theta)

tf.gradients(grad_theta, theta)

Any plan to support it in the future?

@dementrock The problem is that the registered gradient routines would get significantly more complicated if both sides were nonscalar, and we don't want to support that kind of complexity. As discussed in #675, it's possible one could implement registered gradients with some sort of automatic machinery to map scalar gradient routines to nonscalar gradient routines, but this is a lot of work and we don't have any plans to do it.

The other problem is that the applications I know of nonscalar gradients aren't that compelling as yet, since they tend to be impractically huge. However, there are cases where higher order gradient information arises where you're differentiating a scalar, specifically Hessian-free Krylov-ish methods where one evaluates the gradient dotted with a suitably chosen vector.

If that last bit is what you're trying to do, or something similar, we'd be happy to accept pull requests to make control flow not interfere. It might be pretty complicated, though.