TensorFlow eager execution API give a wrong answer for some function

[yt7589]

Why TensorFlow eager execution API give a wrong answer for this function?

def dp1_f1(x):
    return 64*x*(1-x)*(math.pow((1-2*x),2) )*math.pow((1-8*x+8*x*x), 2)

I want to get dy/dx value. I can get this value by numeric method just as below:

def dp_numeric_diff(x):
    delta_x = 0.0001
    return (dp1_f1(x+delta_x)-dp1_f1(x))/delta_x

I use TensorFlow eager execution API to calculate this value:

def dp_ad_tfe(x):
    tf.enable_eager_execution()
    tfe = tf.contrib.eager
    grad_lx = tfe.gradients_function(dp1_f1)
    x = 3.0
    y = dp1_f1(x)
    rst = grad_lx(x)
    return y, rst[0]

I call this function with code below:

numeric_diff = dp_numeric_diff(x)
print('Numeric method：{0}'.format(numeric_diff))
v, d = dp_ad_tfe(x)
print('TFE：{0}'.format(d))

It will display something like this:

Numeric method：-75290405.66440672
TFE：-19208000.0

I am sure that the numeric method is right. What's wrong with my TensorFlow eager execution code? By the way the same TensorFlow eager execution code can get correct answer for simple function like x^2.

[yt7589]

If I change math.pow function with this one then TensorFlow eager execution will give the right answer:

def f3(x, n):
    rst = 1
    for i in range(n):
        rst *= x
    return rst

It seems that TensorFlow eager execution API can't deal with math.pow function. I had tested tf.pow and numpy.power function. I found that TensorFlow eager execution API can't cope with this kind of functions as well.

[asimshankar]

tf.contrib.eager.gradients_function can only differentiate through TensorFlow operations.
So it can't differentiate through math.pow but it can through tf.pow. Could you elaborate on which version of TensorFlow you're using? I tried the following on TensorFlow 1.8 and it seems to be correct:

import tensorflow as tf
tf.enable_eager_execution()

def dp1_f1(x):
    return 64*x*(1-x)*(tf.pow((1-2*x),2) )*tf.pow((1-8*x+8*x*x), 2)

def dp_numeric_diff(x):
    delta_x = 0.0001
    return (dp1_f1(x+delta_x)-dp1_f1(x))/delta_x

grad = tf.contrib.eager.gradients_function(dp1_f1)

print(dp_numeric_diff(3.0).numpy()) # Prints -75300000.0
print(grad(3.0)[0].numpy())         # Prints -75279680.0

[yt7589]

Thanks! @asimshankar My TensorFlow version is 1.9rc0 CPU version. I had git clone the latest source code of TensorFlow and compiled the CPU version.

[asimshankar]

@yt7589 - do you see the same issue if you use the release binaries of 1.9.0-rc0? (Because I don't :)
If not, then perhaps something is fishy with the process you're using to build from source?

[yt7589]

@asimshankar I had not used binary release of 1.9.0-rc0. Because my CPU doesn't support AVX instruction set.

[tensorflowbutler]

Nagging Assignee @asimshankar: It has been 15 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

[asimshankar]

@yt7589 : Seems like the problem isn't reproducible on the release binaries of TensorFlow (last I tried was with the final 1.9.0 release).

So I'm at a bit of a loss without more detail on your setup. Thanks.

[tensorflowbutler]

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

[asimshankar]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!