Jacobian 0 when non-differentiable

[svigerske]

I have a x^0.3 * y^0.7 and try to differentiate at 0. I would expect some NaN or Inf in the Jacobian or some error, but I get a (0,0):

#include <cstdio>
#include "cppad/cppad.hpp"

using namespace std;

template <class Type>
void eval(const vector<Type>& x, vector<Type>& y)
{
   y[0] = pow(x[0], 0.3) * pow(x[1], 0.7);
}

int main(int argc, char** argv)
{
   vector< double> x(2);
   vector< CppAD::AD<double> > X(2);
   vector< CppAD::AD<double> > Y(1);
   
   X[0] = x[0] = 0.0;
   X[1] = x[1] = 0.0;
   
   CppAD::Independent(X);

   eval(X, Y);

   CppAD::ADFun<double> f;
   f.Dependent(X, Y);

   printf("val in 0: %g\n", CppAD::Value(Y[0]));

   vector<double> jac = f.Jacobian(x);
   
   printf("grad in 0: %g %g\n", jac[0], jac[1]);

   return 0;
}

[bradbell]

I have been able to reproduce the problem; see
https://github.com/coin-or/CppAD/blob/master/bug/pow.sh

Thanks.

[bradbell]

I believe the change to include/cppad/local/pow_op.hpp in
2196bce#diff-bc3c06208a0f4bedde79191e991128d9
fixes this problem.

Please test it and if you agree, close this bug.

[svigerske]

This this seems to work for a simple pow(x,0.3).
But if I multiply two pow as in my example above, I still get a (0,0) and no error.
Adapted pow.sh:

#! /bin/bash -e
# vim: set expandtab:
# -----------------------------------------------------------------------------
# CppAD: C++ Algorithmic Differentiation: Copyright (C) 2003-18 Bradley M. Bell
#
# CppAD is distributed under the terms of the
#              Eclipse Public License Version 2.0.
#
# This Source Code may also be made available under the following
# Secondary License when the conditions for such availability set forth
# in the Eclipse Public License, Version 2.0 are satisfied:
#       GNU General Public License, Version 2.0 or later.
# -----------------------------------------------------------------------------
name=`echo $0 | sed -e 's|^bug/||' -e 's|\.sh$||'`
if [ "$0" != "bug/$name.sh" ]
then
    echo 'usage: bug/pow.sh'
    exit 1
fi
# -----------------------------------------------------------------------------
if [ -e build/bug ]
then
    rm -r build/bug
fi
mkdir -p build/bug
cd build/bug
# cmake ../..
# -----------------------------------------------------------------------------
cat << EOF
Issue 43:
f(x) = x_0^0.5 and differentiate at x = (0,0).
Expect some NaN or Inf in the Jacobian or some error,
but get a jac = {0, 0}.
EOF
cat << EOF > $name.cpp
# include <cstdio>
# include "cppad/cppad.hpp"


int main(int argc, char** argv)
{   bool ok = true;

    using std::cout;
    using CppAD::AD;
    using CppAD::vector;
    //
    vector< double> x(2), y(1), w(1), dw(2);
    vector< AD<double> > ax(2), ay(1);
    //
    ax[0] = 0.0;
    ax[1] = 0.0;
    //
    CppAD::Independent(ax);
    ay[0] = pow(ax[0], 0.5) * pow(ax[1], 0.5);
    CppAD::ADFun<double> f(ax, ay);
    //
    x[0]  = 0.0;
    x[1]  = 0.0;
    y     = f.Forward(0, x);
    w[0]  = 1.0;
    dw    = f.Reverse(1, w);
    //
    cout << "dw[0] = " << dw << "\n";
    //
    ok &= y[0] == 0.0;
    ok &= ! std::isfinite( dw[0] );
    ok &= ! std::isfinite( dw[1] );
    //
    if( ! ok )
        return 1;
    return 0;
}
EOF
cxx_flags='-Wall -pedantic-errors -std=c++11 -Wshadow -Wconversion -g -O0'
eigen_dir="$HOME/prefix/eigen/include"
echo "g++ -I../../include -isystem $eigen_dir $cxx_flags $name.cpp -o $name"
g++ -I../../include -isystem $eigen_dir $cxx_flags $name.cpp -o $name
#
echo "build/bug/$name"
if ! ./$name
then
    echo
    echo "build/bug/$name: Error"
    exit 1
fi
echo
# -----------------------------------------------------------------------------
echo "bug/$name.sh: OK"
exit 0

[bradbell]

Now we have a real problem, in the multiplication
pow(ax[0], 0.5) * pow(ax[1], 0.5);
the left term is zero and the right term is zero

Here is the issue, during reverse mode, zero for a partial derivative has a speical meaning of variable selection, instead of the normal meaning of just zero. This enables computing partial derivatives where the partial w.r.t one viable is a number and w.r.t. another variable is a nan. The problem is the double use of zero both for selection and for the value zero; see
http://coin-or.github.io/CppAD/doc/azmul.htm

What is really needed here is a different floating point value that is equal to absolute zero and different from normal zero (so that one does not need an extra boolean for every floating point value).
I am going to have to think about this. In the meantime, keep this issue open.

[bradbell]

I have committed a modified version of your test that also gives an unexpected zero for the derivative of
f(x) = pow(x, .5) * pow(x, .5) = x
In addition, I have fixed the higher order version of the previous change and added more comments about what is going on here; see
75fca6f

[bradbell]

I am moving this issue to discussion #83