How to compute partial derivative?

[jerryli1981]

Hi All,
I need to compute partial derivative for function like below,

def func(x,y):
return np.abs (x - y)

I need the derivative of x, the derivate of y.
Thanks, Any suggestions will help

[jerryli1981]

By the way, both x and y are vectors with float values

[brunojacobs]

Note that your function func(x,y) returns a vector instead of a scalar. Therefore, you need to use the jacobian function in autograd and not grad. Example code:

import autograd.numpy as np
from autograd import jacobian

def f(x, y):
    return np.abs(x - y)

jacobian_f_wrt_x = jacobian(f, 0) # 0 indicates first input element in f(x, y)
jacobian_f_wrt_y = jacobian(f, 1) # 1 indicates second input element in f(x, y)

x = np.arange(-4, 6, 2, dtype=float)
y = np.zeros(5) + 1.5

print(jacobian_f_wrt_x(x, y))
print(jacobian_f_wrt_y(x, y))

Here jacobian_f_wrt_x(x, y) returns the partial derivative of each element in x with respect to each element in the output of f(x, y). Because x contains 5 items and f(x,y) returns 5 items, this will be a 5x5 matrix. The (i,j)-th element in this matrix corresponds to the partial derivative of output[i] with respect to x[j].

Likewise, jacobian_f_wrt_y(x, y) returns the partial derivative of each element in y with respect to each element in the output of f(x,y).

I hope this helps.

[jerryli1981]

Thank you very much!
Current I use elementwise_grad instead of jacobian. I found elementwise_grad return the vector that is what I want. However, jacobian, return a matrix (d*d). I checked the values in jacobian. They are the same as elementwise_grad.

So I think element wise_grad satisfy my need. Am I correct?

[brunojacobs]

Yes and no. The jacobian function gives you the complete jacobian of your vector function f, with respect to the argument x or y. See: https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant). What the elementwis_grad function does is take the partial derivative of output[i] with respect to input[i]. These partial derivates are equivalent to the diagonal elements of the Jacobian matrix. In your function abs(x - y) all the off-diagonal elements are zero. However, this does not mean that the Jacobian matrix and elementwise_grad give the same output, as the latter only returns the diagonal of the Jacobian matrix.

Again, I can probably best illustrate this with an example, by adding an interaction between x and y to the function:

import autograd.numpy as np
from autograd import jacobian

def f(x, y):
    return np.abs(x - y + np.dot(x,y))

jacobian_f_wrt_x = jacobian(f, 0) # 0 indicates first input element in f(x, y)
jacobian_f_wrt_y = jacobian(f, 1) # 1 indicates second input element in f(x, y)

x = np.arange(-4, 6, 2, dtype=float)
y = np.zeros(5) + 1.5

print(jacobian_f_wrt_x(x, y))
print(jacobian_f_wrt_y(x, y))

Now the output has changed to

[[-2.5 -1.5 -1.5 -1.5 -1.5]
 [-1.5 -2.5 -1.5 -1.5 -1.5]
 [-1.5 -1.5 -2.5 -1.5 -1.5]
 [ 1.5  1.5  1.5  2.5  1.5]
 [ 1.5  1.5  1.5  1.5  2.5]]
[[ 5.  2.  0. -2. -4.]
 [ 4.  3.  0. -2. -4.]
 [ 4.  2.  1. -2. -4.]
 [-4. -2.  0.  1.  4.]
 [-4. -2.  0.  2.  3.]]

Hope this helps.

[jerryli1981]

Thanks for you greatful explanations. Now I know what should I do in my project.

[mattjj]

Thanks for the explanations, @brunojacobs! Indeed as pointed out, elementwise_grad is a special case of jacobian.

I thought I'd chime in with one more idea: you might try gluing x and y together in a tuple so you can get both of their gradients at once:

import autograd.numpy as np
from autograd import grad, elementwise_grad

def f((x, y)):
    return np.abs(x - y)

x = np.arange(-4, 6, 2, dtype=float)
y = np.zeros(5) + 1.5

print elementwise_grad(f)((x, y))

In [1]: run issue66
(array([-1., -1., -1.,  1.,  1.]), array([ 1.,  1.,  1., -1., -1.]))

In Python 3 you might have to do something like

def f(xy):
    x, y = xy
    return np.abs(x - y)