Can't calculate derivative exception while sympifying of two arguments function derivative

[daju1]

Starting with a partial derivative
of a function of two variables:

sage: v_x, j_x, x, y = var("v_x, j_x, x, y")
sage: F = function("F")(x, y)
sage: expression = F.diff(x, 3).diff(y, 1).subs(x == j_x + v_x)
sage: expression
D[0, 0, 0, 1](F)(j_x + v_x, y)

and trying to "simpify" it (convert it to Sympy),
I have met the following exception:

sage: ex = expression._sympy_()
f_sympy F(j_x + v_x, y)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-32-da30dd17641c> in <module>
----> 1 ex = expression._sympy_()

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._sympy_ (build/cythonized/sage/symbolic/expression.cpp:12810)()
   1786         """
   1787         from sage.symbolic.expression_conversions import sympy_converter
-> 1788         return sympy_converter(self)
   1789 
   1790     def _fricas_init_(self):

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression_conversions.py in __call__(self, ex)
    711                           super().__call__(ex))
    712         #print("ex", ex)
--> 713         return super().__call__(ex)
    714 
    715     def pyobject(self, ex, obj):

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression_conversions.py in __call__(self, ex)
    219             return self.relation(ex, operator)
    220         elif isinstance(operator, FDerivativeOperator):
--> 221             return self.derivative(ex, operator)
    222         elif operator == tuple:
    223             return self.tuple(ex)

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression_conversions.py in derivative(self, ex, operator)
    938         f_sympy = f._sympy_()(*_args)
    939         print("f_sympy", f_sympy)
--> 940         result = f_sympy.diff(*sympy_arg)
    941         if subs_new:
    942             return sympy.Subs(result, subs_new, subs_old)

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sympy/core/expr.py in diff(self, *symbols, **assumptions)
   3500     def diff(self, *symbols, **assumptions):
   3501         assumptions.setdefault("evaluate", True)
-> 3502         return _derivative_dispatch(self, *symbols, **assumptions)
   3503 
   3504     ###########################################################################

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sympy/core/function.py in _derivative_dispatch(expr, *variables, **kwargs)
   1945         from sympy.tensor.array.array_derivatives import ArrayDerivative
   1946         return ArrayDerivative(expr, *variables, **kwargs)
-> 1947     return Derivative(expr, *variables, **kwargs)
   1948 
   1949 

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sympy/core/function.py in __new__(cls, expr, *variables, **kwargs)
   1371             if not v._diff_wrt:
   1372                 __ = ''  # filler to make error message neater
-> 1373                 raise ValueError(filldedent('''
   1374                     Can't calculate derivative wrt %s.%s''' % (v,
   1375                     __)))

ValueError: 
Can't calculate derivative wrt j_x + v_x.

Maybe the exception raised by the Sympy library
can be handled in Sage?

Upstream: Fixed upstream, in a later stable release.

CC: @EmmanuelCharpentier @slel @frederichan-IMJPRG @nbruin @mkoeppe

Component: symbolics

Author: Alexey Drozdov

Branch/Commit: u/gh-daju1/can_t_calculate_derivative_exception_while_sympifying_of_two_arguments_function_derivative @ 5c1eb31

Issue created by migration from https://trac.sagemath.org/ticket/32394

[daju1]

Branch: u/gh-daju1/can_t_calculate_derivative_exception_while_sympifying_of_two_arguments_function_derivative

[daju1]

Commit: 15f5578

[daju1]

New commits:

15f5578

Trac #32394: Can't calculate derivative exception while sympifying of two arguments function derivative if agrument is a sum or a mul

[daju1]

Author: gh-daju1

[daju1]

Upstream: Fixed upstream, in a later stable release.

[daju1]

comment:7

Additional issue found if

sage: x, y = var("x, y")
sage: F = function("F")(x, y)
sage: expression = F.diff(x, 3).diff(y, 1).subs(x == 1.5)
sage: expression
D[0, 0, 0, 1](F)(1.50000000000000, y)

but

sage: ex = expression._sympy_()

gives

ValueError: 
Can't calculate derivative wrt 1.50000000000000.

[sagetrac-git]

Changed commit from 15f5578 to 5c1eb31

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5c1eb31

Trac #32394: Can't calculate derivative exception while sympifying of function derivative if its agrument is a number

[mkoeppe]

comment:11

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.

[slel]

Changed author from gh-daju1 to Alexey Drozdov

[EmmanuelCharpentier]

comment:13

I'm not sure that deriving with respect to the sum of two arguments makes sense. Consider :

sage: g=function("g")
sage: g(a, b, c).diff(a)
diff(g(a, b, c), a)
sage: g(a, b, c).diff(a+b)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-53-1221c21f3a0d> in <module>
----> 1 g(a, b, c).diff(a+b)

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.derivative (build/cythonized/sage/symbolic/expression.cpp:54855)()
   4633             ValueError: No differentiation variable specified.
   4634         """
-> 4635         return multi_derivative(self, args)
   4636 
   4637     diff = differentiate = derivative

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/misc/derivative.pyx in sage.misc.derivative.multi_derivative (build/cythonized/sage/misc/derivative.c:3281)()
    220 
    221     for arg in derivative_parse(args):
--> 222         F = F._derivative(arg)
    223     return F
    224 

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._derivative (build/cythonized/sage/symbolic/expression.cpp:55328)()
   4701         cdef Expression symbol = self.coerce_in(symb)
   4702         if not is_a_symbol(symbol._gobj):
-> 4703             raise TypeError("argument symb must be a symbol")
   4704         cdef GEx x
   4705         sig_on()

TypeError: argument symb must be a symbol

In our case :

sage: F.diff(x,3).diff(y,1).subs({x:j_x+v_x})
D[0, 0, 0, 1](F)(j_x + v_x, y)

So far, so good. But note that this is a three-argument function :

sage: F.diff(x,3).diff(y,1).subs({x:j_x+v_x}).arguments()
(j_x, v_x, y)

and, as exemplified above, deriving wrt the sum of two of them has no easy meaning...

Would this :

sage: F.diff(x,3).diff(y,1)._sympy_().subs({x._sympy_():j_x._sympy_()+v_x._sympy_()})
Subs(Derivative(F(x, y), (x, 3), y), x, j_x + v_x)

accomplish what you meant ?

[daju1]

Attachment: Ticket 32394 Can't calculate derivative wrt j_x + v_x.ipynb.gz

[daju1]

comment:15

Replying to @EmmanuelCharpentier:

I'm not sure that deriving with respect to the sum of two arguments makes sense. Consider :

sage: g=function("g")
sage: g(a, b, c).diff(a)
diff(g(a, b, c), a)
sage: g(a, b, c).diff(a+b)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-53-1221c21f3a0d> in <module>
----> 1 g(a, b, c).diff(a+b)

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.derivative (build/cythonized/sage/symbolic/expression.cpp:54855)()
   4633             ValueError: No differentiation variable specified.
   4634         """
-> 4635         return multi_derivative(self, args)
   4636 
   4637     diff = differentiate = derivative

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/misc/derivative.pyx in sage.misc.derivative.multi_derivative (build/cythonized/sage/misc/derivative.c:3281)()
    220 
    221     for arg in derivative_parse(args):
--> 222         F = F._derivative(arg)
    223     return F
    224 

/usr/local/sage-9/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._derivative (build/cythonized/sage/symbolic/expression.cpp:55328)()
   4701         cdef Expression symbol = self.coerce_in(symb)
   4702         if not is_a_symbol(symbol._gobj):
-> 4703             raise TypeError("argument symb must be a symbol")
   4704         cdef GEx x
   4705         sig_on()

TypeError: argument symb must be a symbol

In our case :

sage: F.diff(x,3).diff(y,1).subs({x:j_x+v_x})
D[0, 0, 0, 1](F)(j_x + v_x, y)

So far, so good. But note that this is a three-argument function :

sage: F.diff(x,3).diff(y,1).subs({x:j_x+v_x}).arguments()
(j_x, v_x, y)

and, as exemplified above, deriving wrt the sum of two of them has no easy meaning...

Would this :

sage: F.diff(x,3).diff(y,1)._sympy_().subs({x._sympy_():j_x._sympy_()+v_x._sympy_()})
Subs(Derivative(F(x, y), (x, 3), y), x, j_x + v_x)

accomplish what you meant ?

Hi, charpent. Please see recently added attachment to understand what I want. I have two functions. First of them:

R_px = euler_maclaurin_R_p(F, x, a_x, b_x,p, f_diff_symb_p=F.diff(x,p))

by executing the following code

                R_p = (-1)^(p+1)*integral(symbolic_sum(f.subs(symb == vx+jx).diff(vx,p)*B(x=vx,p=p)/fact(n=p), \
                                              jx, a, b-1, hold=hold_sum), \
                                          (vx,0,1), hold=hold_int)

gives:

R_px = integrate(sum(1/12*(2*v_x^3 - 3*v_x^2 + v_x)*D[0, 0, 0](F)(j_x + v_x, y), j_x, n_x, b_x - 1), v_x, 0, 1)

Here please note that deriving is really wrt v_x. As for me when I need to have sum of derivatives it is no defference how to derive: wrt v_x or wrt (j_x + v_x) which means just x in the interval from some integer to that integer + 1.

But the second function

sumy_R_px = sum_dfdx_bernoulis(R_px, y, a_y, b_y, p)

gives Exception here:

/tmp/ipykernel_21688/1831337102.py in sum_dfdx_bernoulis(f, symb, a, b, p)
      5     dfdx_a_bernoullis = []
      6     for k in range(Integer(1),Integer(1)+p):
----> 7         dfdx_a_bernoullis += [(f.diff(symb,k-Integer(1)))*(bernoulli(k)/factorial(k))]
      8 
      9     sum_dfdx_a_bernoullis = sum(dfdx_a_bernoullis)

even if symb is y

[daju1]

Attachment: Ticket 32394 Can't calculate derivative wrt pi sqrt.ipynb.gz

[daju1]

comment:16

Another case found which is related with current ticket, inside the attached Jupiter notebook, bellow:

u, a, k_m = var("u, a, k_m")
n_x, n_y, a_x, a_y, b_x, b_y = var("n_x, n_y, a_x, a_y, b_x, b_y")
p = 4
f = function('f')(var("k_km"))
Fu = lambda u, n_x, n_y, a, k_m : sqrt(n_x^2 + n_y^2 + u^2)*f(k_km=pi*sqrt(n_x^2 + n_y^2 + u^2)/(a*k_m))
Fu(u, n_x, n_y, a, k_m)

sqrt(n_x^2 + n_y^2 + u^2)*f(pi*sqrt(n_x^2 + n_y^2 + u^2)/(a*k_m))

def sum_dfdx_bernoulis(f,symb,a,b,p):
    if logging:
        print("f", f)
        print("symb,a,b", symb, a, b)
    dfdx_a_bernoullis = []
    for k in range(1,1+p):
        dfdx_a_bernoullis += [(f.diff(symb,k-1))*(bernoulli(k)/factorial(k))]
        
    sum_dfdx_a_bernoullis = sum(dfdx_a_bernoullis)

    if logging:
        print("sum_dfdx_a_bernoullis", sum_dfdx_a_bernoullis)
        print("sum_dfdx_a_bernoullis(a)", sum_dfdx_a_bernoullis.subs(symb == a))
        if Infinity != b:
            print("sum_dfdx_a_bernoullis(b)", sum_dfdx_a_bernoullis.subs(symb == b))

    s = - sum_dfdx_a_bernoullis.subs(symb == a)
    if Infinity != b:
        s += sum_dfdx_a_bernoullis.subs(symb == b)
    return s

logging = True
sum_int_Fu = sum_dfdx_bernoulis(integrate(Fu(u, n_x, n_y, a, k_m),(n_x, a_x, b_x), algorithm="sympy"),  n_y, a_y, b_y, p)
print("sum_int_Fu=",sum_int_Fu)

f integrate(sqrt(n_x^2 + n_y^2 + u^2)*f(pi*sqrt(n_x^2 + n_y^2 + u^2)/(a*k_m)), n_x, a_x, b_x)
symb,a,b n_y a_y b_y
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-7e52451d50cb> in <module>
      1 logging = True
----> 2 sum_int_Fu = sum_dfdx_bernoulis(integrate(Fu(u, n_x, n_y, a, k_m),(n_x, a_x, b_x), algorithm="sympy"),  n_y, a_y, b_y, p)
      3 print("sum_int_Fu=",sum_int_Fu)
      4 display(Math(latex(sum_int_Fu)))

<ipython-input-5-9d1e47158c9f> in sum_dfdx_bernoulis(f, symb, a, b, p)
      5     dfdx_a_bernoullis = []
      6     for k in range(Integer(1),Integer(1)+p):
----> 7         dfdx_a_bernoullis += [(f.diff(symb,k-Integer(1)))*(bernoulli(k)/factorial(k))]
      8 
      9     sum_dfdx_a_bernoullis = sum(dfdx_a_bernoullis)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.derivative (build/cythonized/sage/symbolic/expression.cpp:26932)()
   4273             ValueError: No differentiation variable specified.
   4274         """
-> 4275         return multi_derivative(self, args)
   4276 
   4277     diff = differentiate = derivative

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/misc/derivative.pyx in sage.misc.derivative.multi_derivative (build/cythonized/sage/misc/derivative.c:3177)()
    220 
    221     for arg in derivative_parse(args):
--> 222         F = F._derivative(arg)
    223     return F
    224 

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._derivative (build/cythonized/sage/symbolic/expression.cpp:27455)()
   4345         sig_on()
   4346         try:
-> 4347             x = self._gobj.diff(ex_to_symbol(symbol._gobj), deg)
   4348         finally:
   4349             sig_off()

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/integration/integral.py in _tderivative_(self, f, x, a, b, diff_param)
    288         if not x.has(diff_param):
    289             # integration variable != differentiation variable
--> 290             ans = definite_integral(f.diff(diff_param), x, a, b)
    291         else:
    292             ans = SR.zero()

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/function.pyx in sage.symbolic.function.BuiltinFunction.__call__ (build/cythonized/sage/symbolic/function.cpp:12501)()
   1152             res = self._evalf_try_(*args)
   1153             if res is None:
-> 1154                 res = super(BuiltinFunction, self).__call__(
   1155                         *args, coerce=coerce, hold=hold)
   1156 

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/function.pyx in sage.symbolic.function.Function.__call__ (build/cythonized/sage/symbolic/function.cpp:7034)()
    595             for i from 0 <= i < len(args):
    596                 vec.push_back((<Expression>args[i])._gobj)
--> 597             res = g_function_evalv(self._serial, vec, hold)
    598         elif self._nargs == 1:
    599             res = g_function_eval1(self._serial,

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/function.pyx in sage.symbolic.function.BuiltinFunction._evalf_or_eval_ (build/cythonized/sage/symbolic/function.cpp:13657)()
   1240         res = self._evalf_try_(*args)
   1241         if res is None:
-> 1242             return self._eval0_(*args)
   1243         else:
   1244             return res

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/integration/integral.py in _eval_(self, f, x, a, b)
    221         for integrator in self.integrators:
    222             try:
--> 223                 A = integrator(*args)
    224             except (NotImplementedError, TypeError,
    225                     AttributeError, RuntimeError):

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/integration/external.py in sympy_integrator(expression, v, a, b)
     61     """
     62     import sympy
---> 63     ex = expression._sympy_()
     64     v = v._sympy_()
     65     if a is None:

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._sympy_ (build/cythonized/sage/symbolic/expression.cpp:12437)()
   1705         """
   1706         from sage.symbolic.expression_conversions import sympy_converter
-> 1707         return sympy_converter(self)
   1708 
   1709     def _fricas_init_(self):

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in __call__(self, ex)
    216                 div = self.get_fake_div(ex)
    217                 return self.arithmetic(div, div.operator())
--> 218             return self.arithmetic(ex, operator)
    219         elif operator in relation_operators:
    220             return self.relation(ex, operator)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in arithmetic(self, ex, operator)
    718         import sympy
    719         operator = arithmetic_operators[operator]
--> 720         ops = [sympy.sympify(self(a), evaluate=False) for a in ex.operands()]
    721         if operator == "+":
    722             return sympy.Add(*ops)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in <listcomp>(.0)
    718         import sympy
    719         operator = arithmetic_operators[operator]
--> 720         ops = [sympy.sympify(self(a), evaluate=False) for a in ex.operands()]
    721         if operator == "+":
    722             return sympy.Add(*ops)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in __call__(self, ex)
    216                 div = self.get_fake_div(ex)
    217                 return self.arithmetic(div, div.operator())
--> 218             return self.arithmetic(ex, operator)
    219         elif operator in relation_operators:
    220             return self.relation(ex, operator)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in arithmetic(self, ex, operator)
    718         import sympy
    719         operator = arithmetic_operators[operator]
--> 720         ops = [sympy.sympify(self(a), evaluate=False) for a in ex.operands()]
    721         if operator == "+":
    722             return sympy.Add(*ops)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in <listcomp>(.0)
    718         import sympy
    719         operator = arithmetic_operators[operator]
--> 720         ops = [sympy.sympify(self(a), evaluate=False) for a in ex.operands()]
    721         if operator == "+":
    722             return sympy.Add(*ops)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in __call__(self, ex)
    220             return self.relation(ex, operator)
    221         elif isinstance(operator, FDerivativeOperator):
--> 222             return self.derivative(ex, operator)
    223         elif operator == tuple:
    224             return self.tuple(ex)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sage/symbolic/expression_conversions.py in derivative(self, ex, operator)
    913 
    914         f_sympy = f._sympy_()(*_args)
--> 915         result = f_sympy.diff(*sympy_arg)
    916         if subs_new:
    917             return sympy.Subs(result, subs_new, subs_old)

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sympy/core/expr.py in diff(self, *symbols, **assumptions)
   3385     def diff(self, *symbols, **assumptions):
   3386         assumptions.setdefault("evaluate", True)
-> 3387         return Derivative(self, *symbols, **assumptions)
   3388 
   3389     ###########################################################################

/opt/sagemath-9.2/local/lib/python3.7/site-packages/sympy/core/function.py in __new__(cls, expr, *variables, **kwargs)
   1328                 raise ValueError(filldedent('''
   1329                     Can't calculate derivative wrt %s.%s''' % (v,
-> 1330                     __)))
   1331 
   1332         # We make a special case for 0th derivative, because there is no

ValueError: 
Can't calculate derivative wrt pi*sqrt(n_x**2 + n_y**2 +
u**2)/(a*k_m).

Also I see that commits provided by me inside current ticket fix this example