derivative of symbolic integration with Infinity limit fails

[daju1]

sage: x,y = var("x,y")
sage: f = integrate(-2*(2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(8*x^3/(x^2 + y^2)^5 - 3*x/(x^2 + y^2)^4), x, 1, +Infinity, hold=True)
sage: f 
integrate(-2*(2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(8*x^3/(x^2 + y^2)^5 - 3*x/(x^2 + y^2)^4), x, 1, +Infinity)
sage: assume(y>0)
sage: f.diff(y)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-f1ce8deb4ff7> in <module>
      1 assume(y>Integer(0))
----> 2 f.diff(y)

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.derivative (build/cythonized/sage/symbolic/expression.cpp:27215)()
   4361             ValueError: No differentiation variable specified.
   4362         """
-> 4363         return multi_derivative(self, args)
   4364 
   4365     diff = differentiate = derivative

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/misc/derivative.pyx in sage.misc.derivative.multi_derivative (build/cythonized/sage/misc/derivative.c:3176)()
    220 
    221     for arg in derivative_parse(args):
--> 222         F = F._derivative(arg)
    223     return F
    224 

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression._derivative (build/cythonized/sage/symbolic/expression.cpp:27738)()
   4433         sig_on()
   4434         try:
-> 4435             x = self._gobj.diff(ex_to_symbol(symbol._gobj), deg)
   4436         finally:
   4437             sig_off()

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/integration/integral.py in _tderivative_(self, f, x, a, b, diff_param)
    294             ans = SR.zero()
    295         if hasattr(b, 'diff'):
--> 296             ans += f.subs(x == b) * b.diff(diff_param)
    297         if hasattr(a, 'diff'):
    298             ans -= f.subs(x == a) * a.diff(diff_param)

/usr3/articles/sagemath_docker_build/sage/local/lib/python3.9/site-packages/sage/symbolic/expression.pyx in sage.symbolic.expression.Expression.substitute (build/cythonized/sage/symbolic/expression.cpp:32201)()
   5579             smap.insert(make_pair((<Expression>self.coerce_in(k))._gobj,
   5580                                   (<Expression>self.coerce_in(v))._gobj))
-> 5581         res = self._gobj.subs_map(smap, 0)
   5582         return new_Expression_from_GEx(self._parent, res)
   5583 

RuntimeError: indeterminate expression: 0 * infinity encountered.

CC: @EmmanuelCharpentier @slel @mkoeppe

Component: symbolics

Author: Alexey Drozdov

Branch/Commit: u/gh-daju1/derivative_of_symbolic_integration_with_infinity_limit_fails @ fd5cf30

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/32184

[daju1]

comment:1

Attachment: derivative of symbolic integration with Infinity limit fails.ipynb.gz

[daju1]

Branch: u/gh-daju1/derivative_of_symbolic_integration_with_infinity_limit_fails

[daju1]

comment:3

Attachment: derivative of symbolic integration with Infinity limit fail fixed.ipynb.gz

New commits:

c5dce36

Trac #32184: derivative of symbolic integration with Infinity limit fails

[daju1]

Commit: c5dce36

[daju1]

comment:4

Using this commit I have successful derivative of symbolic integration with Infinity limit

sage: f.diff(y)
16*integrate((2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(10*x^3*y/(x^2 + y^2)^6 - 3*x*y/(x^2 + y^2)^5), x, 1, +Infinity)

[mkoeppe]

Author: Alexey Drozdov

[mkoeppe]

comment:5

Needs doctests - https://www.sagemath.org/git-developer-guide/coding_basics.html#writing-testable-examples

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

42fccb2

Trac: #32184 testable example added

[sagetrac-git]

Changed commit from c5dce36 to 42fccb2

[mkoeppe]

comment:9

I don't know this code very well, but I would suggest replacing b.diff(diff_param) != 0 by not b.diff(diff_param).is_trivial_zero() if that already fixes the example.

Also this markup

+        :trac:`32184 fixed::
+            sage: x,y = var("x,y")

is not correct, there needs to be a backquote after the ticket number and an empty line after ::. Best to build the HTML documentation and inspect the output to make sure it is OK.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c1f1d15

Trac: #32184 testable example fixed

[sagetrac-git]

Changed commit from 42fccb2 to c1f1d15

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

78e4c1e

Trac: #32184 replacing b.diff(diff_param) != 0 by not b.diff(diff_param).is_trivial_zero() fixes the example

[sagetrac-git]

Changed commit from c1f1d15 to 78e4c1e

[daju1]

comment:13

The HTML documentation is OK now too

[mkoeppe]

comment:14

With this version I am getting:

Trying (line 308):    f.diff(y)
Expecting:
    16*integrate((2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(10*x^3*y/(x^2 + y^2)^6 - 3*x*y/(x^2 + y^2)^5), x, 1, +Infinity)
**********************************************************************
File "src/sage/symbolic/integration/integral.py", line 308, in sage.symbolic.integration.integral.DefiniteIntegral._tderivative_
Failed example:
    f.diff(y)
Expected:
    16*integrate((2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(10*x^3*y/(x^2 + y^2)^6 - 3*x*y/(x^2 + y^2)^5), x, 1, +Infinity)
Got:
    16*y*(integrate(14*x^6/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-6*x^4*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-42*x^5*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(18*x^3*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(42*x^4*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-18*x^2*y^2*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-14*x^3*floor(x)^3/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(6*x*y^2*floor(x)^3/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-21*x^5/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(9*x^3*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(42*x^4*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-18*x^2*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-21*x^3*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(9*x*y^2*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(7*x^4/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-3*x^2*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-7*x^3*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(3*x*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity))

[mkoeppe]

Reviewer: Matthias Koeppe

[daju1]

comment:15

Hi, Matthias Koeppe!
you have very strange result as for me: on my code stack I have

./sage -t --long src/sage/symbolic/integration/integral.py
too few successful tests, not using stored timings
Running doctests with ID 2021-07-26-13-15-25-32b680b2.
Git branch: t/32184/derivative_of_symbolic_integration_with_infinity_limit_fails
Using --optional=build,debian,dochtml,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --random-seed=0 src/sage/symbolic/integration/integral.py
    [221 tests, 33.23 s]
----------------------------------------------------------------------
All tests passed!

Interesting, can you provide your git status and git log (for example in the attachment).
Thanks

[daju1]

comment:16

Attachment: track_32184_git_status.txt

[DaveWitteMorris]

comment:17

A patchbot confirms the error in comment:14, so the ticket needs work, not review. The successful log that was posted seems to use 9.4b4, not the current sage version, which is 9.4rc0.

[daju1]

comment:18

Ok, thanks gh-DaveWitteMorris

I have reproduced the issue just by merging of my current branch with develop branch

**********************************************************************
File "src/sage/symbolic/integration/integral.py", line 308, in sage.symbolic.integration.integral.DefiniteIntegral._tderivative_
Failed example:
    f.diff(y)
Expected:
    16*integrate((2*(x - floor(x))^3 - 3*(x - floor(x))^2 + x - floor(x))*(10*x^3*y/(x^2 + y^2)^6 - 3*x*y/(x^2 + y^2)^5), x, 1, +Infinity)
Got:
    DefiniteIntegral integrator <function maxima_integrator at 0x7fc2a8ed8040>
    DefiniteIntegral integrator <function giac_integrator at 0x7fc2a8ed8280>
    DefiniteIntegral integrator <function sympy_integrator at 0x7fc2a8ed80d0>
    16*y*(integrate(14*x^6/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-6*x^4*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-42*x^5*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(18*x^3*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(42*x^4*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-18*x^2*y^2*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-14*x^3*floor(x)^3/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(6*x*y^2*floor(x)^3/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-21*x^5/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(9*x^3*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(42*x^4*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-18*x^2*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-21*x^3*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(9*x*y^2*floor(x)^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(7*x^4/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-3*x^2*y^2/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(-7*x^3*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity) + integrate(3*x*y^2*floor(x)/(x^12 + 6*x^10*y^2 + 15*x^8*y^4 + 20*x^6*y^6 + 15*x^4*y^8 + 6*x^2*y^10 + y^12), x, 1, +Infinity))
**********************************************************************

Seems like we have new behavior of sympy_integrator in the develop branch

[daju1]

comment:19

My investigation of git log have shown that behavior of integrator was changed in the ticket

Fix wrong selection of integrator in some cases
#32002

by commit

enhanced switching between integrators
sagemath/sagetrac-mirror@bd78869

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b2416e2	Merge tag '9.4.rc0' into t/32184/derivative_of_symbolic_integration_with_infinity_limit_fails
3d9ce1d	Trac: #32184 testable example fixed according with enhanced switching between integrators

[sagetrac-git]

Changed commit from 78e4c1e to 3d9ce1d

[sagetrac-git]

Changed commit from 3d9ce1d to fd5cf30

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

fd5cf30

Merge tag '9.4.rc2' into t/32184/derivative_of_symbolic_integration_with_infinity_limit_fails

[mkoeppe]

comment:24

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.