N + Subs + Zeta + Derivative = maximum recursion depth exceeded

[latot]

Hi, here the problem:

>>> x=symbols('x')
>>> p=zeta(x)
>>> p=p.diff(x, 3)
>>> p=p.subs(x, 4)
>>> N(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.4/site-packages/sympy/core/evalf.py", line 1481, in N
    return sympify(x).evalf(n, **options)
#The next 2 lines x490 times
  File "/usr/lib64/python3.4/site-packages/sympy/core/function.py", line 561, in _eval_derivative
    l.append(df * da)
  File "/usr/lib64/python3.4/site-packages/sympy/core/decorators.py", line 77, in __sympifyit_wrapper
    return func(a, b)
  File "/usr/lib64/python3.4/site-packages/sympy/core/decorators.py", line 118, in binary_op_wrapper
    return func(self, other)
  File "/usr/lib64/python3.4/site-packages/sympy/core/expr.py", line 141, in __mul__
    return Mul(self, other)
  File "/usr/lib64/python3.4/site-packages/sympy/core/cache.py", line 93, in wrapper
    retval = cfunc(*args, **kwargs)
  File "/usr/lib64/python3.4/functools.py", line 458, in wrapper
    key = make_key(args, kwds, typed)
  File "/usr/lib64/python3.4/functools.py", line 382, in _make_key
    return _HashedSeq(key)
  File "/usr/lib64/python3.4/functools.py", line 351, in __init__
    self.hashvalue = hash(tup)
  File "/usr/lib64/python3.4/site-packages/sympy/core/basic.py", line 110, in __hash__
    h = hash((type(self).__name__,) + self._hashable_content())
RuntimeError: maximum recursion depth exceeded

Thx. Cya.

[cbm755]

For some reason that I haven't tracked down, this can actually crash Python on Fedora 30, Intel hardware, SymPy 1.4

Python 3.7.3 (default, Mar 27 2019, 13:36:35) 
[GCC 9.0.1 20190227 (Red Hat 9.0.1-0.8)] on linux

>>> A = zeta(x).diff(x,2).subs(x, 2)
⎛  2      ⎞│   
⎜ d       ⎟│   
⎜───(ζ(x))⎟│   
⎜  2      ⎟│   
⎝dx       ⎠│x=2

>>> N(A)
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007f5c7a9c3680 (most recent call first):
  File "/usr/lib/python3.7/site-packages/sympy/core/sympify.py", line 293 in sympify
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 51 in <genexpr>
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 52 in __new__
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 146 in <lambda>
  File "/usr/lib/python3.7/site-packages/sympy/core/sympify.py", line 293 in sympify
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 51 in <genexpr>
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 52 in __new__
  File "/usr/lib/python3.7/site-packages/sympy/core/containers.py", line 146 in <lambda>
  File "/usr/lib/python3.7/site-packages/sympy/core/sympify.py", line 293 in sympify
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 1218 in __new__
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 1556 in _eval_derivative
  File "/usr/lib/python3.7/site-packages/sympy/core/basic.py", line 1663 in _visit_eval_derivative_scalar
  File "/usr/lib/python3.7/site-packages/sympy/core/basic.py", line 1658 in _accept_eval_derivative
  File "/usr/lib/python3.7/site-packages/sympy/core/basic.py", line 1682 in _eval_derivative_n_times
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 1392 in __new__
  File "/usr/lib/python3.7/site-packages/sympy/core/expr.py", line 3095 in diff
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2115 in doit
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  File "/usr/lib/python3.7/site-packages/sympy/core/function.py", line 2136 in evalf
  ...
Aborted (core dumped)

That is, not just recursion in Python but a proper stack overflow... Ouch.

The first derivative still causes a "maximum recusion depth exceeded" exception as noted above.

[latot]

maybe now pthon3.7 have less limit than the previos versions, maybe test sys.setrecursionlimit(99999)?

[oscarbenjamin]

A here is an object that doits to itself:

In [1]: A = zeta(x).diff(x,2).subs(x, 2)                                                                                          

In [2]: A                                                                                                                         
Out[2]: 
⎛  2      ⎞│   
⎜ d       ⎟│   
⎜───(ζ(x))⎟│   
⎜  2      ⎟│   
⎝dx       ⎠│x=2

In [3]: A.doit()                                                                                                                  
Out[3]: 
⎛  2      ⎞│   
⎜ d       ⎟│   
⎜───(ζ(x))⎟│   
⎜  2      ⎟│   
⎝dx       ⎠│x=2

So this will give infinite recursion:

sympy/sympy/core/function.py

Lines 2135 to 2136 in c162866

	def evalf(self, prec=None, **options):
	return self.doit().evalf(prec, **options)

[latot]

mm, that sounds more like a sympy bug.

[sachin-4099]

I would like to take up this issue. Please guide me as to what needs to be done.
@oscarbenjamin @latot @cbm755

[latot]

Ouuu, this problem seems to be..., this really need a test for sympy.

Checking deeper, this seems to not belong to Zeta function, other example stieltjes, i don't really know where the bug is, but, that line from @oscarbenjamin , the doit(), seems to always be returning the same object, this will happen with any function that doesn't have a solution with elementary functions, or maybe implemented functions?

My big question to know why this happen, Why evalf works with Zeta, and fails with the derivative?

It almost seems that Zeta, have a different evalf with their derivative.

Maybe track in both cases how works evalf, and check in what section do differents things.

Hope this can help, and if other one can say other idea how work on this would be great, i don't know very well how works sympy.

Thx.

[oscarbenjamin]

The idea is that a Subs.evalf() can only really work by doing the substitution so doit forces the substitution to happen. Normally Subs.doit will return a non-Subs object. Certain cases like derivatives that can't be evaluated can only be represented as Subs so doit can't do anything.

Perhaps the fallback for evalf here could be to numerically evaluate the derivative:

In [81]: h = S(10)**-10                                                                                                           

In [82]: h                                                                                                                        
Out[82]: 1/10000000000

In [83]: differentiate_finite(zeta(x).diff(x), x, points=[2-h, 2, 2+h])                                                           
Out[83]: 
                         2                                                                              
  100000000000000000000⋅π                           ⎛20000000001⎞                          ⎛19999999999⎞
- ──────────────────────── + 100000000000000000000⋅ζ⎜───────────⎟ + 100000000000000000000⋅ζ⎜───────────⎟
             3                                      ⎝10000000000⎠                          ⎝10000000000⎠

In [84]: _.evalf()                                                                                                                
Out[84]: 1.98928023429890

[latot]

Yes, that wold works.

I have a question.
I don't get the why Zeta can be evaluated and their derivative not.

[oscarbenjamin]

I don't get the why Zeta can be evaluated and their derivative not.

I'm not sure exactly what you mean by "evaluated" there. SymPy does not have defined symbolic functions that can represent the derivatives of the zeta function. Those could be added and if they were then they could use explicit algorithms for numerical approximate evaluation:
https://en.wikipedia.org/wiki/Riemann_zeta_function#Numerical_algorithms

Otherwise if you mean the fact that zeta(x).diff(x).subs(x, 1).evalf() doesn't work then that's the whole point of this issue.

I think that a reasonable solution here is to add something to Subs.evalf that checks whether doit has returned a Subs and whether the expression in that Subs is a Derivative and then uses numerical differentiation (around the point to be substituted) to approximate the result.

[latot]

Is there a way to know this, before?, i mean, intead check if doit() return a Subs, when the function is constructed, check this and set it as a propperty in a variable, so we will not need check this every time.

[oscarbenjamin]

There isn't a reasonable way to know if doit will return a Subs without calling doit.

[latot]

In that case, when the expression is constructed, run doit and check there isn't vars (if there is a function f(x, y) and we only set x, doit will return the same), and if return the same save it in a var in the object.

Is to just avoid call doit every time that we want get some type of aproximation.