Regression: lambdify of Derivative instance now throws PrintMethodNotImplementedError

[nicholasferguson]

I cant find any examples to do:
Set the printer option 'strict' to False

Where can I find an example?

[nicholasferguson]

found hack solution... changed 'strict' :False
It was 'strict':None

That error msg needs to be more precise. It eats up research time, in a terrible fashion.

class CodePrinter(StrPrinter):

_default_settings: dict[str, Any] = {
    'order': None,
    'full_prec': 'auto',
    'error_on_reserved': False,
    'reserved_word_suffix': '_',
    'human': True,
    'inline': False,
    'allow_unknown_functions': False,
    'strict': False #None  # True or False; None => True if human == True
}

[oscarbenjamin]

It eats up research time, in a terrible fashion.

Development time is eaten up by people opening issues without doing the obvious things like showing some code to reproduce the issue.

[nicholasferguson]

https://github.com/nicholasferguson/SympyANDmpmath

[bjodah]

@nicholasferguson I authored the error message. The goal of this error message was of course to waste less research time (since sympy used to silently generate broken code), but we (I) might have unintentionally introduced a bug, so thanks for opening an issue (I think we all want the same thing here, and everyone involved are donating their time).

From your repository I could create this reproducer:

import mpmath
from sympy import Symbol, zeta, lambdify, Derivative

s = Symbol('s')
z = zeta(s)
zp = Derivative(z, s)
f = lambdify(s, zp, modules=['mpmath', {'Derivative': lambda expr, z: mpmath.zeta(z, derivative=1)}])
print(f(2))

which fails for the merge commit (b93a982) of gh-25913 (but passes when doing git checkout b93a982~1).

If you print the docstring of our generated function f for the older version of sympy it looks like this:

>>> print(f.__doc__)
Created with lambdify. Signature:

func(s)

Expression:

Derivative(zeta(s), s)

Source code:

def _lambdifygenerated(s):
    return (  # Not supported in Python with mpmath:
  # Derivative
Derivative(zeta(s), s))


Imported modules:

I see that there's a printer setting called "allow_unknown_functions" (which looks like it is being enabled by lambdify) that alters when CodePrinter calls the method _print_not_supported:

sympy/sympy/printing/codeprinter.py

Line 458 in 2565eb3

if expr.is_Function and self._settings.get('allow_unknown_functions', False):

So .is_Function needs to be True, let's check if that holds for our Derivative instance:

>>> zp.is_Function
False

So, with a bit of luck it would be enough to change that if-statement to if (expr.is_Function or expr.is_Derivative) ....

EDIT: (and if the fix turns out to be more complicated, we could resort to passing strict=False from lambdify)

[bjodah]

While trying to fix this I realize that this code doesn't make much sense:

>>> f = lambdify(s, zp, modules=['mpmath', {'Derivative': lambda expr, z: mpmath.zeta(z, derivative=1)}])

any Derivative instance would map to mpmath.zeta, an expression containing different derivatives would return non-meaningful results.

The proper fix would be to first replace the particular derivative of interest with a Function instance (e.g. Function('my_zeta_derivative')(s)). And then pass this function as a known function to lambdify. That this ever "worked" is mostly an accident.

Furthermore, the previous generated code performed unnecessary work (it evaluated the primitive function at the point of interest). I have a local patch of codeprinter.py which generates a "functioning" lambdify callable:

(Pdb) print(f.__doc__)
Created with lambdify. Signature:

func(s)

Expression:

Derivative(zeta(s), s)

Source code:

def _lambdifygenerated(s):
    return Derivative(zeta(s), (s, 1))


Imported modules:

but this is rather pointless. If we can come up with a legitimate use of passing 'Derivative' as a known function, I can use that as a test case for my patch, and submit it as a PR. But the original code snippet to reproduce the issue is not suitable since it doesn't do what you might expect.

The proper way to customize how derivatives are evaluated in the lambdified function would probably be along the lines of:

>>> from sympy import symbols, zeta, lambdify, Function
>>> import mpmath
>>> s = symbols('s')
>>> expr = 42 + 17*zeta(s).diff(s)
>>> zeta_diff = Function('zeta_diff')(s)
>>> expr2 = expr.replace(zeta(s).diff(s), zeta_diff)
>>> expr2
17*zeta_diff(s) + 42
>>> f = lambdify(s, expr2, ['mpmath', {'zeta_diff': lambda arg: mpmath.zeta(arg, derivative=1)}])
>>> f(2)
mpf('26.061679676630654')

[nicholasferguson]

That is why Aaron Meurer in his blog, which in repository is a pdf file .... writes

We now use lambdify to convert the SymPy expressions Z and D into functions that are evaluated using mpmath. A technical difficulty here is that the derivative of the zeta function
does not have a closed-form expression. mpmath's zeta can evaluate
but it doesn't yet work with sympy.lambdify (see SymPy issue 11802). So we have to manually define "Derivative" in lambdify, knowing that it will be the derivative of zeta when it is called. Beware that this is only correct for this specific expression where we know that Derivative will be Derivative(zeta(s), s).

[asmeurer]

This is the issue referenced in the blog post #11802

[bjodah]

Thank you @asmeurer, I opened gh-26678. Let me know what you think.