symbolic_expression_from_maxima_string() much too slow

[rwst]

sage: from sage.calculus.calculus import symbolic_expression_from_maxima_string as sefms
sage: var('v1,v2,v3')
(v1, v2, v3)
sage: ex=-1/3*(pi + 1)*(3*(euler_gamma - e)*(pi - 3*v1 - v1/arcsech(1) + e^(-1)/pi) - 6*v3^2*arcsinh(-v1 + e)/v2 - v2 - 3*log_gamma(v1*v3)/v2 - 3*e^(-254) + 3)*(-catalan/v3)^(twinprime*pi - 1/2*v1 - 1/2*v2)*inverse_jacobi_cs(1, v3)/jacobi_sc(1/arccos(-1/(v1*csc(v3))), v3/v1 + e) - 1/4*(2*v3^2*(e + 1) + ((e*log_integral(arcsech(exp_integral_e1(v2^mertens - 1) - 4)) + 15*v1 + jacobi_dn(v2, pi))*v1*e^(-1) + golden_ratio*pi^v1*(1/v3^12 + jacobi_ds(-10, csc(v3^2)))^(v2 - 1/2/v2)*sinh(v2*e)/((v1 + v2 + v3 + 1)*v2))/(v1^2*inverse_jacobi_nc(v1, -e)) - 2*bessel_Y(v3, v2))/v2 + v3/inverse_jacobi_sc(1, v2) - (v1 + 1)/((v2 + v3)*(2*(v1 + e)*(v3 - 1)/(pi + v1) + (-v3*sech(v1*v3)/v2)^(-e/v1))) + inverse_jacobi_cn(pi + v1*v3, pi - v3) + jacobi_sn(e, arctanh(-(-log_integral(2) + log_integral(jacobi_ds(-1, v3)))^v2*e)^(1/7*(18*v2 - v3)*(14*v2 + e)/(v3*arctan(1/v2)*kronecker_delta(v1, v3))))
sage: timeit('sefms(str(ex))')
5 loops, best of 3: 2.19/2.14/2.15 s per loop (without #6882)
5 loops, best of 3: 2.13/2.11/2.12 s per loop (with #6882)

It calls 120 times the function MaximaLib._eval_line() which takes 17ms in average = 2 seconds. 17 ms is an eternity. A lot of simplification attempts are going on in _eval_line. For this expression I count ten calls to trigsimp, trigexpand, radcan, fullratsimp. Why would we want to call Maxima when converting *from Maxima, at all? And if it's necessary to build the expression, why simplify when it's supposed to be already simplified?

Component: interfaces

Keywords: maxima, conversion, optimization, speed

Issue created by migration from https://trac.sagemath.org/ticket/16594

[nbruin]

comment:1

First a technicality (which I don't think affects the point you're making): You're feeding sefms a string that's not coming from maxima, so it's not entirely obvious that str(ex) is valid input for sefms. For instance:

sage: maxima_calculus(euler_gamma)
%gamma
sage: maxima_calculus(pi)
%pi

so sefms is getting all kinds of symbols here that in maxima don't represent what they're meant to.
In fact, if I try to convert properly, I get an error for this expression:

sage: maxima_calculus(ex)
TypeError: ECL says: Error executing code in Maxima: expt: undefined: 0 to a negative exponent.

The same problem arises when we try to avoid string manipulations as much as possible:

sage: t=maxima_calculus(str(ex))
sage: from sage.interfaces.maxima_lib import max_to_sr
sage: %time max_to_sr(t.ecl())
CPU times: user 1.6 s, sys: 3 ms, total: 1.6 s

Indeed, the fact that these expressions aren't valid prevents us from checking that the results are equal:

sage: bool(max_to_sr(t.ecl()) == sefms(str(ex)))
TypeError: ECL says: Error executing code in Maxima: expt: undefined: 0 to a negative exponent.

I think the case would be better made with an example that's actually valid.

That said, I suspect the following happens: when we reconstruct an expression (from a string or otherwise), we call the symbolic functions in question, relying on them producing an appropriate symbolic expression. However, many symbolic functions try to do some simplifications before giving up and just returning a symbolic expression. As you're experiencing, some of these routines may be trying a little much for cases where, given the source of the expression, we expect very little simplification to occur.

Bypassing this, however, would be rather complicated. Of course, we could just transliterate the syntax trees (in essence, the other direction sr_to_max does that--it does NOT call the maxima translations but just puts their symbols in an expression tree) but in reality we don't have the information for doing so. It would also mean that the current decentralized architecture where every symbolic function gets a chance to validate its arguments would need to be revisited.

The most reasonable solution might be to equip all symbolic function-producing call sequences (or at least the ones that interfaces produce) have an extra optional argument dont_do_anything_fancy=True, to avoid any expensive argument manipulations. Perhaps this can be coordinated with hold=True which already exists on some.

You would then have to insert that optional parameter at all relevant spots.

In short: it's not sefms itself, it's the fact that sefms is accessing sage's symbolic functions via their end-user interface, because we don't have another one.