Integration with Symbolics.jl

[saschatimme]

We should keep track of https://github.com/JuliaSymbolics/Symbolics.jl . Maybe at some point we can even remove the symengine dependency in favour of Symbolics.jl

[alok]

is there still interest in this? i would be interested to learn how both libraries work by implementing the integration, with light mentorship.

[PBrdng]

I think yes, there is still interest. But the mentorship should be done by @saschatimme (because he coded this part) and I don't know if he has the capacities at the moment.

[azoviktor]

I have been observing that HC.jl is slow when dealing with large Expressions, e.g. with determinants of 10x10 matrices of monomials (of degrees 1 or 2). I think this is due to inefficient evaluation of determinants and their derivatives. In order to evaluate a determinant we can evaluate a matrix and then compute its determinant. There is also a formula for the derivative.

Having this in mind, it seems like it would be good to have an abstract type AbstractExpression with an interface evaluate, differentiate and others for unusual kinds of expressions so that a user will implement these himself. Have you thought about this?

I think I could use https://www.juliahomotopycontinuation.org/HomotopyContinuation.jl/stable/systems/#Interface in order to implement evaluate! and evaluate_and_jacobian! methods for systems that include such complicated expressions, but it is a bit of a crutch solution of the problem.

[saschatimme]

Yes, some sort of special handling for determinants would be really nice but it is definitely not trivial to integrate it into our symbolics pipeline. Right now every symbolic function is assumed to be taking in one or two arguments (We have our own efficient interpreter to evaluate system). Additionally your evaluation function for determinants needs to properly work with higher order precision, and we actually don't just need the first order derivative but we need to be able to derive a local Taylor series up to order 4. So its complicated 🙃

@azoviktor Did you try to run HC with the compile=false flag? This could help if the slowness is triggered by compilation time.

Otherwise, my general recommendation is to avoid as much as possible the use of determinants. In particular 10x10 sounds very challenging from a numerical point of view. It is (when possible) much better to replace your equations by a modified system in higher dimension. The basic idea is to replace det(A) = 0 by something like [A * v; vT v - 1] = 1

[azoviktor]

@saschatimme, I see, indeed complicated :-)

The basic idea is to replace det(A) = 0 by something like [A * v; vT v - 1] = 1

yes, this is exactly what I did. I was just wondering what is the best way to deal with this.

Also I would like to ask, why do you prefer Symbolics.jl more than symengine? Is it because you would like to get rid of calling C? Do you think Symbolics.jl could be faster and calling C slows things down?

[saschatimme]

Using Symbolics.jl would allow some niceties like evaluating arbitrary Julia types with the symbolic engine (with symengine we can only do the symengine supported ones). Also based on Symbolics.jl / ModelingToolkit there is some nice work for rewriting symbolic system to improve evaluation performance etc.