Make MatchPy a dependency of SymPy on pip/conda

[Upabjojr]

MatchPy could become the default pattern matching library for SymPy.

The only GPL'd dependency in MatchPy was removed in HPAC/matchpy#64

MatchPy only supports Python 3.6+, which was a problem until a year ago. Since we dropped Python 3.5 support, we can now make MatchPy a hard dependency.

I've also suggested that the unit test framework in MatchPy be extended to test SymPy's pattern matcher. This can be done as soon as a new release of SymPy is out.

So, what about making MatchPy a pip/conda dependency of SymPy?

[sylee957]

Should we be able to use matchpy for backend for wild symbols?

[oscarbenjamin]

How would we use matchpy? Would it be for end users to use as well or is it something to be used internally?

[Upabjojr]

How would we use matchpy? Would it be for end users to use as well or is it something to be used internally?

Both. MatchPy can replace our pattern matcher.

[Upabjojr]

Should we be able to use matchpy for backend for wild symbols?

It's way more powerful than our current pattern matchers. We already have the backend in sympy/utilities/matchpy_connector.py . I'm simply suggesting that MatchPy should be installed automatically after calling pip install sympy... The same way as mpmath is installed.

[oscargus]

Would it make sense to move rubi to a separate package, sympy-rubi, still hosted under the sympy umbrella? Which then could have matchpy as a dependency.

Not that I am objecting matchpy as such, it can probably be quite useful elsewhere. But it would either way make sense to move rubi to a separate package as there is quite a lot of "dead code" in the repo at the moment because of this.

[Upabjojr]

Would it make sense to move rubi to a separate package, sympy-rubi, still hosted under the sympy umbrella? Which then could have matchpy as a dependency.

Yes, that's a good idea.

[oscarbenjamin]

Is the plan still to make rubi a separate package?

[oscargus]

I think it still makes sense. Gave it a go a few months ago, but never really managed to create a new repo with rubi including all the history. I do not recall exactly where things went wrong.

[oscargus]

Here is the repo, but only some basic info is pushed.

https://github.com/sympy/sympy_rubi

[oscarbenjamin]

If rubi is going go go to a separate repo then it doesn't need the 1.11 milestone (it can be moved any time).

[Upabjojr]

SymPy now has a new Wolfram Mathematica language parser that should be able to deprecate parts of Rubi's code generator.

[asmeurer]

Does SymPy ship rubi with the release? I thought we had removed it, but I just made a test sdist and it seems to be included. It might be good to remove it from there, even if we don't remove it from the repo yet. It's a lot of code to ship with the package given that it's effectively useless.

[asmeurer]

sympy/integrals/rubi is 7.9 MB uncompressed (that's just the plain text files, not counting the size of the pyc files). For reference, sympy itself is 33 MB, so nearly a quarter of the uncompressed package size is RUBI. Personally I think that would all be fine if RUBI actually worked, but right now it's all effectively dead code.

[Upabjojr]

I'm wondering if we are going to get some GSoC student in the future to complete the RUBI project. It's a pity because there isn't actually so much work to do to make it operational.

[asmeurer]

@Upabjojr If it's not there already, can you write down exactly what needs to be done on the GSoC ideas page? If you want to write something more detailed you can make a separate page on the wiki to outline everything.

Something like a NumFOCUS small development grant would also be an option. That would mean we don't have to wait for the next round of GSoC, and it would also allow paying someone who isn't necessarily new to the project to work on it (GSoC is only for students and people new to open source). If anyone is interested in this (including you yourself Francesco), please reach out.

[Upabjojr]

@Upabjojr If it's not there already, can you write down exactly what needs to be done on the GSoC ideas page? If you want to write something more detailed you can make a separate page on the wiki to outline everything.

This is my latest edit to the GSoC ideas:
https://github.com/sympy/sympy/wiki/GSoC-Ideas#create-a-wolfram-mathematica-interpreter

It's more generic than the RUBI project. It involves creating a generic interpreter or translator for most of Mathematica's code. We are not that far off, MatchPy already implements the hardest parts, the RUBI module contains some kind of code generators that could be readapted, and SymPy's MathematicaParser in the development branch is much more powerful than our previous parsers.

Something like a NumFOCUS small development grant would also be an option. That would mean we don't have to wait for the next round of GSoC, and it would also allow paying someone who isn't necessarily new to the project to work on it (GSoC is only for students and people new to open source).

We had three students working on the RUBI project in the past. Since then, no more applications have arrived. Maybe we should try to better advertise this opportunity?

[oscarbenjamin]

We need to strip out most of the ideas from the GSOC ideas page. The list should be short and should only contain projects that we have (recently) decided are high priority and that can definitely be mentored. There are too many ideas there which leaves people confused about what to focus on.

[Upabjojr]

We need to strip out most of the ideas from the GSOC ideas page. The list should be short and should only contain projects that we have (recently) decided are high priority and that can definitely be mentored. There are too many ideas there which leaves people confused about what to focus on.

You are right. A lot of stuff is leftover from past ideas.