Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spkg-configure.m4 for maxima #32867

Closed
orlitzky opened this issue Nov 13, 2021 · 72 comments
Closed

spkg-configure.m4 for maxima #32867

orlitzky opened this issue Nov 13, 2021 · 72 comments

Comments

@orlitzky
Copy link
Contributor

We handled ECL by itself in #29617, so now maxima needs its own ticket.

This would actually be "easy" at this point if not for our custom matrixexp.patch (https://sourceforge.net/p/maxima/bugs/2596/).

Depends on #33718

CC: @slel @antonio-rojas

Component: build: configure

Keywords: maxima

Author: Michael Orlitzky

Branch/Commit: u/mjo/ticket/32867 @ 544b601

Reviewer: https://github.com/sagemath/sage/actions/runs/1992568379

Issue created by migration from https://trac.sagemath.org/ticket/32867

@orlitzky orlitzky added this to the sage-9.5 milestone Nov 13, 2021
@orlitzky
Copy link
Contributor Author

Author: Michael Orlitzky

@orlitzky
Copy link
Contributor Author

Commit: 88992bf

@orlitzky
Copy link
Contributor Author

comment:1

Here's an otherwise-working branch that fails only matrixexp.patch tests.


New commits:

ab54000Trac #32867: don't export MAXIMA_PREFIX.
0e29761Trac #32867: remove the MAXIMA sage_conf variable.
88992bfTrac #32867: new spkg-configure.m4 for maxima.

@orlitzky
Copy link
Contributor Author

Branch: u/mjo/ticket/32867

@slel
Copy link
Member

slel commented Nov 18, 2021

comment:2

I opened #32898 so that removing matrixexp.patch
has its own ticket.

@tornaria
Copy link
Contributor

comment:3

This works for me on void linux.

Note that debian, arch, and I think also gentoo all include matrixexp.patch; I'll include it in void as well, so this ticket should not depend on #32898.

@slel
Copy link
Member

slel commented Nov 28, 2021

comment:4

Maxima is packaged for Cygwin as "maxima",
perhaps add a distros/cygwin.txt file?

Or that can be a separate ticket.

@slel
Copy link
Member

slel commented Nov 28, 2021

Changed keywords from none to maxima

@orlitzky
Copy link
Contributor Author

comment:5

We don't have the patch in Gentoo...

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Nov 28, 2021

Changed commit from 88992bf to 3306df2

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Nov 28, 2021

Branch pushed to git repo; I updated commit sha1. New commits:

3306df2Trac #32867: maxima package information for Cygwin.

@tornaria
Copy link
Contributor

comment:7

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

For the command line interface, it seems to me setting MAXIMA to "maxima" instead of "maxima -l ecl" would work (btw: why eliminate the config option MAXIMA and hardcode "maxima -l ecl" everywhere instead of just changing MAXIMA in a single place?)

For the library interface I don't know if it is possible to compile maxima.fas for sbcl; system packages for maxima that are usually compiled for sbcl don't include the library.

@orlitzky
Copy link
Contributor Author

comment:8

Replying to @tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Sadly it's not possible. Sage interfaces with maxima at a very low level through ECL; seesage/libs/ecl.pyx and sage/interfaces/maxima_lib.py.

For the command line interface, it seems to me setting MAXIMA to "maxima" instead of "maxima -l ecl" would work (btw: why eliminate the config option MAXIMA and hardcode "maxima -l ecl" everywhere instead of just changing MAXIMA in a single place?)

Given that we need ECL maxima for Sage, there aren't any other good values for that variable. The -l ecl is necessary to support system installations of maxima with (for example) both SBCL and ECL, where running just "maxima" would run it with SBCL. And the -l ecl flag doesn't do any harm with the maxima SPKG, because we know that the SPKG is built with ECL. The executable name "maxima" is standard and PATH manipulations in the sage build system ensure that the correct one gets used, so you're almost forced to have MAXIMA="maxima -l ecl".

With that in mind, removing the variable eliminates a "configurable" option that isn't actually configurable in practice, and also gets rid of a tiny bit of coupling between modules.

For the library interface I don't know if it is possible to compile maxima.fas for sbcl; system packages for maxima that are usually compiled for sbcl don't include the library.

No, until very recently even the ability to create an ECL library required a custom patch. That patch has finally been accepted upstream, but I don't think anything similar is possible with the other lisp implementations. (Sage is married to ECL for the moment either way.)

@mkoeppe mkoeppe modified the milestones: sage-9.5, sage-9.6 Dec 18, 2021
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Dec 23, 2021

Changed commit from 3306df2 to 49197f5

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Dec 23, 2021

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

a647806Trac #32867: don't export MAXIMA_PREFIX.
8cf10beTrac #32867: remove the MAXIMA sage_conf variable.
5009767Trac #32867: new spkg-configure.m4 for maxima.
d0f1651Trac #32867: maxima package information for Cygwin.
853ff79Trac #32867: disable maxima test that requires matrixexp.patch.
49197f5Trac #32867: add maxima package information for Gentoo.

@orlitzky
Copy link
Contributor Author

comment:11

I've disabled (for now) the one remaining test that needs matrixexp.patch, with a pointer to #32898. Otherwise we could wait another decade for this.

@nbruin
Copy link
Contributor

nbruin commented Dec 23, 2021

comment:12

Replying to @tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Adding to mjo's answer: for the expect-based interface, it may be doable to use SBCL. There are peculiarities in the way the stdin/stdout character streams are driven in different LISPs that may interfere with a simple drop-in replacement.

The main use in sage now, however, is through maxima_lib which is really using that the E in ECL means "embeddable": we link in the ecl, start it through its binary interface, and then instruct it to load maxima.fas. Our interfacing is using an ABI, not I/O streams, and we directly access the memory objects that ECL constructs to translate them to python (or leave them wrapped). I'm pretty sure this is all fundamentally impossible with SBCL. We are getting huge latency and efficiency gains through the ABI interface compared to I/O, so in many cases I suspect we can afford slower execution in ECL, and I think we don't want to go back to expect-based interfaces. That's why we're tied to ECL.

@tornaria
Copy link
Contributor

comment:13

Replying to @nbruin:

Replying to @tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Adding to mjo's answer: for the expect-based interface, it may be doable to use SBCL. There are peculiarities in the way the stdin/stdout character streams are driven in different LISPs that may interfere with a simple drop-in replacement.

I tried it (using sbcl maxima for the binary but maxima-ecl for maxima_lib) and I think it worked ok (as in: doctests passed).

The main use in sage now, however, is through maxima_lib which is really using that the E in ECL means "embeddable": we link in the ecl, start it through its binary interface, and then instruct it to load maxima.fas. Our interfacing is using an ABI, not I/O streams, and we directly access the memory objects that ECL constructs to translate them to python (or leave them wrapped). I'm pretty sure this is all fundamentally impossible with SBCL. We are getting huge latency and efficiency gains through the ABI interface compared to I/O, so in many cases I suspect we can afford slower execution in ECL, and I think we don't want to go back to expect-based interfaces. That's why we're tied to ECL.

In theory it may be possible (see http://www.sbcl.org/manual/#Calling-Lisp-From-C) although it might be a lot of work. The speed gains could be considerable for that part of sage, I don't know if it's overall worth the trouble.

@mkoeppe
Copy link
Member

mkoeppe commented Dec 25, 2021

comment:14

Isn't a version check for maxima necessary? On the oldest supported platform, system maxima is 5.32.1. https://repology.org/project/maxima/versions

@orlitzky
Copy link
Contributor Author

comment:15

Replying to @mkoeppe:

Isn't a version check for maxima necessary? On the oldest supported platform, system maxima is 5.32.1. https://repology.org/project/maxima/versions

In spirit yes, but in practice... maybe. The spkg-configure.m4 checks for a maxima with support for building maxima.fas as an ECL library, and that commit hasn't even made it into a released version of maxima yet. The only reason any system maxima would be carrying it as a custom patch is to be able to use maxima with sage, but that means that the system maxima probably works with sage. Up to some version (mis)matching anyway.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Dec 26, 2021

Changed commit from 49197f5 to b615d8e

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Dec 26, 2021

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8a56d85Trac #32867: don't export MAXIMA_PREFIX.
5ab8dacTrac #32867: remove the MAXIMA sage_conf variable.
0f9d66aTrac #32867: new spkg-configure.m4 for maxima.
38e916eTrac #32867: maxima package information for Cygwin.
c2aa14aTrac #32867: disable maxima tests that require matrixexp.patch.
b615d8eTrac #32867: add maxima package information for Gentoo.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 12, 2022

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

6b5dc22Trac #32867: don't export MAXIMA_PREFIX.
25489cfTrac #32867: remove the MAXIMA sage_conf variable.
4655570Trac #32867: new spkg-configure.m4 for maxima.
88f5bb7Trac #32867: maxima package information for Cygwin.
511c067Trac #32867: disable maxima tests that require matrixexp.patch.
bf826bbTrac #32867: add maxima package information for Gentoo.

dimpase pushed a commit to dimpase/sage that referenced this issue May 5, 2023
… directory.

This just fixes a test. The problem was noticed on Arch where, due to
some other circumstances, maxima was executed from within /usr/sbin.
But all we know for sure is that maxima will live in PATH. So we
modify the test to look only for "maxima", and not its parent
directory.
dimpase pushed a commit to dimpase/sage that referenced this issue May 5, 2023
The ECL library is now contained in a separate maxima-fas package that
will pull in maxima itself.
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
This should work automatically, and the previous behavior is
incorrect if we intend to use the system copy of maxima.
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
This new spkg-configure.m4 checks for both a "maxima" executable, and
the usability of a "maxima" package by ECL. (The latter requires a
patched maxima until a new release with commit a0d7a43e523 is made.)
Notably absent for the moment is a version check, but that should be
okay: at the moment, the only system copies of maxima that support
being used as an ECL library are those that are patched, and the only
reason to patch them is if you intend to use them with SageMath. So in
short, if a distro patches their maxima to use it with sage, we expect
it to work with sage.

This commit also moves the SAGE_MAXIMA_FAS variable handling from
ECL's spkg-configure.m4 to maxima's (where it would have belonged in
the first place, if maxima had an spkg-configure.m4 at the time.)
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
This patch won't be present in a system copy of maxima (at least for
now), so we disable these tests to avoid spurious failures once we
start accepting maxima from the system.
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
We have some old code and an old test for a memory limit that was
present in ECL 12 years ago, worked around in ticket 6772. The memory
limit is no longer in place, and would not have applied to a non-ECL
maxima from the system anyway. Since the test itself is purposely
wasteful (to ensure that we can use lots of RAM), we take this
opportunity to remove it and speed up the test suite a bit.
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
… directory.

This just fixes a test. The problem was noticed on Arch where, due to
some other circumstances, maxima was executed from within /usr/sbin.
But all we know for sure is that maxima will live in PATH. So we
modify the test to look only for "maxima", and not its parent
directory.
dimpase pushed a commit to dimpase/sage that referenced this issue May 8, 2023
The ECL library is now contained in a separate maxima-fas package that
will pull in maxima itself.
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
This should work automatically, and the previous behavior is
incorrect if we intend to use the system copy of maxima.
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
This new spkg-configure.m4 checks for both a "maxima" executable, and
the usability of a "maxima" package by ECL. (The latter requires a
patched maxima until a new release with commit a0d7a43e523 is made.)
Notably absent for the moment is a version check, but that should be
okay: at the moment, the only system copies of maxima that support
being used as an ECL library are those that are patched, and the only
reason to patch them is if you intend to use them with SageMath. So in
short, if a distro patches their maxima to use it with sage, we expect
it to work with sage.

This commit also moves the SAGE_MAXIMA_FAS variable handling from
ECL's spkg-configure.m4 to maxima's (where it would have belonged in
the first place, if maxima had an spkg-configure.m4 at the time.)
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
This patch won't be present in a system copy of maxima (at least for
now), so we disable these tests to avoid spurious failures once we
start accepting maxima from the system.
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
We have some old code and an old test for a memory limit that was
present in ECL 12 years ago, worked around in ticket 6772. The memory
limit is no longer in place, and would not have applied to a non-ECL
maxima from the system anyway. Since the test itself is purposely
wasteful (to ensure that we can use lots of RAM), we take this
opportunity to remove it and speed up the test suite a bit.
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
… directory.

This just fixes a test. The problem was noticed on Arch where, due to
some other circumstances, maxima was executed from within /usr/sbin.
But all we know for sure is that maxima will live in PATH. So we
modify the test to look only for "maxima", and not its parent
directory.
dimpase pushed a commit to dimpase/sage that referenced this issue May 17, 2023
The ECL library is now contained in a separate maxima-fas package that
will pull in maxima itself.
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
This should work automatically, and the previous behavior is
incorrect if we intend to use the system copy of maxima.
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
This new spkg-configure.m4 checks for both a "maxima" executable, and
the usability of a "maxima" package by ECL. (The latter requires a
patched maxima until a new release with commit a0d7a43e523 is made.)
Notably absent for the moment is a version check, but that should be
okay: at the moment, the only system copies of maxima that support
being used as an ECL library are those that are patched, and the only
reason to patch them is if you intend to use them with SageMath. So in
short, if a distro patches their maxima to use it with sage, we expect
it to work with sage.

This commit also moves the SAGE_MAXIMA_FAS variable handling from
ECL's spkg-configure.m4 to maxima's (where it would have belonged in
the first place, if maxima had an spkg-configure.m4 at the time.)
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
This patch won't be present in a system copy of maxima (at least for
now), so we disable these tests to avoid spurious failures once we
start accepting maxima from the system.
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
We have some old code and an old test for a memory limit that was
present in ECL 12 years ago, worked around in ticket 6772. The memory
limit is no longer in place, and would not have applied to a non-ECL
maxima from the system anyway. Since the test itself is purposely
wasteful (to ensure that we can use lots of RAM), we take this
opportunity to remove it and speed up the test suite a bit.
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
… directory.

This just fixes a test. The problem was noticed on Arch where, due to
some other circumstances, maxima was executed from within /usr/sbin.
But all we know for sure is that maxima will live in PATH. So we
modify the test to look only for "maxima", and not its parent
directory.
dimpase pushed a commit to dimpase/sage that referenced this issue May 22, 2023
The ECL library is now contained in a separate maxima-fas package that
will pull in maxima itself.
@dimpase
Copy link
Member

dimpase commented Jun 26, 2023

done in #35615

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants