maxima fails to (natively) build using ecl

[dkwo]

Is this a new report?

Yes

System Info

6.5 series kernel on apple silicon (aarch64), as well as Void 6.5.5_2 x86_64 GenuineIntel uptodate rrFFFFFFFFFFFFFF

Package(s) Affected

maxima-5.47.0

Does a report exist for this bug with the project's home (upstream) and/or another distro?

https://sourceforge.net/p/maxima/bugs/4188/

but this is possibly due to our own https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/use-shared-library.patch

Expected behaviour

The prior version 5.46.0 builds fine.

Actual behaviour

In the do_build phase:

Making all in info
make[2] Entering directory '/builddir/maxima-5.47.0/doc/info'
make[3] Entering directory '/builddir/maxima-5.47.0/doc/info'
/usr/bin/env perl build_index.pl maxima.info ':crlf' > maxima-index.lisp
../../maxima-local --batch-lisp=build-html-index.lisp
/builddir/maxima-5.47.0/src/binary-ecl/maxima: error while loading shared libraries: binary-ecl/libmaxima-ecl.so: cannot open shared object file: No such file or directory
make[3]: *** [Makefile:1653: maxima-index-html.lisp] Error 127

Steps to reproduce

build natively on any of those archs using ./xbps-src pkg -o ecl,~sbcl maxima

[dkwo]

Downgrading to 5.46.0 fixes the issue.

[dkwo]

@tornaria is downgrading an option? will it interfere with sage?

[dkwo]

./xbps-src pkg -o ecl maxima builds fine on x86_64.
on a side note, we carry matrixexp.patch (which sagemath does not, but they carry infodir.patch which we don't).

[tornaria]

./xbps-src pkg -o ecl maxima builds fine on x86_64.

Not sure I understand what you mean. The default for x86_64 is ecl + sbcl. For aarch64 the default is only ecl. The sbcl binary is faster, so when building ecl + sbcl the installer will set sbcl as the default. I.e., when you run just maxima you are running sbcl; if you run maxima -l ecl you are running ecl. The ecl binary we move to the maxima-ecl package, so if building only ecl maybe maxima won't work without the maxima-ecl package installed (can you check that? if so we can try to do something about it).

OTOH, when you say "downgrading to 5.46.0 fixes the issue", does that mean that you rebuilt using 5.46.0, or that you used an old xbps file? If the latter, maybe the issue is not with maxima itself, but with some change in ecl which was recently updated. Maybe there is something wrong with ecl on aarch64? Can you try different combinations:

• does the maxima 5.47.0 that is cross built (i.e. the one in the repos) work?
• what if you rebuild ecl natively, then build maxima natively, does that change something?

on a side note, we carry matrixexp.patch (which sagemath does not, but they carry infodir.patch which we don't).

I don't think the infodir.patch does anything outside windows. I believe we carry matrixexp.patch because its still a actual bug worth fixing. IIRC the sagemath crowd decided to workaround the bug in sage itself and thus dropped the patch (since most maxima available in distros have the bug).

[dkwo]

Let me clarify: the error was in do_build phase of maxima itself, not maxima-ecl.
Even on x86_64, ./xbps-src pkg -o ecl,~sbcl maxima fails in the same way, so this is not aarch64-related, rather ecl-related.
After rebuilding ecl natively, nothing changes.
maxima is currently nocross (not sure why), so I was rebuilding it with 5.46.0 and confirmed it builds fine.
Thanks for the clarification about patches.

does this make sense?

[robert-dodier]

Hi, I'm a Maxima developer and I'm working on the upstream bug report. @tornaria you mentioned matrixexp.patch; where can I find that patch? I will take a look at merging it into Maxima.

[dkwo]

Thanks for helping. The patches are here: https://github.com/void-linux/void-packages/tree/master/srcpkgs/maxima/patches

[gunterkoenigsmann]

Might the problem be that we use the (potentially cross-built) Maxima during the build process, possibly not telling it where to find it's libraries?

[robert-dodier]

@dkwo Thanks for linking the patches, I will take a look at them.

About libmaxima-ecl.so, I seem to recall now that it's part of the Maxima installation for Sage. Is the Maxima build machinery that you are working with somehow derived from the stuff for Sage? If so, I guess my advice is to start over with a vanilla installation. Is interoperability with Sage a requirement for a Maxima-only installation? I don't know what the requirements for Void Linux are, you can tell me.

[gunterkoenigsmann]

Are you cross-compiling for the arm platform?
AFAIK the build process for the current Maxima involves starting Maxima...

[dkwo]

@gunterkoenigsmann I'm building natively, both for the x86_64 and aarch64 archs.

@robert-dodier I'm building just maxima here, not as part of sage. it's downloaded from ${SOURCEFORGE_SITE}/maxima/maxima-${version}.tar.gz and built using the gnu-configure build-style in https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/template (the only modification are the patches I linked before, as well as the

post_configure() {
	# do not rebuild these files if they exist
	touch -c doc/info/*.html
	touch -c doc/info/maxima.info*
	touch -c doc/info/maxima_toc.html
	touch -c interfaces/xmaxima/doc/xmaxima.html
}

which I'm not sure is relevant here.

Let me emphasize that the failure only happens with maxima 5.47 (while 5.46 builds fine) and only when building with ecl (if using sbcl, it builds fine).

[dkwo]

the gnu-configure build style runs standard

do_configure() {
	: ${configure_script:=./configure}

	export lt_cv_sys_lib_dlsearch_path_spec="/usr/lib64 /usr/lib32 /usr/lib /lib /usr/local/lib"
	${configure_script} ${configure_args}
}
do_build() {
	: ${make_cmd:=make}

	export lt_cv_sys_lib_dlsearch_path_spec="/usr/lib64 /usr/lib32 /usr/lib /lib /usr/local/lib"
	${make_cmd} ${makejobs} ${make_build_args} ${make_build_target}
}

[dkwo]

Removing all the patches and the post_configure phase still fails.

[dkwo]

let me attach the config.log
maxima.config.log
and the build log
maxima-build.log

[robert-dodier]

Thanks for the info. Sorry for the trouble, but can you post the content of src/autoconf-variables.lisp also?

[robert-dodier]

By the way, FWIW, I just built current Maxima from Git with ECL 23.9.9 on an Ubuntu 16.04 system. ldd reports:

$ ldd src/binary-ecl/maxima
	linux-vdso.so.1 =>  (0x00007fff35be6000)
	libecl.so.23.9 => /usr/local/lib/libecl.so.23.9 (0x00007f1d77502000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1d771f9000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1d76e2f000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f1d76baf000)
	libgc.so.1 => /usr/lib/x86_64-linux-gnu/libgc.so.1 (0x00007f1d7693d000)
	libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f1d76735000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1d76518000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1d76314000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1d77aea000)

I am more than a little puzzled by the presence of libmaxima-ecl.so in the output you cited ... I don't understand how the reference to that library is being generated.

[dkwo]

No worries. Here 's my src/autoconf-variables.lisp:

$ cat masterdir/builddir/maxima-5.47.0/src/autoconf-variables.lisp 
; -*- Lisp -*-
(in-package :maxima)

(defparameter *autoconf-prefix* "/usr")
(defparameter *autoconf-exec_prefix* "/usr")
(defparameter *autoconf-package* "maxima")
(defparameter *autoconf-version* "5.47.0")
(defparameter *autoconf-libdir* "/usr/lib64")
(defparameter *autoconf-libexecdir* "/usr/libexec")
(defparameter *autoconf-datadir* "/usr/share")
(defparameter *autoconf-infodir* "/usr/share/info")
(defparameter *autoconf-host* "x86_64-unknown-linux-gnu")
;; This variable is kept for backwards compatibility reasons:
;; We seem to be in the fortunate position that we sometimes need to check for windows.
;; But at least until dec 2015 we didn't need to check for a specific windows flavour.
(defparameter *autoconf-win32* "false")
(defparameter *autoconf-windows* "false")
(defparameter *autoconf-ld-flags* "-Wl,-z,relro -Wl,-z,now -Wl,--as-needed    ")

;; This will be T if this was a lisp-only build
(defparameter *autoconf-lisp-only-build* (eq t 'nil))
 
(defparameter *maxima-source-root* "/builddir/maxima-5.47.0")
(defparameter *maxima-default-layout-autotools* "true")

Do you think it makes sense to also upstream this patch? https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/handle-multiple-ldflags.patch

Actually, could this patch be creating troubles? https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/use-shared-library.patch

[dkwo]

Yes, after removing the use-shared-library.patch the build goes through! it fails at a later stage, but that's expected:

=> maxima-5.47.0_3: running post_build ...
patchelf: getting info about 'src/binary-ecl/libmaxima-ecl.so': No such file or directory
=> ERROR: maxima-5.47.0_3: post_build: 'patchelf --remove-rpath src/binary-ecl/libmaxima-ecl.so' exited with 1
=> ERROR:   in post_build() at srcpkgs/maxima/template:55

@tornaria can you comment?

[dkwo]

removing that patch, as well as

--- a/srcpkgs/maxima/template
+++ b/srcpkgs/maxima/template
@@ -52,7 +52,6 @@ post_configure() {
 post_build() {
        if [ "$build_option_ecl" ]; then
                # everything will go in the same directory, use rpath=$ORIGIN
-               patchelf --remove-rpath src/binary-ecl/libmaxima-ecl.so
                patchelf --set-rpath \$ORIGIN \
                        src/binary-ecl/{maxima,maxima.fas}
                patchelf \
@@ -135,7 +134,6 @@ maxima-ecl_package() {
        pkg_install() {
                BINARY_ECL=/usr/lib/maxima/${version}/binary-ecl
                vmove ${BINARY_ECL}
-               vinstall src/binary-ecl/libmaxima-ecl.so 755 ${BINARY_ECL}
                vinstall src/binary-ecl/maxima.fas 755 ${BINARY_ECL}
 
                # symlink maxima.fas in ECLDIR

passes checks with -Q.

[tornaria]

The requirement for us is:

• running maxima standalone binary works (of course)
• in ecl, the following works: (require :maxima)

This is achieved by creating both a binary executable maxima and a binary module maxima.fas. They are both installed in /usr/lib/maxima/5.47.0/binary-ecl/.

This is supported by maxima just fine.
However, the two files produced maxima and maxima.fas both contain the whole of maxima code.

Now let me explain the purpose of the use-shared-library.patch and the libmaxima-ecl.so dynamic library. By default, maxima will statically link everything so all the code is duplicated between maxima and maxima.fas. What my patch does is to place all the code in libmaxima-ecl.so and have maxima and maxima.fas dynamically load that one.

So now I think I have a clue of what is the trouble. It seems maxima needs to run maxima to finish building. When maxima is built for sbcl + ecl, everything is fine (maxima is run via sbcl). But when maxima is built for ecl, the dynamic library is not found. We may be able to fix that so the maxima executable actually works at build time (maybe using LD_LIBRARY_PATH ?).

[dkwo]

@robert-dodier Thanks for helping with this, and sorry for bothering you.

[robert-dodier]

@dkwo No problem, I'm glad to help. Looks like you have figured out what the basic problem is, good luck from here on out.

[tornaria]

@dkwo No problem, I'm glad to help. Looks like you have figured out what the basic problem is, good luck from here on out.

@robert-dodier sorry for the confusion with my bad use-shared-library.patch. I've since improved it so the binary that is built works without having to fiddle with patchelf.

Please let me know if I can help to upstream some of the patches we have in https://github.com/tornaria/void-packages/tree/maxima/srcpkgs/maxima/patches. I think at least the following would be quite reasonable to upstream:

• handle-multiple-ldflags.patch Handle multiple LDFLAGS.
• matrixexp.patch Fix error in matrix exponentiation.
• maxima-sbcl-with-gmp.patch Patch maxima initialization for sbcl so it uses gmp for arithmetic.
• musl-round.patch Add a bit more room for rounding errors which cause two tests to fail on x86_64-musl.

Also a comment about documentation. The tarball includes all the documentation so it is not necessary to rebuild. However:

• Configure will write makefiles and maybe a version file, and building documentation depends on the makefiles so they are rebuild nonetheless. I workaround that by touching the target files after configure.
• There's an option to configure to skip building documentation. Unfortunately, it also disables installing the documentation, so it doesn't help us.

[gunterkoenigsmann]

If you were willing to file them at https://sourceforge.net/p/maxima/patches/ it would make them more visible for the Maxima developers