libgfortran *must* get shipped with the Sage binaries

[williamstein]

I'm suddenly very concerned that Sage binaries won't work at all on computers without libgfortran.so installed. Does Sage even start up on such a box?

Yep.  If I take one of the Sage build machines, remove libgfortran, then start Sage I get:

$ sage
BOOM!

.... ImportError: libgfortran.so.3: cannot open shared object file: No such file or directory

----------------

Not good, since most Linux installs won't have libgfortran.  If I then reinstall gfortran, and copy libgfortran.so  to SAGE_ROOT/local/lib/, then uninstall gfortran, then Sage works fine again.

cp /usr/lib/libgfortran.so.3.0.0 local/lib/libgfortran.so.3

wstein@ubuntu910-64:/tmp/wstein/farm/sage-4.3.1$ ./sage
----------------------------------------------------------------------
| Sage Version 4.3.1, Release Date: 2010-01-20                       |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
sage: import scipy.linalg
sage:

CC: @jdemeyer @sagetrac-tmonteil

Component: build

Keywords: days43

Issue created by migration from https://trac.sagemath.org/ticket/8049

[sagetrac-drkirkby]

comment:1

I made the point the other day, that the library should be included in the binary, along with C++ library.

Dave

[sagetrac-drkirkby]

comment:2

Actually, I think the C library too. The C library is very small anyway. If Sage is linked with a new library, and someone has an older one on their system, it could be a problem.

'ldd' might be useful to get the location on Solaris, Linux or HP-UX, but not on OS X. Perhaps there is a more widely supported command, but I don't know of it.

The configure script for gcc has these options.

Fine tuning of the installation directories:
  --bindir=DIR           user executables [EPREFIX/bin]
  --libdir=DIR           object code libraries [EPREFIX/lib]

so while one could guess at the location of libraries from the location of the gfortran binary, it is not guaranteed to be right.

On Solaris, in 64-bit mode, it will be in a subdirectory too

drkirkby@hawk:~$ find /usr/local -name libgfortran.so
/usr/local/gcc-4.3.4/lib/amd64/libgfortran.so
/usr/local/gcc-4.3.4/lib/libgfortran.so
/usr/local/gcc-4.5-20100114/lib/amd64/libgfortran.so
/usr/local/gcc-4.5-20100114/lib/libgfortran.so
/usr/local/lib/amd64/libgfortran.so
/usr/local/lib/libgfortran.so
/usr/local/gcc-4.3.4-GNU-assembler-Sun-linker/lib/libgfortran.so
/usr/local/gcc-4.3.4-GNU-assembler-Sun-linker/lib/amd64/libgfortran.so

Dave

[dimpase]

comment:3

Replying to @sagetrac-drkirkby:

Actually, I think the C library too. The C library is very small anyway. If Sage is linked with a new library, and someone has an older one on their system, it could be a problem.

'ldd' might be useful to get the location on Solaris, Linux or HP-UX, but not on OS X. Perhaps there is a more widely supported command, but I don't know of it.

The OSX analog of ldd is called otool (otool -L), to be precise:
{{
$ otool -L liblinboxsage.dylib
liblinboxsage.dylib:
/tmp/sage-4.3.1/local/lib/liblinboxsage.0.dylib (compatibility version 1.0.0, current version 1.0.0)
libntl.dylib (compatibility version 0.0.0, current version 0.0.0)
/tmp/sage-4.3.1/local/lib/libgmpxx.3.dylib (compatibility version 5.0.0, current version 5.6.0)
/tmp/sage-4.3.1/local/lib/libgmp.3.dylib (compatibility version 8.0.0, current version 8.6.0)
/tmp/sage-4.3.1/local/lib/libgivaro.0.dylib (compatibility version 1.0.0, current version 1.2.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0)
/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
}}

Just in case,
Dima

[williamstein]

comment:4

I fixed this by writing a custom dist script that I use for building binaries.

[sagetrac-drkirkby]

comment:5

Does this do C and C++, or just Fortran? You can be 100% sure that the user needs the C and C++ libraries. You might well get away with it on most linux distros if gcc is installed. You certainly can't get away with it on Solaris, where a recent gcc is not installed unless someone has taken the steps to do so.

Dave

[sagetrac-startakovsky]

comment:6

Hi, this is my first ticket, I just joined trac. I have the following error when I try to import numpy:

ImportError: libgfortran.so.3: cannot open shared object file: No such file or directory

I have the latest releasted Sage Binary (5.3) installed on my linux machine, running Xubuntu.

[dimpase]

comment:7

Replying to @sagetrac-startakovsky:

Hi, this is my first ticket, I just joined trac. I have the following error when I try to import numpy:

ImportError: libgfortran.so.3: cannot open shared object file: No such file or directory

I have the latest releasted Sage Binary (5.3) installed on my linux machine, running Xubuntu.

To help your particular installation, you can just do

sudo apt-get install libgfortran3

We'll have to look into this more carefully, though.

[koffie]

comment:8

It would be interesting to know whether this problem already occurred in sage 4.8 . The last thing I know that changed in relation to fortran is #12369 where the gcc package was added and the old fortran package removed. I think it would be good to open a new ticket for this issue, leaving this ticket as an "archive" for the old issue.

[sagetrac-tmonteil]

Changed keywords from none to days43

[sagetrac-tmonteil]

comment:9

Hi, is it possible to re-open this ticket ? Currently, libgfortran is not shipped with Sage.

[jdemeyer]

comment:10

Replying to @sagetrac-tmonteil:

Hi, is it possible to re-open this ticket ?

It is certainly possible to re-open this ticket, but you should specify in more detail why.

In sage-5.3, libgfortran is shipped with the official Sage binaries, except for Ubuntu 12.04. In sage-5.4, libgfortran will also be shipped with the Ubuntu 12.04 binaries, see #13515.

[sagetrac-tcoffee]

comment:11

Correct me if this issue has been superseded, but I just installed this binary

http://boxen.math.washington.edu/home/sagemath/sage-mirror/linux/64bit/sage-6.3-x86_64-Linux-Ubuntu_10.04_x86_64.tar.gz

and it appears not to include libgfortran.

I obtained the same ImportError as above when attempting to import numpy, and installing libgfortran3 fixed the problem.

[dimpase]

comment:12

Replying to @sagetrac-tcoffee:

Correct me if this issue has been superseded, but I just installed this binary

http://boxen.math.washington.edu/home/sagemath/sage-mirror/linux/64bit/sage-6.3-x86_64-Linux-Ubuntu_10.04_x86_64.tar.gz

and it appears not to include libgfortran.

I imagine libgfortran comes from the gcc spkg nowadays, and this binary has no trace of gcc in it.

[jdemeyer]

comment:13

Yes, binaries should be built with SAGE_INSTALL_GCC=yes, but apparently that's not the case.

[nexttime]

comment:14

Replying to @jdemeyer:

Yes, binaries should be built with SAGE_INSTALL_GCC=yes, but apparently that's not the case.

Does that make sense for distros with a more recent toolchain than Sage currently ships?

[jdemeyer]

comment:15

Replying to @nexttime:

Does that make sense for distros with a more recent toolchain than Sage currently ships?

Yes, because you need to ship the libraries that you link against, such as libgfortran.so.

[nexttime]

comment:16

Replying to @jdemeyer:

Replying to @nexttime:

Does that make sense for distros with a more recent toolchain than Sage currently ships?

Yes, because you need to ship the libraries that you link against, such as libgfortran.so.

Well, these should be the "native" ones from the distro the binary was built on (and for).

So unless the distro's toolchain is exceptionally outdated or broken (I also wouldn't say Lucid's is), I think we shouldn't SAGE_INSTALL_GCC.

If we ship older libraries than the distro by default comes with / uses, we're IMHO just asking for trouble. (Not all of them are properly versioned w.r.t. dynamically loading the "correct" one, IIRC. So running other applications from within the Sage environment might fail.)

Requiring the user installing a Sage binary to eventually also install some stuff with the distro's package manager first isn't too much to ask for, provided that's sufficiently documented and/or a (start-up) script tells her/him to do so...

After all, the whole point of building binaries for specific distros is that we know which, or can expect that such and such (versions of) libraries and programs are present (or at least available through the package manager).

[jdemeyer]

comment:17

Replying to @nexttime:

or at least available through the package manager

That's exactly the whole point here, this isn't about versions. We do not want that users need to install something with their package manager. Sage binaries should work "out of the box".

[nexttime]

comment:18

Replying to @jdemeyer:

Replying to @nexttime:

or at least available through the package manager

That's exactly the whole point here, this isn't about versions. We do not want that users need to install something with their package manager. Sage binaries should work "out of the box".

"Installing [with the distro's package manager]" (usually) doesn't mean building/compiling something, which is the point of bdists.

The Sage binary for Foonux 17.12 shouldn't contain any part of that distro, because that'd just be redundant (not to mention potentially bypassing security updates etc.).

But my main point was that using Sage's GCC for building bdists in general is not the same as shipping the distro's libraries -- the former may cause further (or different) trouble. So if you think we should ship libgfortran, just copy the system's one into $SAGE_LOCAL/lib; no need to install Sage's GCC just for that reason, whose libgfortran might (or typically will) be even older than the system's.

[nexttime]

comment:19

P.S.: If you want to run Sage really "out of the box", use one of the Live CDs (or probably the VirtualBox image), or SMC... ;-)

The only flaw with the bdists currently is that Sage apparently doesn't check on (first) start-up whether its prerequisites are present, instead a probably mysterious or hard to understand error might(!) occur. That's the bug.

[jdemeyer]

comment:20

Replying to @nexttime:

"Installing [with the distro's package manager]" (usually) doesn't mean building/compiling something, which is the point of bdists.

The Sage binary for Foonux 17.12 shouldn't contain any part of that distro, because that'd just be redundant (not to mention potentially bypassing security updates etc.).

But my main point was that using Sage's GCC for building bdists in general is not the same as shipping the distro's libraries -- the former may cause further (or different) trouble. So if you think we should ship libgfortran, just copy the system's one into $SAGE_LOCAL/lib; no need to install Sage's GCC just for that reason, whose libgfortran might (or typically will) be even older than the system's.

This discussion should really be moved to sage-release instead of this ticket.

[nexttime]

comment:21

Replying to @jdemeyer:

This discussion should really be moved to sage-release instead of this ticket.

I was thinking of replying there, but thought it was more on topic (or had the right audience) here.

We should probably invite fbissey, Snark and Jan Groenewald et al. to the party, in case they're not already cc'ed (haven't checked).

[kcrisman]

comment:22

Once again, lack of fortran or lapack or something causes trouble - see this discussion, which references this ticket.

[jdemeyer]

comment:23

Replying to @kcrisman:

Once again, lack of fortran or lapack or something causes trouble - see this discussion, which references this ticket.

Not once again, it's the same issue as [comment:13]. It has been fixed in the sense that the next binaries which will be released will not have this problem.

[sagetrac-tmonteil]

comment:24

See also this recent question on ask.sagemath.org showing that the problem still exists with 6.3 Ubuntu binary.

[jdemeyer]

comment:25

Replying to @sagetrac-tmonteil:

See also this recent question on ask.sagemath.org showing that the problem still exists with 6.3 Ubuntu binary.

Yes of course it does, in case you haven't read the recent comments: It has been fixed in the sense that the next binaries which will be released will not have this problem.