Upgrade the Readline spkg to 6.1

[sagetrac-cwitty]

Under Arch Linux, Sage fails to build, giving this error message:

bash: symbol lookup error: bash: undefined symbol: rl_filename_rewrite_hook

in the middle of the sqlite build (the next package built after readline). This can also happen on openSUSE Linux.

This is a new symbol that was added in readline 6.1; so I'm pretty sure the problem is because our readline 6.0 is missing that symbol, so trying to run Arch's /bin/bash with our LD_LIBRARY_PATH will fail.

We should upgrade our readline spkg to 6.1; I bet that would fix the problem.

Threads: sage-devel, sage-support.

Related: #9530, #9987

An updated .spkg can be found at
http://boxen.math.washington.edu/home/kirkby/readline-6.1.spkg
which is based on readline-6.0.p4.spkg from #9530.

CC: @TimDumol

Component: packages: standard

Author: David Kirkby

Reviewer: Florent Hivert, Leif Leonhardy, Jeroen Demeyer, Volker Braun

Merged: sage-4.6.1.alpha2

Issue created by migration from https://trac.sagemath.org/ticket/9523

[sagetrac-gostrc]

comment:1

Hello, I'm the maintainer of sage-mathematics in the AUR. I think I have worked around this issue with 4.5-2 by not building sage's libreadline, and as a result, using the system's readline :)

BTW, this error is supposedly worked around by sage, by checking if uname -r | grep ARCH returns anything. Which means that people with custom kernels are likely to be experiencing this problem.

[sagetrac-gostrc]

comment:2

Oh ya, +1 from me to upgrade sage's libreadline to 6.1 so it can get rid of the internal workarounds and the workaround in my PKGBUILD :P

[hivert]

comment:3

Just to let you now ! The very same problem occurs with the new openSuSE 11.3...

Florent

[qed777]

comment:4

Replying to @hivert:

Just to let you now ! The very same problem occurs with the new openSuSE 11.3...

Indeed, there's a recent thread about readline and openSUSE on sage-support.

[qed777]

comment:7

Replying to @qed777:

Replying to @hivert:

Just to let you now ! The very same problem occurs with the new openSuSE 11.3...

Indeed, there's a recent thread about readline and openSUSE on sage-support.

Another.

We should try to fix this in 4.6 or 4.6.1.

[nexttime]

comment:8

Note that this wouldn't happen with proper shared library versioning... :|

[qed777]

comment:9

Replying to @qed777:

Replying to @qed777:

Replying to @hivert:

Just to let you now ! The very same problem occurs with the new openSuSE 11.3...

Indeed, there's a recent thread about readline and openSUSE on sage-support.

Another.

And at AskSage.

[sagetrac-drkirkby]

comment:10

Somewhat related is #9987

Readline can definitely be built on AIX - see for example

http://www.perzl.org/aix/index.php?n=Main.Readline

so something is wrong in how Sage is using readline.

Dave

[sagetrac-drkirkby]

comment:13

I've updated the source code, and cleaned up the package a bit. If others want to clean it up further, feel free, but I don't want to spend a lot of time on this. It is not even causing any problems on any systems I'm using. The .spkg can be found here.

http://boxen.math.washington.edu/home/kirkby/patches/readline-6.1.spkg

I've checked the new .spkg actually builds on the following systems

• AIX 5.3 (my own RS/6000 7025 F50 'aixbox') I did not try to resolve Shared readline library uses a .so extension on AIX when it should be .a #9987
• HP-UX 11.11B (my own HP C3600 'hpbox')
• Linux (sage.math)
• OpenSolaris 06/2009 (my own Sun Ultra 27 'hawk' which is a Sage buildbot slave.)
• OS X (bsd.math)
• Solaris 10 SPARC ('mark' on skynet, which is a Sun Blade 2500)

I've checked that the whole of Sage builds, and passes all doctests

----------------------------------------------------------------------
All tests passed!
Total time for all tests: 1740.4 seconds
drkirkby@hawk:~/sage-4.6.rc0$ 

with the updated .spkg on only the following system.

• OpenSolaris (my own Sun Ultra 27 'hawk' which is a Sage buildbot slave)

I've not checked this on any of the systems which have been known to cause issues with readline (FreeBSD, ArchLinux, OpenSUSE etc).

Dave

[sagetrac-drkirkby]

comment:14

I just noticed I had not made all the necessary changes to a patch that was included in the .spkg. The patch would not apply cleanly to the update version of the source, so I had to do it manually, but it looks like I forgot a couple of bits.

Leave it with me.

[sagetrac-drkirkby]

Mercurial patch - adds a rather useless spkg-check file, cleans the package a little.

[sagetrac-drkirkby]

comment:15

Attachment: 9523-update-readline.patch.gz

This now needs review.

Please test, especially on platforms where there has been problems with readline, which appear to be many!

I've now added an spkg-check file and run make check as there is a check target, but it does not actually do anything useful at this time. Hopefully the readline developers will add some self-tests.

The package can be found at http://boxen.math.washington.edu/home/kirkby/patches/readline-6.1.spkg

Dave

[nexttime]

comment:53

Replying to @jdemeyer:

Why do we need to special-case OpenSUSE in the first place? I thought the whole point of this ticket was not to have any special cases any more.

:D If you convince the openSUSE and ArchLinux developers to never make bash depend on a newer libreadline than Sage ships...

Otherwise the problem remains or rearises with future releases of these OSs.

[nexttime]

comment:54

We could just always build Sage's readline and - before installing it - test if it works with /bin/bash by e.g.

if env LD_LIBRARY_PATH="." bash -c "echo 'Bash works with this version of readline.'"; then
    $MAKE install
    ...
else
    echo "Bash doesn't work with Sage's version of readline - using the system's one."
    # Can this cause trouble with *older* system libreadlines?
    # We still need a *development* version of readline btw.
    exit 0
fi

[jdemeyer]

comment:55

Replying to @nexttime:

Replying to @jdemeyer:

Why do we need to special-case OpenSUSE in the first place? I thought the whole point of this ticket was not to have any special cases any more.

:D If you convince the openSUSE and ArchLinux developers to never make bash depend on a newer libreadline than Sage ships...

Otherwise the problem remains or rearises with future releases of these OSs.

True, but this problem is not specific to these OSs and applies to every OS.

[jdemeyer]

Changed work issues from rebase to readline-6.0.p4.spkg (#9530) to none

[nexttime]

comment:56

Replying to @jdemeyer:

Replying to @nexttime:

Replying to @jdemeyer:

Why do we need to special-case OpenSUSE in the first place? I thought the whole point of this ticket was not to have any special cases any more.

:D If you convince the openSUSE and ArchLinux developers to never make bash depend on a newer libreadline than Sage ships...

Otherwise the problem remains or rearises with future releases of these OSs.

True, but this problem is not specific to these OSs and applies to every OS.

Not really. Others work fine, so I consider testing if bash works with our readline also an (implicit) special case.

Would you be happy with that?

(I think at least it doesn't hurt doing so, i.e. testing bash against Sage's readline before installing it. Better to give a concise error message than risking other weird build errors.)

[sagetrac-drkirkby]

comment:57

Replying to @jdemeyer:

Replying to @nexttime:

:D If you convince the openSUSE and ArchLinux developers to never make bash depend on a newer libreadline than Sage ships...

Otherwise the problem remains or rearises with future releases of these OSs.

True, but this problem is not specific to these OSs and applies to every OS.

It seems to be specific to these two Linux distributions - though perhaps there are others, as one Linux distro tends to be based on another. Mint is based on Ubuntu, which is itself based on Debian.

Most other distros don't ship with a bash that is dynamically linked to readline. It's never been a problem on Solaris or OS X either. Although I've never built Sage fully on either AIX or HP-UX, I'm not aware of any bash/readline issues on those operating systems either.

Unless someone is willing to set up an OpenSUSE system for people to test on, I can't really see how we can support the latest release.

http://wiki.sagemath.org/SupportedPlatforms#Linux

says 11.1 is supported, and 11.2 and 11.3 are known to be broken.

I simply don't have access to the hardware/software to test this.

William posted a few weeks ago he was wanting people to administer virtual machines. Unless someone is going to do this for the latest OpenSUSE, then I can't see how we can support it. I already admin two machines myself which are buildbot slavs (hawk and t2).

Dave

[nexttime]

comment:58

Replying to @sagetrac-drkirkby:

Unless someone is willing to set up an OpenSUSE system for people to test on, I can't really see how we can support the latest release.

http://wiki.sagemath.org/SupportedPlatforms#Linux

says 11.1 is supported, and 11.2 and 11.3 are known to be broken.

I simply don't have access to the hardware/software to test this.

Just provide an spkg and let the others test (and review) it... ;-)

(until we get more build slaves, running these distros.)

William posted a few weeks ago he was wanting people to administer virtual machines. Unless someone is going to do this for the latest OpenSUSE, then I can't see how we can support it. I already admin two machines myself which are buildbot slavs (hawk and t2).

---

As mentioned on sage-devel, a work-around for a Bash broken by Sage's readline is to set its RPATH or RUNPATH (with chrpath). We could give a hint to that in an error (or warning) message in case we detect installing our readline would break bash (but don't know for sure using the system's libreadline will work for us).

[jdemeyer]

comment:59

Replying to @nexttime:

Not really. Others work fine, so I consider testing if bash works with our readline also an (implicit) special case.

Would you be happy with that?

Well, I don't care too much about this, but if it's possible to test whether bash works, that certainly is a better solution.

[nexttime]

comment:60

Replying to @nexttime:

As mentioned on sage-devel, a work-around for a Bash broken by Sage's readline is to set its RPATH or RUNPATH (with chrpath). We could give a hint to that in an error (or warning) message in case we detect installing our readline would break bash (but don't know for sure using the system's libreadline will work for us).

s/set its RPATH/change an existing RPATH or RUNPATH/

or use patchelf, which also supports creating / adding such tags.

Another work-around is to set up a bash wrapper that (re)sets LD_LIBRARY_PATH... ;-)

[sagetrac-Koen]

comment:61

Replying to @sagetrac-drkirkby:

Unless someone is willing to set up an OpenSUSE system for people to test on, I can't really see how we can support the latest release.

http://wiki.sagemath.org/SupportedPlatforms#Linux

says 11.1 is supported, and 11.2 and 11.3 are known to be broken.

I simply don't have access to the hardware/software to test this.

I've been testing openSUSE lately - and theory and practice are completely reversed. Sage does not build on 11.1 due to readline 5.x being the default there. Whereas on openSUSE 11.2 and 11.3, Sage builds properly (a recent 4.6.1.rc0 snapshot).
However, I'm not sure how to test if my final Sage 'works' w.r.t. the readline/bash problem, so I will only claim that it builds and sage starts properly.

[sagetrac-Koen]

comment:62

Note: it might be reasonable to drop support for openSUSE 11.1, because it will stop receiving security updates after December 31st, 2010 (the release is 2 years old now).

[nexttime]

comment:63

Replying to @sagetrac-Koen:

Note: it might be reasonable to drop support for openSUSE 11.1, because it will stop receiving security updates after December 31st, 2010 (the release is 2 years old now).

We shouldn't have to drop support for that, installing / using our 6.1 package should work there as well.

(Bash would still use the system's 5.x version.)

We just have to fix / remove the old copying stuff... (and I would add the mentioned sanity check).

[sagetrac-drkirkby]

comment:64

Replying to @sagetrac-Koen:

Note: it might be reasonable to drop support for openSUSE 11.1, because it will stop receiving security updates after December 31st, 2010 (the release is 2 years old now).

It's a shame that support is dropped so soon in the Linux world - this contrasts widely with professional Unix systems like Solaris. Solaris 8 was released in February 2000 and will be supported until March 2012 (i.e. supported for 12 years). Similar patterns will be seen on AIX and HP-UX I expect.

Not everyone runs the latest version of the operating system. For many people, they don't update the OS until they buy a new computer. I consider myself pretty IT literate, but looking at my computers, many don't have the latest releases. I grew out of the habit of updating the OS because a new one came out.

Dave

[sagetrac-drkirkby]

comment:65

Replying to @nexttime:

Replying to @sagetrac-drkirkby:

Unless someone is willing to set up an OpenSUSE system for people to test on, I can't really see how we can support the latest release.

http://wiki.sagemath.org/SupportedPlatforms#Linux

says 11.1 is supported, and 11.2 and 11.3 are known to be broken.

I simply don't have access to the hardware/software to test this.

Just provide an spkg and let the others test (and review) it... ;-)

(until we get more build slaves, running these distros.)

I've done that. I created the package. It's now marked as "needs work" but it is going to need to be worked on by someone else.

In general, I very much like the approach taken by autoconf, where instead of having a huge lookup table detailing what version of what OS supports this function or that function, it actually tests the functionality. Overall that seems a far more logical approach to me, and seems to be what Jeroen is proposing. If we can test the functionality of bash, rather than having code that attempts to find a specific Linux release, then testing seems a better way forward.

Dave

[nexttime]

comment:66

Replying to @sagetrac-drkirkby:

I've done that. I created the package. It's now marked as "needs work" but it is going to need to be worked on by someone else.

Ok. If you're not going to change it further, I can do that in the next days.

In general, I very much like the approach taken by autoconf, where instead of having a huge lookup table detailing what version of what OS supports this function or that function, it actually tests the functionality. Overall that seems a far more logical approach to me, and seems to be what Jeroen is proposing. If we can test the functionality of bash, rather than having code that attempts to find a specific Linux release, then testing seems a better way forward.

Well, autotools, or the scripts their files are built from, have a lot of knowledge coded into them (like chess programs, or e.g. gcc, too), i.e. they also - at least partially - detect the system and make the choices based on that.

(And packages using autotools still have configure options like --with-package-xy=/path/to/package-xy, --with-included-package-xy and --with-system-package-xy, and lots of --disable-* and --enable-* one sometimes has to specify manually. Try e.g. building a "customized" version of a recent GCC, with dozens of settings also for GMP, MPFR, MPC, PPL and CLooG, probably other packages like gettext and zlib as well. Also, GCC does drop support of older platforms, OSs and architectures, due to a lack of developer resources.)

Feel free to extend Sage's configure (which really could do much more, setting appropriate environment variables [like your famous CFLAG64] etc.)...

But Sage is (also) a distro with many "foreign" packages, not just a program, and to make things work together, we have to make choices normally a user would make - manually - for each of Sage's packages.

The user can still fake Sage's readline was already installed such that Sage will use the system's one, but that requires some more reading and typing than just issuing make (or double-clicking the Makefile). ;-)

Same for other packages.

[jdemeyer]

comment:67

Thanks for the work on this ticket. Do you think it is realistic to fix this spkg in the next days? If not, I will unmerge this spkg and release Sage 4.6.1 with the old readline spkg (which also has problems).

[jdemeyer]

comment:68

Additional note: I don't mind merging a partially-fixed readline 6.1 (with some issues remaining) if there is a clear improvement over the old readline 6.0.p4 spkg. In that case, we should try to converge on a 6.1 spkg which can get positive_review and leave further changes to a different ticket.

[sagetrac-drkirkby]

comment:69

Replying to @sagetrac-Koen:

Replying to @sagetrac-drkirkby:

Unless someone is willing to set up an OpenSUSE system for people to test on, I can't really see how we can support the latest release.

http://wiki.sagemath.org/SupportedPlatforms#Linux

says 11.1 is supported, and 11.2 and 11.3 are known to be broken.

I simply don't have access to the hardware/software to test this.

I've been testing openSUSE lately - and theory and practice are completely reversed. Sage does not build on 11.1 due to readline 5.x being the default there. Whereas on openSUSE 11.2 and 11.3, Sage builds properly (a recent 4.6.1.rc0 snapshot).
However, I'm not sure how to test if my final Sage 'works' w.r.t. the readline/bash problem, so I will only claim that it builds and sage starts properly.

This is a can of worms. Sage certainly was built on 11.1 on 21^st October.

http://build.sagemath.org/sage/builders/openSUSE%2011.1-64%20%28menas%29

I'll leave others to judge if my package is better or worst than the present one. Obviously if someone can improve the readline package soon, it would be good to get an improved version in Sage. But if nobody has the time/resources to do so, then perhaps merging my 6.1 will be preferable to leaving the old one.

Dave

[jdemeyer]

comment:70

Replying to @sagetrac-drkirkby:

But if nobody has the time/resources to do so, then perhaps merging my 6.1 will be preferable to leaving the old one.

With these words, I propose the current spkg http://boxen.math.washington.edu/home/kirkby/readline-6.1.spkg] for review.

[vbraun]

comment:71

This ticket certainly illustrates why screwing with LD_PRELOAD / LD_LIBRARY_PATH is considered bad practice for any nontrivial project. The imho only correct fix is to explicitly set the RPATH/RUNPATH in all of Sage's binaries, and not set LD_LIBRARY_PATH. But then, thats for another ticket...

As far as readline is concerned, I think the current state is a definite improvement. Since there is really no remaining issue that can be fixed easy, I give this a positive review so that we can go ahead with releasing Sage-4.6.1.

[vbraun]

Changed reviewer from Florent Hivert, Leif Leonhardy, Jeroen Demeyer to Florent Hivert, Leif Leonhardy, Jeroen Demeyer, Volker Braun