sdh_configure should use bash

[jdemeyer]

Some configure scripts are buggy when they are run with shells other than bash. See #23451 for example. Since Sage requires bash anyway, we can easily work around this by forcing bash to run configure.

CC: @embray @dimpase

Component: porting

Author: Jeroen Demeyer

Branch: f0e2bd8

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/24491

[jdemeyer]

Branch: u/jdemeyer/sdh_configure_should_use_bash

[jdemeyer]

Commit: 36eb05d

[jdemeyer]

New commits:

36eb05d

sdh_configure should always use bash

[jdemeyer]

comment:3

This doesn't actually work due to way how configure works.

[sagetrac-git]

Changed commit from 36eb05d to b8d5ca2

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b8d5ca2

sdh_configure should always use bash

[dimpase]

comment:5

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

[jdemeyer]

comment:7

Replying to @dimpase:

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

Given that $SHELL is bash, no it should not.

[dimpase]

comment:8

Replying to @jdemeyer:

Replying to @dimpase:

Shouldn't

export CONFIG_SHELL="$SHELL"

rather be

export CONFIG_SHELL=`command -v bash`

Given that $SHELL is bash, no it should not.

Why would $SHELL always be bash? E.g. in the context of #23451 it can be dash, no?

[jdemeyer]

comment:9

Replying to @dimpase:

Why would $SHELL always be bash?

Because the Sage build scripts are always run with bash.

[tornaria]

comment:10

Are there any other packages with non-portable configure scripts, other than the two mentioned in #23451? I think I was able to build all of sage using dash 0.5.9 with only those two "fixed".

Using /bin/sh when possible is a way to give extra testing to configure scripts, which are meant to ensure portability of software. Sage packages some mathematical software that is not included in distros, so they won't get exposure to diverse environments otherwise.

Also, a script that works with several different shells is a script that is less likely to break when bash changes, or with older versions of bash, etc.

---

Anyway, maybe it is a good idea to do

if [ -z "$CONFIG_SHELL" ]; then
    export CONFIG_SHELL="$SHELL"
fi

to allow overriding of $CONFIG_SHELL ?

[dimpase]

comment:11

Replying to @jdemeyer:

Replying to @dimpase:

Why would $SHELL always be bash?

Because the Sage build scripts are always run with bash.

I don't know what you exactly mean.
Just check the 1st line of the top ./configure. There is no bash there.

[kiwifb]

comment:12

Some autotool scripts will also break if you use CONFIG_SHELL=bash I don't know if there are any in sage but I have experienced it. Real autotool script usually expect sh but a lot of the time you are lucky and can use bash (gcc is a case in point).

My experience comes from AIX where sh is a real separate program rather than bash or dash called with a special profile. AIX's sh is so slow that it will take several days to compile gcc if you don't use CONFIG_SHELL=bash. In that light it is tempting to use bash all the time, until things starts to fail left and right in some libtool scripts later on.

[jdemeyer]

comment:13

Replying to @dimpase:

Just check the 1st line of the top ./configure.

I don't mean configure. I mean the scripts in build/bin.

[dimpase]

comment:14

Replying to @jdemeyer:

Replying to @dimpase:

Just check the 1st line of the top ./configure.

I don't mean configure. I mean the scripts in build/bin.

But this does not guarantee that sdh_configure is only launched from there.
Someone could source the sdh_configure in a non-bash session and shoot themselves in the foot...

[jdemeyer]

comment:15

Replying to @kiwifb:

Some autotool scripts will also break if you use CONFIG_SHELL=bash I don't know if there are any in sage but I have experienced it. Real autotool script usually expect sh but a lot of the time you are lucky and can use bash (gcc is a case in point).

On some systems, /bin/sh is bash:

jdemeyer@sage4:~$ /bin/sh --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Given that bash is meant to be POSIX-compliant, it should not be a problem to force the use of bash.

[jdemeyer]

comment:16

Replying to @tornaria:

Using /bin/sh when possible is a way to give extra testing to configure scripts, which are meant to ensure portability of software.

I disagree completely with this point. It's not the goal of Sage to use our users for testing the portability of configure scripts.

We should ensure that Sage can be installed on as many systems as possible. I claim that using bash for configure scripts is more portable than using /bin/sh. So that is what we should do.

[sagetrac-git]

Changed commit from b8d5ca2 to d103ebf

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d103ebf

sdh_configure should always use bash

[sagetrac-git]

Changed commit from d103ebf to f0e2bd8

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f0e2bd8

sdh_configure should always use bash

[kiwifb]

comment:19

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

[jdemeyer]

comment:20

Replying to @kiwifb:

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

I always assumed that bash is essentially sh with extra features (a bit like C++ is C with extra features). Meaning that if something works with sh, it almost certainly works with bash. Of course there are edge cases where it might not work, but I doubt that it is as much a problem as you think.

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:21

My experience with FreeBSD (with /bin/sh not being bash at all) tells me that making /bin/sh a link to bash allows Sage to build just fine. It breaks other components of the OS, naturally, but it's beside the point, which is that it's actually safe to use bash everywhere where there is /bin/sh (at least the one on FreeBSD) invocation in Sage.

[embray]

comment:22

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

[kiwifb]

comment:23

Replying to @jdemeyer:

Replying to @kiwifb:

And on some system sh is actually dash. There are differences between bash, dash and sh (as one would expect). When sh is actually bash for example it is not just a link. When bash is invoked as sh a special profile is loaded, some bash functions are defined to match sh requirements and so on. Probably similarly for dash. If you just CONFIG_SHELL=bash everywhere you will get a surprise sooner rather than later because those functions won't be there.

I always assumed that bash is essentially sh with extra features (a bit like C++ is C with extra features). Meaning that if something works with sh, it almost certainly works with bash. Of course there are edge cases where it might not work, but I doubt that it is as much a problem as you think.

Been there, done that, got the T-shirt. Proof is in the pudding, someone should do a complete build. My experience is that a libtool component will break sooner rather than later.

[kiwifb]

comment:24

Replying to @embray:

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

GNU make is needed. Once upon a time (in 2008) me and a debian developper at the time tried to patch packages to be able to use other make. We did go exactly nowhere. Fully autotooled packages are probably OK, it is all the custom ones that are troublesome - and it is so easy to use GNUism in make because they are so useful...

[kiwifb]

comment:25

Not much documentation on what bash does when it is invoked as sh. According to the doc it change mainly two things: the files it reads at start up and it is basically run with the --posix option. So adding --posix as an option may be enough to get out of all the potential problems. I wish I had an opportunity to test that when I found the various problems I mentioned earlier.

[vbraun]

Changed branch from u/jdemeyer/sdh_configure_should_use_bash to f0e2bd8

[embray]

comment:27

Replying to @kiwifb:

Replying to @embray:

+1

Related question I've been wondering about for a while (while we're discussing it here): Do we require GNU make for Sage, or should any POSIX-compatible make work? I would very much like to make use of GNU-specific features (e.g. #21524) and I'm not 100% sure if all of Sage's packages will work with a generic make but it's never been clear to me if that's a requirement or not.

GNU make is needed. Once upon a time (in 2008) me and a debian developper at the time tried to patch packages to be able to use other make. We did go exactly nowhere. Fully autotooled packages are probably OK, it is all the custom ones that are troublesome - and it is so easy to use GNUism in make because they are so useful...

Good, I'm glad. While this is a problem autotools is supposed to solve that's not a universal solution anyways.

[embray]

Changed commit from f0e2bd8 to none