make SPACK_SHELL detection more robust

[carns]

See discussions in #7475 and #8443. This patch attempts to make setup-env's shell detection work correctly when it is sourced within a shell script environment. The target use cases are continuous integration environments or job scheduler scripts.

Test case:

#!/bin/bash
  
echo step: sourcing setup-env.sh
. /home/pcarns/working/src/spack/share/spack/setup-env.sh
echo step: loading mpich
spack load -r mpich
echo step: listing modules
module list

Without this PR, it produces the following output:

pcarns@carns-x1:~$ ./foo.sh 
step: sourcing setup-env.sh
step: loading mpich
init.c(379):ERROR:109: Unknown shell type 'foo.sh'
step: listing modules
init.c(379):ERROR:109: Unknown shell type 'foo.sh'

With this PR applied, it produces the following output:

pcarns@carns-x1:~$ ./foo.sh 
step: sourcing setup-env.sh
step: loading mpich
step: listing modules
Currently Loaded Modulefiles:
  1) mpich-3.2.1-gcc-8.2.0-36v3ogy

[tgamblin]

It looks like the script here is from a GPL project, so I'm a little worried about including it verbatim (though I agree it's very short).

Looks like @xdelaruelle is the original author -- @xdelaruelle, are you ok with adding this snippet to Spack as Apache-2.0/MIT?

[xdelaruelle]

Looks like @xdelaruelle is the original author -- @xdelaruelle, are you ok with adding this snippet to Spack as Apache-2.0/MIT?

@tgamblin Sure no problem

[scheibelp]

I've been investigating the error this reports to solve and I can replicate it on a Cray machine but not on Linux/Mac. The behavior of ps on Cray when executed via ./source differs from Linux. The description of source on Cray appears different as well: "run the commands in the TCL interpreter...".

From my experiments, replacing . /home/pcarns/working/src/spack/share/spack/setup-env.sh with /bin/bash /home/pcarns/working/src/spack/share/spack/setup-env.sh produces the correct result, and also (strangely) running /bin/bash foo.sh instead of just ./foo.sh.

@carns Do you know of any command on Cray that replicates the behavior of source on Linux? If not, I could see this PR being convenient and I think an appropriate commit message would be:

allow using ". setup-env.sh" in scripts on Cray systems

that being said, based on the above, I'm not sure if this is the best approach (i.e. based on my reading of source on a Cray machine, I presume that folks who want to achieve the same effect as . from Linux would not use . on Cray). My knowledge of Cray's is limited and I'd appreciate insight if you have it (and I'd also be interested if you cannot replicate the results I got above e.g. for /bin/bash foo.sh).

[carns]

spack-shell-test.txt

I attached a test case that digs into this a little more carefully. You can download it, rename it to spack-shell-test.sh, and adjust the path to setup-env.sh to run it. It takes one command line argument to control whether it invokes setup-env.sh via "." "source" or simply running it with "/bin/bash". That third option isn't helpful because the caller won't inherit the spack variables in that case; I just included it for a sanity check.

On a plain Ubuntu 18.10 laptop, I get the following results from running it each way, against origin/develop a76c50d:

pcarns@carns-x1:~/working/dbg/spack-shell$ ./spack-shell-test.sh .
spack load mpich output:
========================
init.c(379):ERROR:109: Unknown shell type 'spack-shell-tes'

pcarns@carns-x1:~/working/dbg/spack-shell$ ./spack-shell-test.sh source
spack load mpich output:
========================
init.c(379):ERROR:109: Unknown shell type 'spack-shell-tes'

If I prefix the test script with /bin/bash then I get this:

pcarns@carns-x1:~/working/dbg/spack-shell$ /bin/bash spack-shell-test.sh .
spack load mpich output:
========================

pcarns@carns-x1:~/working/dbg/spack-shell$ /bin/bash spack-shell-test.sh source
spack load mpich output:
========================

So on this machine (just a normal Linux box, not HPC or Cray), it doesn't matter whether you use . or source, but explicitly running the script with /bin/bash works around the problem. I still think it warrants a fix even for this kind of machine, though, because some schedulers and CI environments will directly execute scripts without an explicit shell command. This example shows that you could work around it by having your job script just relay to a second script using an explicit bash invocation, but it's confusing to have to do that.

At any rate, I'm happy to adjust the commit message or PR description if we can make it more clear, but I don't believe that this is a Cray-specific problem.

[carns]

FYI, I updated the source branch for this PR just now, but there is no change to the commit in question. I just needed to rebase to pick up unrelated recent commits from origin/develop.

[scheibelp]

On a plain Ubuntu 18.10 laptop, I get the following results from running it each way, against origin/develop

Thanks for checking that, I was mistaken about reproducing my results on Linux, I could only get that on Mac.

I looked into it more and I was getting confused about how the executable name is set based on the presence of #!:

• On the Linux system I tried (and I think in your case as well), when #! is present, spack-shell-test.sh is the executable name
• On the Mac system I tried, the executable name is /bin/bash spack-shell-test.sh

So this is a more-robust approach for Linux as well and good to merge (and now that I understand why it works I can create a sensible commit message).

[scheibelp]

Thanks!

[carns]

Great, thanks @scheibelp . I've confirmed that the current origin/develop branch of spack is working great within our regression jobs now.