cc: convert compiler wrappers to POSIX sh

[tgamblin]

Fixes #8703.
Closes #25380.

Putting up an initial PR to start the process of converting cc to POSIX shell.

Remaining to do:

• initial shellcheck-based fixes
• replace arrays with posix sh tricks
• read
• heredocs (is this needed?)
• additional fixes

[tgamblin]

@haampie: what is the issue with heredocs? They're supported by Bourne shell so not sure why they're listed as needed.

[tgamblin]

Looks like the initial changes weren't entirely sound -- tests are failing.

[christoph-conrads]

POSIX shells do not support arrays, cf. StackOverflow: Arrays in unix shell? Error message on Alpine Linux with Ash:

/home/john/spack/lib/spack/env/gcc/gcc: line 26: syntax error: unexpected "("

[christoph-conrads]

• Here strings (<<<) are a POSIX feature
• read -a creates an array; POSIX shells do not feature arrays; dash complains with dash: 1: read: Illegal option -a

[christoph-conrads]

POSIX shells do not support arrays, cf. StackOverflow: Arrays in unix shell? Error message on Alpine Linux with Ash:

/home/john/spack/lib/spack/env/gcc/gcc: line 26: syntax error: unexpected "("

[christoph-conrads]

Putting up an initial PR to start the process of converting cc to POSIX shell.

Why not Python?

• There must be a Python interpreter available; you cannot run Spack without it
• A lot of features:
  • assertions
  • data structures like arrays, maps, and so on
  • optional static typing
  • exceptions and/or proper return values
• Absence of error sources
  • confusion between quoted/unquoted parameter substitution, parameters with/without newlines
  • the interaction between set -e and using the exit status as a branching condition

[tgamblin]

POSIX shells do not support arrays

Yep, this is about removing arrays and emulating them -- see http://www.etalabs.net/sh_tricks.html.

[tgamblin]

@christoph-conrads:

Why not Python?

It'd be great if we could. In fact, we started this way. If you go back to 0f3b80c (in 2014), you'll see that we replaced the then-python version with bash. This was done because python startup overhead is really high and was making builds extremely slow. We frequently saw slowdowns of ~6x for builds, just from the wrappers.

[christoph-conrads]

Yep, this is about removing arrays and emulating them -- see http://www.etalabs.net/sh_tricks.html.

I see.

Unlike “enhanced” Bourne shells such as Bash, the POSIX shell does not have array types. However, with a bit of inefficiency, you can get array-like semantics in a pinch using pure POSIX sh. The trick is that you do have one (and only one) array — the positional parameters “$1”, “$2”, etc. — and you can swap things in and out of this array.

😬

Have you considered having the shell code generated?

[christoph-conrads]

@christoph-conrads:

Why not Python?

It'd be great if we could. In fact, we started this way. If you go back to 0f3b80c (in 2014), you'll see that we replaced the then-python version with bash. This was done because python startup overhead is really high and was making builds extremely slow. We frequently saw slowdowns of ~6x for builds, just from the wrappers.

Thank you for the link to the commit.

[haampie]

@christoph-conrads see #25380 (comment) and below

[christoph-conrads]

@christoph-conrads see #25380 (comment) and below

From the discussion:

Generating the compiler wrapper, benchmarks, everything. You've discussed all approaches that came to my mind.

[tgamblin]

closing this as @cosmicexplorer is working on #26048.