Fix for bootstrapping llvm@13:14+gold

[wyphan]

Hopefully closes #29350 and #31706.

[trws]

Excellent, thank you!

[trws]

Hmm, looks like the headers are not found in CI. It looks like the binutils is not rebuilt though, so I'm not sure if the installed version had them. @scottwittenburg any idea why a variant change wouldn't trigger build of the dependency? https://gitlab.spack.io/spack/spack/-/jobs/2713882

[scottwittenburg]

any idea why a variant change wouldn't trigger build of the dependency?

It looks like the e4s package preferences had already turned on +headers for binutils, so maybe this didn't affect the hash of that particular dependency.

[haampie]

I'm not sure if this is a good idea. We don't do this for any <pkg>-dev type system package, including python, for which headers are required when bootstrapping clingo from sources. If you mark a spec as external, you should have the equivalent to what spack would install on your system.

[wyphan]

@haampie Yes, the problem is that some distros ship the headers separately as binutils-dev/devel, while the binutils package itself only contains the executables. This complicates things since Spack externals detection currently only works with executables, so spack external find might incorrectly mark binutils as external when it detects nm or readelf in PATH (and thus suggests that the system-installed version is complete, headers and all). This is why the header search is only performed if binutils is external.

[trws]

We really need to do this much more generally for externals, in fact the same problem comes up for llvm when building on most debian-based distributions if the libc6-dev or multilib packages are missing, as came up in another issue not too long ago. I'm not sure if this one is particularly worth special casing, perhaps tweak it a bit to let us list a set of headers and packages to print a message for? It's, sadly, a much wider problem than just binutils.

[wyphan]

@trws Is it this one? #31706

[trws]

Yup.

[bernhardkaindl]

Squashed and rebased to the latest develop branch

[bernhardkaindl]

It correctly alerts me to install the missing packages on my host, but fails in CI as discussed above already.

I agree that we'd need generic header checks for external packages,
or add external binutils only with "~headers" like I propose in #40214 as a quick fix.