Missing spec variant in upstream installation leads to KeyError

[nilsvu]

Steps to reproduce

• Install openblas in a Spack tree from before OpenBLAS: Add a symbol suffix option #27500
• Clone a new Spack and set the old Spack install tree as upstream.
• Install any package that depends on openblas, e.g. py-scipy. Use --reuse or another means to make sure the existing openblas installation is used. I encountered this issue when installing from an environment lockfile, which was created with the new Spack clone but uses the upstream openblas installation. The lockfile doesn't list the new 'symbol_suffix' parameter, which leads to the KeyError below.

Error message

==> Error: KeyError: 'symbol_suffix'

spack/var/spack/repos/builtin/packages/openblas/package.py:353, in libs:
        350        # Look for openblas{symbol_suffix}
        351        name = 'libopenblas'
        352        search_shared = bool(spec.variants['shared'].value)
  >>    353        suffix = spec.variants['symbol_suffix'].value
        354        if suffix != 'none':
        355            name += suffix
        356

Information on your system

• Spack: 0.17.1-677-c2e1a12cdf
• Python: 3.8.7
• Platform: linux-centos7-haswell
• Concretizer: clingo

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output

[alalazo]

This is an issue with the implementation of Openblas.libs, which doesn't handle that case. It should be fixed in openblas.

[haampie]

Hm, @alalazo I think in general most packages are written in a way where they assume the variants defined above are usable in methods below. We'd have to communicate/review better that that's not a safe assumption?

[alalazo]

The issue is using:

spec.variants['symbol_suffix'].value

directly. In those cases it's better to use get and provide a default.

[haampie]

Hm, but then we're repeating defaults everywhere.

[alalazo]

Hm, but then we're repeating defaults everywhere.

Not variant defaults. I mean something like:

variant = spec.variants.get('symbol_suffix')
if variant is None:
   # Handle the case of the variant being missing.

I actually have an idea for this kind of issues. What if we can run a post-install hook to record the results of the possible calls to libs in a JSON file to be stored under <prefix>/.spack? We can change our protocol to prefer what is hard-coded in the JSON file if it exists.

[alalazo]

We discussed this issue last Wed. at the community telco. The outcome is, for the time being, to treat these kind of issues as style issues for packages. See here for the minutes.

[anselmic]

Hello,

I have a similar problem with version 0.19.0.dev0 when installing py-numpy:

==> Installing py-numpy-1.23.4-qnrciem6i4vxx5sayxzuv2zjvfigw7ko
==> No binary for py-numpy-1.23.4-qnrciem6i4vxx5sayxzuv2zjvfigw7ko found: installing from source
==> Error: KeyError: 'symbol_suffix'

/cm/shared/apps/spack/current/var/spack/repos/builtin/packages/openblas/package.py:420, in libs:
        417        # Look for openblas{symbol_suffix}
        418        name = "libopenblas"
        419        search_shared = bool(spec.variants["shared"].value)
  >>    420        suffix = spec.variants["symbol_suffix"].value
        421        if suffix != "none":
        422            name += suffix
        423

It is not clear from the previous discussion if there is a workaround.
Thanks

[alalazo]

It is not clear from the previous discussion if there is a workaround.

You can install a new openblas that has the variant. Also, the fix to this is to use get instead of direct access if we want backward compatibility of recipes for spec['openblas'].libs

[NessieCanCode]

Hello,

I have a similar problem with version 0.19.0.dev0 when installing py-numpy:

==> Installing py-numpy-1.23.4-qnrciem6i4vxx5sayxzuv2zjvfigw7ko
==> No binary for py-numpy-1.23.4-qnrciem6i4vxx5sayxzuv2zjvfigw7ko found: installing from source
==> Error: KeyError: 'symbol_suffix'

/cm/shared/apps/spack/current/var/spack/repos/builtin/packages/openblas/package.py:420, in libs:
        417        # Look for openblas{symbol_suffix}
        418        name = "libopenblas"
        419        search_shared = bool(spec.variants["shared"].value)
  >>    420        suffix = spec.variants["symbol_suffix"].value
        421        if suffix != "none":
        422            name += suffix
        423

It is not clear from the previous discussion if there is a workaround. Thanks

a quick workaround is: spack install <package_name> ^openblas symbol_suffix=none