Package missing attribute legacy_buildsystem

[vsoch]

Hi there! We do nightly builds for all of the flux packages, and flux-pmix just failed:

https://github.com/flux-framework/spack/actions/runs/3334569875/jobs/5517664275

With:

==> Error: AttributeError: 'LuaLuaposix' object has no attribute 'legacy_buildsystem'

The 'flux-core' package cannot find an attribute while trying to build from sources. This might be due to a change in Spack's package format to support multiple build-systems for a single package. You can fix this by updating the build recipe, and you can also report the issue as a bug. More information at https://spack.readthedocs.io/en/latest/packaging_guide.html#installation-procedure

/opt/spack/lib/spack/spack/build_environment.py:1075, in _setup_pkg_and_run:
       1072        tb_string = traceback.format_exc()
       1073
       1074        # build up some context from the offending package so we can
  >>   1075        # show that, too.
       1076        package_context = get_package_context(tb)
       1077
       1078        logfile = None

I believe as (possibly) a result of this large changeset?

https://github.com/spack/spack/pull/30738/files#diff-a84edde530c6ead1a43248bba686526c1a19670976147ec8638a20d3adb47bbd

Could you please let us know how to best fix this up? Pinging @grondo to stay in the loop - and since this is the first detected failure, remember that we can make the change to the package https://github.com/flux-framework/spack to re-test the build before we open a PR here. Our automation will handle the testing and follow up PR etc. I volunteer to do this fix as soon as I know what the resolution is, given that it's not an issue in spack (in which case we just need to have that fixed and do another nightly test!)

[alalazo]

Should be fixed in the linked PR. Feel free to ping or assign me for anything related to #30738

[alalazo]

Also, are you installing from binaries concrete specs that have been generated before #30738, right?¹

Footnotes

1. This bug is triggered when reading old concrete specs that don't have the build_system variant, which seems not what people want in CI - so that's why I am asking. ↩

[alalazo]

Ok, found it. If I understand correctly this workflow is restoring an old cache:

The bug should be fixed by the corresponding PR. I just want to point out, in case this is not intentional, that the flux workflow seems not to be testing the latest develop - but a combination of installations that happened at different point in time with different Spack versions.

[vsoch]

Sort of? We actually do target upstream spack develop: https://github.com/sciworks/spack-updater/blob/5bef4d6611e2c63fe31ae33516adaa274eca8f10/scripts/update_package.py#L48-L53 and you can see that at the top:

but we use a cache for packages, which hugely speeds up the install:

https://github.com/sciworks/spack-updater/blob/5bef4d6611e2c63fe31ae33516adaa274eca8f10/build/action.yaml#L42-L46

And that is done just for dependencies - so the dependencies would be cached! But I think for this case it says installing from source, so unlikely hitting the cache here?

No binary for flux-core-0.44.0-4awku4hmjntcakjlkayhsgi2qe54ghhh found: installing from source

And this looks perfect! https://github.com/spack/spack/pull/33551/files If this goes in sometime today, we should use it for the builds tonight. Thanks for the speedy turnaround @alalazo !

[alalazo]

But I think for this case it says installing from source, so unlikely hitting the cache here?

Spack is trying to get information from the build system of lua-luaposix in case it wants to influence the build environment of flux-core. Then lua-luaposix fails due to the bug solved in #33551. Spack requires that attribute because lua-luaposix wasn't rebuilt after #30738, so we have to guess its build system.

[vsoch]

Gotcha! So this should be fixed by your PR?

[alalazo]

And thanks for reporting the bug!

[alalazo]

Yes, this was essentially an oversight over 7k lines 😆

[vsoch]

Don't worry @alalazo you're amazing in the work you do for spack, tiny bugs are totally expected! We have this workflow for exactly that reason and I'm always happy to report and help figure them out. <3

[vsoch]

Just wanted to follow up everything is green 🍏 and working tonight - hooray! Thanks again 💚

https://github.com/flux-framework/spack/actions/runs/3342865881