Clingo fails to use externals with patches=abcde variants
#28201
Comments
psakievich
changed the title
|
spack.lock.txt Uploading corrected config files. |
|
I'm finally getting to this, apologies for the delay.
I think this is not a bug, but the expected behavior. If you ask for reuse, Spack will try to reuse whatever you installed if it fits the requirements. If there are installed specs built by Spack with their dependencies that can be reused, those are preferred to the externals declared in I'm also marking this as unreproducible, since I can't obtain the same results as you report above. I think there's at least a custom repository that is missing and might enter into concretization.: repos:
- $spack/../repos/exawindWhat I currently get are errors like: $ spack -e . concretize
==> Starting concretization
==> Error: variant "asan" not found in package "amr-wind"which point to an external that is constrained with variants that don't exist in the builtin repository. |
|
Let me know if you still have a reliable reproducer for this and I'll get back to the issue asap. |
alalazo
added
unreproducible
and removed
triage
|
For the record, @psakievich provided a reproducer: $ docker pull ecpe4s/exawind-snapshot:2022-04-25
$ docker run -it --rm ecpe4s/exawind-snapshot:2022-04-25
root@867c1a118880 > quick-develop -n demo -s nalu-wind@masterCurrently trying to trim the issue down to its root cause. |
|
The issue is triggered by having the external specs not buildable and overconstrained with the 10c10
< - spec: berkeley-db@18.1.40%gcc@9.4.0+cxx~docs+stl patches=b231fcc arch=linux-ubuntu20.04-x86_64
---
> - spec: berkeley-db@18.1.40%gcc@9.4.0+cxx~docs+stl arch=linux-ubuntu20.04-x86_64
16c16
< cxxstd=14 patches=a440f96 visibility=hidden arch=linux-ubuntu20.04-x86_64
---
> cxxstd=14 visibility=hidden arch=linux-ubuntu20.04-x86_64
48c48
< - spec: findutils@4.8.0%gcc@9.4.0 patches=440b954 arch=linux-ubuntu20.04-x86_64
---
> - spec: findutils@4.8.0%gcc@9.4.0 arch=linux-ubuntu20.04-x86_64
59c59
< api=default build_type=RelWithDebInfo patches=2a1e311 arch=linux-ubuntu20.04-x86_64
---
> api=default build_type=RelWithDebInfo arch=linux-ubuntu20.04-x86_64
106c106
< - spec: m4@1.4.19%gcc@9.4.0+sigsegv patches=9dc5fbd,bfdffa7 arch=linux-ubuntu20.04-x86_64
---
> - spec: m4@1.4.19%gcc@9.4.0+sigsegv arch=linux-ubuntu20.04-x86_64
116c116
< - spec: metis@5.1.0%gcc@9.4.0~gdb~int64~real64+shared build_type=Release patches=4991da9,b1225da
---
> - spec: metis@5.1.0%gcc@9.4.0~gdb~int64~real64+shared build_type=Release
139c139
< patches=2c88dfb arch=linux-ubuntu20.04-x86_64
---
> arch=linux-ubuntu20.04-x86_64
166c166
< - spec: parmetis@4.0.3%gcc@9.4.0~gdb~int64~ipo+shared build_type=Release patches=4f89253,50ed208,704b84f
---
> - spec: parmetis@4.0.3%gcc@9.4.0~gdb~int64~ipo+shared build_type=Release
188c188
< cxxstd=11 patches=9880387 arch=linux-ubuntu20.04-x86_64
---
> cxxstd=11 arch=linux-ubuntu20.04-x86_64
214c214
< - spec: zlib@1.2.12%gcc@9.4.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu20.04-x86_64
---
> - spec: zlib@1.2.12%gcc@9.4.0+optimize+pic+shared arch=linux-ubuntu20.04-x86_64I can concretize correctly. Screenshot attached. |
|
This issue is unrelated to spack:
specs:
- hdf5~mpi
packages:
zlib:
externals:
- spec: zlib@1.2.12 patches=abcde
prefix: /usr
buildable: falseTrying to concretize the environment above gives the same unsat result as shown here. |
|
@alalazo thank you for this. I'm curious for a suggestion on how to fix this for our project which is using the spack api to generate our external specs. The source code is here: https://github.com/psakievich/spack-manager/blob/189807ef1a4ca8b545892a74ba7e979cd95adc1f/spack-scripting/scripting/cmd/manager_cmds/external.py#L95 Should we just manually prune out patches from the spec like we do with the dev_path or is there a more proper way to do this in the api? |
|
It seems like this is more our fault than spack's. But it would be a nice feature to be able to generate a spec that is not over constrained but still has the variants. |
|
Part of the problem is that the I would say that the proper way to do this would be to expose the DB with the specs and add the |
alalazo
changed the title
patches=abcde variant
alalazo
changed the title
patches=abcde variantpatches=abcde variants
|
So this is how we are generating the external specs but we are probably doing something that is not quite kosher because we communicate across spack instances. The features allows a user to point to another environment (often from another spack instance with a different database) and pull all the specs in that environment into an |
|
This seems like it is likely an issue with us pushing spack into a new territory. We just want to take complete control over the packages we are building with for developer environments and not rely on what the concretizer may choose for us. It might be worth following up more on slack or in one of the weekly meetups. I think I know everything I need to know to be able to close this issue from my end. This could also be fixed in Spack proper, but I think it warrants more discussion to decide how, and if it is worth it to pursue. |
|
In a few days we'll merge #28504 and then make reuse the default. I think your use case should then work when you e.g. use the other installation tree as an upstream. Let's keep this one open a bit more and verify it works when we get there. |
|
@psakievich Was this issue solved in the end? |
|
@alalazo we solved it our end with some string manipulation. It's not clear to me that spack needs a solution for this. I'll go ahead and close it. |
Steps to reproduce
The environment from the attached files fails to concretize unless I use the
--reuseflag, but if I use original concretizer it concretizes successfully. The error message below is not very helpful.externals.yaml.txt
include.yaml.txt
spack.lock.txt
spack.yaml.txt
Error message
nalu-windis not an external in this case, but all of its dependencies are.Also it looks like clingo is just getting this wrong even with the
--reuseflag. When I use the original concretizer all of the dependencies of externals are dropped off the DAG as they should be:but when I use clingo it resolves all of them as if the externals didn't exist:
Here is the output from the package blame:
Information on your system
General information
spack debug reportand reported the version of Spack/Python/PlatformThe text was updated successfully, but these errors were encountered: