Spack installing multiple copies of the same spec

[adamjstewart]

Lately I've had a lot of trouble with Spack installing things that are already installed. For example, I installed 2 different packages which depend on jpeg. Now I have 2 copies of jpeg installed. Both are the same version. jpeg doesn't have any dependencies or variants, so that isn't causing problems. But somehow they resolve to a different hash, so Spack installs it twice.

This problem isn't restricted to jpeg. I'm having the exact same problem with szip, which also only has 1 version and no variants or dependencies. I installed szip, then I installed hdf, which depends on szip, so now I have 2 copies. At one point, I installed mpich+verbs and mpichverbs. Then I installed hdf5 ^mpich+verbs and hdf5 ^mpichverbs. Now I have 4 different installations of mpich, but they're all the same version.

Did Spack recently change how it's spec hash is calculated? That could affect older installations. Or am I just going crazy? Here are some examples that are on my machine:

$ spack find -ld jpeg %gcc
==> 2 installed packages.
-- linux-x86_64 / gcc@5.3.0 -------------------------------------
ek6lol7    jpeg@9a

6t2bktj    jpeg@9a

$ spack find -ld szip %gcc
==> 2 installed packages.
-- linux-x86_64 / gcc@5.3.0 -------------------------------------
mmd6ubg    szip@2.1

6xhnf6b    szip@2.1

$ spack find -ld mpich %gcc
==> 4 installed packages.
-- linux-x86_64 / gcc@5.3.0 -------------------------------------
p4pthu4    mpich@3.2~verbs

wntaxzk    mpich@3.2+verbs

2nbkpti    mpich@3.2~verbs

wraqufu    mpich@3.2+verbs

[tgamblin]

Ok two things affect this. In your case it looks like it is the first one.

First:

Did Spack recently change how it's spec hash is calculated?

Yes. When we merged external package support, there was a namespace field added to the spec so that you can tell where each package came from (i.e., the builtin spack repo, some other external repo, etc.). Older specs will have no namespace and will therefore have a different hash.

I probably should've emphasized this more on the concall and in the notes. I am trying to keep the number of things that change hashing to a minimum but we have a lot of fundamental new features making their way into the code base, and sometimes it's necessary. The next item should mitigate the impact in the future...

Second:

Look at #311 -- the satisfaction rules for dependencies don't currently look at what is installed, but they'll reuse it if a sub-dag has the same hash after concretization. We decided in #311 that people would much rather have a best-effort attempt to satisfy things using what is already installed. So once that is fixed, this kind of thing should stop being so annoying -- Spack would just take the namespace-less older spec and use it.

Ok, so how to solve this? You should be able to see the differences if you do spack find -N. Unfortunately, there is not a good way syntactically to refer to namespace-less specs so they are hard to grab in bulk. #369 will address this to some extent (uninstall by hash). We also have some features planned to allow you to rebuild everything that depends on a particular spec, and maybe swap in another, but that may not be in there soon.

[adamjstewart]

Thank you! This answers my question perfectly. As long as the hashing function changes infrequently, I don't mind uninstalling and reinstalling some older packages. That's the joy of working on the development branch, haha.