Installed package IDs are not unique

[snoyberg]

I'm under the impression the the ipids for registered packages are generated by Cabal, but I may be mistaken. If in fact this is done by GHC, let me know and I'll move the issue to GHC Trac.

Original issue is at commercialhaskell/stack#992. Essentially:

• User built package foo-1 and bar-1, where bar depends on foo-1, and registered them both into package database A. They ended up with hashes ABC and DEF.
• User made a modified version of foo-1. Then foo-1 and bar-1 were both compiled and registered into package database B. As expected, foo-1 ended up with a different hash (HIG), but bar-1 ended up with the same DEF hash, even though it was installed in a different location and depended on a different installed copy of foo.
• User tried to compile with the original foo-1-ABC and bar-1-DEF, with both package databases A and B available. GHC selected the bar-1-DEF from package database B, which required foo-1-HIG, which was not included (and conflicted with foo-1-ABC), leading to a build failure.

I worked around this in stack by unregistering unneeded packages from package database B (known there as the local database). However, I think I've seen evidence that this situation can occur when the Haskell Platform is being used, since you can end up with two copies the same hash with different dependencies.

As I see it, there are two answers to this:

1. I've misunderstood, and installed package IDs are not intended to be globally unique. In which case: I think we need some kind of globally unique way of referring to an installed package (include the package database when specifying the hash?)
2. This is a mistaken, and more information needs to be included when generating the hash.

[23Skidoo]

/cc @ezyang

[ezyang]

The short answer is, Nix-like Cabal is going to fix this.

The longer answer is, it looks like GHC's ABI calculation (which is used to compute is behaving differently than you might expect. Let me first explain how ABI calculation works. When you depend on an external package, there are two ways that this factors into the ABI: (1) for every transitively reachable package, GHC will include the source package ID (GHC 7.8)/package key (GHC 7.10) in the ABI hash, and (2) for every used declaration/directly imported module, GHC will include the declaration/module hash in the ABI hash. Notice that (2) is completely sufficient to ensure that GHC doesn't make an unsound compilation. So I sort of suspect dep_pkgs were sort of added as some sort of hack to make sure that the ABI changed if you changed versions.

So, as you have observed, by the definition of ABI, it doesn't have to change if an upstream package changes, but all of the things the module in question cares about don't change (are ABI compatible). But this can lead to the situation you described here, where you really want the IPIDs to say something about how the package database is set up, but Cabal gets the IPID from GHC, and GHC really doesn't care about that... (as long as it can find the right package ids/package keys in the non-ignored portion of the database).

So how do you fix this? Stop using the ABI hash for IPIDs. Just make it something that is entirely Cabal managed. You pick your own naming policy. Done.

[23Skidoo]

@ezyang Thanks for the explanation. Closing, since the fix is in the works.

[ezyang]

Can we get this cited somewhere reasonable? I was pretty surprised to find out ABIs didn't track depends, though it made sense after I looked at the code.

[23Skidoo]

Not sure what is the appropriate place. Cabal documentation is a bit higher-level. FAQ, maybe?