new-build computes build-plan that triggers "This package indirectly depends on multiple versions of the same package" warning

[hvr]

The following minimal .cabal file exposes the isssue:

Cabal-version:       2.2
Name:                bug
Version:             0

Library
  default-language:    Haskell2010
  build-depends:       bytestring ^>= 0.10.8.2
                     , hspec-discover == 2.5.2
  build-tool-depends:  hspec-discover:hspec-discover == 2.5.1

When building with a GHC version that ships with an older verison of bytestring, e.g. GHC 8.0.2 or 7.10.3, you get an install-plan that triggers the warning:

$ rm -rf dist-newstyle/; cnb -w ghc-8.0.2  -j1
Resolving dependencies...
Build profile: -w ghc-8.0.2 -O1
In order, the following will be built (use -v for more details):
 - bug-0 (lib) (first run)
Configuring library for bug-0..
Warning:
    This package indirectly depends on multiple versions of the same package. This is very likely to cause a compile failure.
      package unix (unix-2.7.2.1) requires bytestring-0.10.8.1
      package bug (bug-0) requires bytestring-0.10.8.2-68d107c33f90edfa54bc6fcd2019a8a688f3a6194a9237130242d6c2fc06e65f
Preprocessing library for bug-0..
Building library for bug-0..

If we dump the install-plan in plan.json, we get the following structure:

Tree
~~~~

bug-0
 ├─ bytestring-0.10.8.2
 │  ├─ base-4.9.1.0
 │  │  ├─ ghc-prim-0.5.0.0
 │  │  │  └─ rts-1.0
 │  │  ├─ integer-gmp-1.0.0.1
 │  │  │  └─ ghc-prim-0.5.0.0 ┄┄
 │  │  └─ rts-1.0 ┄┄
 │  ├─ deepseq-1.4.2.0
 │  │  ├─ array-0.5.1.1
 │  │  │  └─ base-4.9.1.0 ┄┄
 │  │  └─ base-4.9.1.0 ┄┄
 │  ├─ ghc-prim-0.5.0.0 ┄┄
 │  └─ integer-gmp-1.0.0.1 ┄┄
 └─ hspec-discover-2.5.1
    ├─ base-4.9.1.0 ┄┄
    ├─ directory-1.3.0.0
    │  ├─ base-4.9.1.0 ┄┄
    │  ├─ filepath-1.4.1.1
    │  │  └─ base-4.9.1.0 ┄┄
    │  ├─ time-1.6.0.1
    │  │  ├─ base-4.9.1.0 ┄┄
    │  │  └─ deepseq-1.4.2.0 ┄┄
    │  └─ unix-2.7.2.1
    │     ├─ base-4.9.1.0 ┄┄
    │     ├─ bytestring-0.10.8.1
    │     │  ├─ base-4.9.1.0 ┄┄
    │     │  ├─ deepseq-1.4.2.0 ┄┄
    │     │  ├─ ghc-prim-0.5.0.0 ┄┄
    │     │  └─ integer-gmp-1.0.0.1 ┄┄
    │     └─ time-1.6.0.1 ┄┄
    └─ filepath-1.4.1.1 ┄┄


hspec-discover-2.5.1
 [hspec-discover-2.5.1:exe:"hspec-discover"]
 ├─ base-4.9.1.0 ┄┄
 ├─ directory-1.3.0.0 ┄┄
 ├─ filepath-1.4.1.1 ┄┄
 └─ hspec-discover-2.5.1 ┄┄


hspec-discover-2.5.2
 [hspec-discover-2.5.2:exe:"hspec-discover"]
 ├─ base-4.9.1.0 ┄┄
 ├─ directory-1.3.3.0
 │  ├─ base-4.9.1.0 ┄┄
 │  ├─ filepath-1.4.1.1 ┄┄
 │  ├─ time-1.6.0.1 ┄┄
 │  └─ unix-2.7.2.2
 │     ├─ base-4.9.1.0 ┄┄
 │     ├─ bytestring-0.10.8.2 ┄┄
 │     └─ time-1.6.0.1 ┄┄
 ├─ filepath-1.4.1.1 ┄┄
 └─ hspec-discover-2.5.2
    ├─ base-4.9.1.0 ┄┄
    ├─ directory-1.3.3.0 ┄┄
    └─ filepath-1.4.1.1 ┄┄

which shows that lib:bug does indeed appear to link two different versions of bytestring!

Another rather critical thing to point out here is that even despite build-depends: hspec-discover == 2.5.2, lib:bug ends up depending on lib:hspec-discover-2.5.1!

    {
      "type": "configured",
      "id": "bug-0-inplace",
      "pkg-name": "bug",
      "pkg-version": "0",
      "flags": {},
      "style": "local",
      "dist-dir": "/tmp/bug/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bug-0",
      "depends": [
        "bytestring-0.10.8.2-68d107c33f90edfa54bc6fcd2019a8a688f3a6194a9237130242d6c2fc06e65f",
        "hspec-discover-2.5.1-789e45bc4a3155e7166301ac8d17e1a846837663dbe2c8d69e8c24870d4e1486"
      ],
      "exe-depends": [
        "hspec-discover-2.5.1-ceb8d6feeab020d3a15d1c84374b7ab0f282e713df45ae2fb35156bddffab3de"
      ],
      "component-name": "lib"
    },

/cc @grayjay

[grayjay]

I'm traveling now, so I probably won't have time to debug this further for a few weeks, though I spent a little time looking into it earlier. I reproduced the warning with GHC 8.0.2. The solver log looked correct: cabal chose hspec-discover-2.5.2 and bytestring-0.10.8.2 for the build-depends dependencies and chose hspec-discover-2.5.1 and the installed bytestring for the dependencies of the hspec-discover executable. I also printed the install plan returned from the solver, and it seemed correct. My guess is that the hspec-discover build-depends dependency is replaced by the build-tool-depends dependency somewhere after the solver, though I'm not sure where new-build modifies the plan. This is a strange bug!