Concurrent `cabal new-build` runs cause non-deterministic failures

[TomMD]

When running concurrent new-builds such as:

cabal new-build -ghc-options=-fmax-errors=1 &
 cabal new-build -ghc-options=-fmax-errors=2

I get non-deterministic errors. Consider the below which appears to reference different versions of the same package:

[project]/Git.hs:29:71: error:
    • Couldn't match type ‘text-1.2.2.2:Data.Text.Internal.Text’
                     with ‘Text’
      NB: ‘Text’
            is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.2’
          ‘text-1.2.2.2:Data.Text.Internal.Text’
            is defined in ‘Data.Text.Internal’ in package ‘text-1.2.2.2’
      Expected type: Text
        Actual type: CommitID
    • In the first argument of ‘T.unpack’, namely ‘cid’
      In the expression: T.unpack cid
      In the second argument of ‘readProcess’, namely
        ‘["log", "--pretty=%P", "-n1", T.unpack cid]’

N.B. type CommitID = Text

[alexbiehl]

IIRC -ghc-options affect the nix-style build hash of a package.

What I think is happening here:

• Both cabal processes build text. They don't race in the store as they have different hashes
• They compile into the same dist-newstyle directory and race for the interface files.

[23Skidoo]

Concurrent updates of the store should work, but I don't think that 2+ concurrent cabal build processes that use the same builddir were ever guaranteed to work... I think there was even a previous ticket about this issue in which we decided that we should fail better (by locking the build dir).

/cc @dcoutts

[hvr]

@23Skidoo especially if those 2+ concurrent processes have different install-plans for the same dist-newstyle (which we definitely don't support, since we have a singleton dist-newstyle/cache folder), which like @alexbiehl points out most likely is happening here, since we don't yet exclude "harmless" (as in, they don't affect the .hi/.o outcome) ghc-options from the hash computation. However, we should really create some lockfile so we can at least fail properly and tell the user that another process is currently locking dist-newstyle/cache to avoid confusion.