Building same project in parallel can result in errors

[mgsloan]

I got errors like the following when running stack build in parallel with stack build --stack-yaml stack-8.0.yaml

/home/mgsloan/fpco/stack/src/main/Main.hs:96:1:
    Bad interface file: /home/mgsloan/fpco/stack/.stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/Stack/Types/Compiler.hi
        mismatched interface file versions (wanted "7103", got "8001")

This is due to the dist directory being shared. We could have lock files on the dist directory. However, it would be better if we allowed parallel builds of different configurations. So how do we vary the dist dir location for different configurations? One tricky constraint is that stack path --dist-dir should still work (possibly with a --stack-yaml command). There are two decent ways I can think of doing this:

1. Hash the configuration, and use a trimmed version of it in the dist path. At first this approach seems promising, in fact possibly even handling CLI invocations that do things like modify the resolver. However, these CLI modifications would also need to be passed into stack path --dist-dir, which would be rather unwieldy.

2. Add a new stack.yaml option config-name: which specifies a name for the configuration, along with a CLI option --config-name. If config-name is absent, it is inferred from the configuration filename by trimming off .yml or .yaml. This name is then used as a portion of the dist-dir path.

Either way, we'd also want to have lock files on the dist dirs, since we do want to prevent simultaneous execution on the same dist dir.

I'm favoring approach (2), because it would allow for a common case to work well without even knowing about config-name. You can have multiple stack configurations in different files, and safely build them in parallel.

[minad]

👍 for locking the build directory. Sometimes I am not careful and start multiple stack builds at the same time. This often results in an unclean state, which needs a reset with stack clean.

[unode]

Just ran into a similar situation where my editor triggered a build while another manual build was running.
Noticed it as system started swapping 🙄.

Locking would also address this "use-case".
Of particular relevance with large projects that can be memory hogs during compilation.

[snoyberg]

See #4739