Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building same project in parallel can result in errors #2730

Closed
mgsloan opened this issue Oct 23, 2016 · 3 comments

Comments

Projects
None yet
5 participants
@mgsloan
Copy link
Collaborator

commented Oct 23, 2016

I got errors like the following when running stack build in parallel with stack build --stack-yaml stack-8.0.yaml

/home/mgsloan/fpco/stack/src/main/Main.hs:96:1:
    Bad interface file: /home/mgsloan/fpco/stack/.stack-work/dist/x86_64-linux/Cabal-1.24.0.0/build/Stack/Types/Compiler.hi
        mismatched interface file versions (wanted "7103", got "8001")

This is due to the dist directory being shared. We could have lock files on the dist directory. However, it would be better if we allowed parallel builds of different configurations. So how do we vary the dist dir location for different configurations? One tricky constraint is that stack path --dist-dir should still work (possibly with a --stack-yaml command). There are two decent ways I can think of doing this:

  1. Hash the configuration, and use a trimmed version of it in the dist path. At first this approach seems promising, in fact possibly even handling CLI invocations that do things like modify the resolver. However, these CLI modifications would also need to be passed into stack path --dist-dir, which would be rather unwieldy.

  2. Add a new stack.yaml option config-name: which specifies a name for the configuration, along with a CLI option --config-name. If config-name is absent, it is inferred from the configuration filename by trimming off .yml or .yaml. This name is then used as a portion of the dist-dir path.

Either way, we'd also want to have lock files on the dist dirs, since we do want to prevent simultaneous execution on the same dist dir.

I'm favoring approach (2), because it would allow for a common case to work well without even knowing about config-name. You can have multiple stack configurations in different files, and safely build them in parallel.

@minad

This comment has been minimized.

Copy link

commented May 9, 2018

👍 for locking the build directory. Sometimes I am not careful and start multiple stack builds at the same time. This often results in an unclean state, which needs a reset with stack clean.

@unode

This comment has been minimized.

Copy link

commented May 9, 2018

Just ran into a similar situation where my editor triggered a build while another manual build was running.
Noticed it as system started swapping 🙄.

Locking would also address this "use-case".
Of particular relevance with large projects that can be memory hogs during compilation.

snoyberg added a commit that referenced this issue Apr 15, 2019

@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

See #4739

snoyberg added a commit that referenced this issue Apr 15, 2019

@snoyberg snoyberg closed this in 2169b50 Apr 30, 2019

snoyberg added a commit that referenced this issue Apr 30, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.