New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoking stack in parallel fails when using built-in docker integration #3400

Closed
SimSaladin opened this Issue Aug 31, 2017 · 3 comments

Comments

Projects
None yet
2 participants
@SimSaladin
Contributor

SimSaladin commented Aug 31, 2017

In our Gitlab CI pipelines (internal, can't provide much details), we are invoking stack many times simultaneusly (build with different params, stack path etc.). And we are using the built-in docker integration (all commands thus start stack --docker. (We also have two different images we use, also simultaneusly, but that probably isn't related. Haven't tested that much.)

In this scenario, some stack commands fail with

SQLite3 returned ErrorBusy while attempting to perform step

Without knowing much about stack internals I suspect $STACK_ROOT/.stack/docker.db that it is failing to interact with when run in parallel.

This is annoying because as it causes pipelines to fail randomly.

Yes this looks like it's more a limitation resulting from using sqlite for tracking docker container state (or whatever that is used for). But maybe there's a way around it?

Steps to reproduce

Invoke multiple stack --docker commands simultaneusly, some of them should fail.

Expected

I expected stack to work just fine with parallel execution even when using docker integration.

Actual

Some (usually, just one) stack commands fail immediately with

SQLite3 returned ErrorBusy while attempting to perform step.

Stack version

$ stack --version
Version 1.5.1, Git revision 600c1f01435a10d127938709556c1682ecfd694e (4861 commits) x86_64 hpack-0.17.1

Method of installation

stack upgrade

@mgsloan mgsloan added this to the P1: Must milestone Aug 31, 2017

@mgsloan mgsloan added the type: bug label Aug 31, 2017

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Aug 31, 2017

Thanks for the report! I have also seen this on occasion, but it never really caused me much issues. I agree it should be fixed.

@SimSaladin

This comment has been minimized.

Contributor

SimSaladin commented Aug 31, 2017

That's good to hear. I could take a chance at implementing a fix, but I can only think of hackish ways around. Then again I'm not familiar with the code base so don't know what to expect. Any nice ideas?

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Aug 31, 2017

@SimSaladin Hey, that'd be great! I believe the relevant code is in Stack.Docker.GlobalDB. I think it makes sense to have a dedicated lockfile for the sql database. For this, use System.FileLock. There are already some examples of its use in the codebase.

mgsloan added a commit that referenced this issue Sep 2, 2017

Merge pull request #3405 from SimSaladin/issue-3400
Fix parallel invocation of `stack --docker` (#3400)

@mgsloan mgsloan closed this Sep 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment