Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: concurrent build and cache clean is unsafe #31948

Open
josharian opened this issue May 9, 2019 · 1 comment

Comments

@josharian
Copy link
Contributor

commented May 9, 2019

Migrated from discussion in #31931.

If you start a build with go build, and run go cache -clean while it is building, the build can fail, because intermediate artifacts get deleted.

A sample failure looks like:

# os/exec
../os/exec/exec.go:24:2: can't open import: "bytes": open /Users/josh/Library/Caches/go-build/85/852a5f87d98090fa893dcf8b168751b8a141cb0f4087b4951cfd9fc5edebe252-d: no such file or directory

@jayconrod wrote:

What should go clean -cache do with concurrent builds?

  • We could have a "clean" lock that would block go build and other commands until go clean -cache finishes.
  • go clean -cache could leave files created after it started. This might be hard to do reliably and portably though.

@bcmills wrote:

I think that locking scheme is possible to implement using the existing cmd/go/internal/lockedfile API.

go build would obtain a read-lock on the file (os.O_CREATE|os.O_RDONLY)
go clean would obtain a write-lock on the file (os.O_CREATE|os.O_WRONLY)

We don't currently rely on file-locking for the cache, but since this would only prevent a build / clean race perhaps it's not so bad.

I am not sure that @jayconrod's second option would work: I believe the problem is that the clean deletes files previously created by this build, which the build is still relying on.

@jayconrod's first option would work as long as go clean -cache is also blocked until go build is done. This makes it look a lot like a RW lock, as @bcmills notes. The only remaining danger is starvation of the clean. (That could actually happen in my particular, unusual case, since I am running many concurrent builds, but that's probably not worth designing around.)

@bcmills

This comment has been minimized.

Copy link
Member

commented May 10, 2019

The only remaining danger is starvation of the clean.

We could address that by adding a second (meta) lock file, serializing acquisition of the clean lock.

Then go build would:

  1. Acquire an exclusive lock on the meta file.
  2. Acquire a shared lock on the clean file.
  3. Release the meta lock.
  4. Perform the build.
  5. Release the clean lock.

go clean would:

  1. Acquire an exclusive lock on the meta file.
  2. Acquire an exclusive lock on the clean file.
  3. Release the meta lock.
  4. Clean the cache.
  5. Release the clean lock.

That would force a total order on the start of commands, but allow the build operations to proceed in parallel once started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.