New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go clean --modcache downloads modules before cleaning #28680

Open
hyangah opened this Issue Nov 8, 2018 · 6 comments

Comments

Projects
None yet
5 participants
@hyangah
Contributor

hyangah commented Nov 8, 2018

$ cat go.mod
module hello

require rsc.io/quote v1.5.2
$ cat main.go
package main
import "rsc.io/quote"
func main() { println(quote.Hello()) }
$ go clean --modcache
go: finding rsc.io/quote v1.5.2
go: finding rsc.io/sampler v1.3.0
go: finding golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: downloading rsc.io/quote v1.5.2
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
$
$ go version
go version devel +644ddaa842 Wed Nov 7 16:12:02 2018 +0000 linux/amd64

Maybe working as intended. But this command output doesn't mention anything about the cleaning operation but is chatty about module downloading. That is confusing, especially when the go.mod has errors so the messages end with error reports.

For example, the following is the message I get when I attempted to clean up the module cache with problematic go.mod (referencing non-existing versions). The module cache got empty, but the message is confusing.

$ go clean --modcache
go: finding github.com/hyangah/thing1 v0.0.2
go: github.com/hyangah/thing1@v0.0.2: unknown revision v0.0.2
go: error loading module requirements

@heschik heschik added the modules label Nov 8, 2018

@bcmills bcmills added the NeedsFix label Nov 9, 2018

@bcmills bcmills added this to the Go1.12 milestone Nov 9, 2018

@bcmills

This comment has been minimized.

Member

bcmills commented Nov 9, 2018

We should definitely not download modules during go clean --modcache.

@hyangah

This comment has been minimized.

Contributor

hyangah commented Nov 9, 2018

Why is this trying to analyze the go.mod and dependency? Can't it simply delete the cache directory? I wondered if it was trying to delete modules selectively (for unknown reasons), but it turned out not. The whole directory under my default GOPATH was gone after the command.

@bcmills

This comment has been minimized.

Member

bcmills commented Nov 9, 2018

Why is this trying to analyze the go.mod and dependency?

I'm not sure. If we figure that out, I think that will point to the fix. 🙂

@sam-sla

This comment has been minimized.

sam-sla commented Nov 19, 2018

I was having a similar problem with go clean -modcache trying to find a local package can't load package: package github.com/foobar: unknown import path "github.com/foobar": cannot find module providing package github.com/foobar. Setting GO111MODULE=off before executing go clean helped me, since go didn't try to find anything and just did the cleanup.

@bcmills bcmills self-assigned this Dec 5, 2018

@bcmills

This comment has been minimized.

Member

bcmills commented Dec 12, 2018

This turned out to be really complicated.

go clean (without arguments) removes extraneous binaries from the current directory. Specifically, it removes the corresponding binary for each .go file in the directory that is not part of the package in that directory, in case it was built using go build ${file}.go.

In order to figure out which files are in the package, the clean command calls out to the loader, which does a bunch of extra work to resolve transitive imports and then comes back with the names of the files that are in the package. (clean -r applies transitively to the packages imported by the named packages, at the moment also ignoring module boundaries, so it has to traverse the whole import graph to figure out the set of affected packages. I guess we also do that without -r to keep the code simpler.)

The loader, in turn, computes the MVS build list, which means it needs to resolve all of the go.mod files for the transitive requirement graph — even if we're not actually cleaning any packages at all.

I think for a proper fix we'll need to rework the interaction between the clean package and the load and modload packages, or define clean to be much more restricted in module mode.

@bcmills bcmills modified the milestones: Go1.12, Go1.13 Dec 12, 2018

@gopherbot

This comment has been minimized.

gopherbot commented Dec 12, 2018

Change https://golang.org/cl/153818 mentions this issue: cmd/go: reproduce #28680

gopherbot pushed a commit that referenced this issue Dec 12, 2018

cmd/go: reproduce #28680
This change encodes the current behavior in mod_clean_cache.txt.
A fix for that behavior will probably have to wait for 1.13.

Updates #28680

Change-Id: I216b5a783971309cc75187502bddccc58c3a9c35
Reviewed-on: https://go-review.googlesource.com/c/153818
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment