Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: unclear error when 'go mod vendor' is run with no packages to vendor #36580

Closed
jayconrod opened this issue Jan 15, 2020 · 9 comments
Closed
Labels
Milestone

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jan 15, 2020

What version of Go are you using (go version)?

$ go version
go version devel +3743d21270 Wed Jan 15 17:53:43 2020 +0000 darwin/amd64

Does this issue reproduce with the latest release?

no, only tip

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/installed"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/opt/go/installed/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/test3/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build143764245=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

In an empty directory:

go mod init example.com/m
go get -d golang.org/x/tools
go mod vendor

What did you expect to see?

This is the error message from 1.13.6. This message is also printed at tip if there are no module requirements.

go: no dependencies to vendor

What did you see instead?

go mod vendor: open /Users/jayconrod/Code/test3/vendor/modules.txt: no such file or directory

This message is printed when go mod vendor tries to write modules.txt after copying packages into the vendor directory. There are no packages to vendor, so it hasn't actually created the vendor directory, and the write fails.

@jayconrod jayconrod added the NeedsFix label Jan 15, 2020
@jayconrod jayconrod added this to the Go1.14 milestone Jan 15, 2020
@jayconrod

This comment has been minimized.

Copy link
Contributor Author

@jayconrod jayconrod commented Jan 15, 2020

Milestoning for Go1.14 because this is a regression, and it would be nice to fix it.

Not a release blocker because it's an error message on an edge case with little impact.

cc @bcmills @matloob

@bcmills bcmills added the modules label Jan 22, 2020
@jcrowgey

This comment has been minimized.

Copy link
Contributor

@jcrowgey jcrowgey commented Jan 25, 2020

I've been learning go for the last year or so. I'd love to learn more about the toolchain compiler and thought I might look around here for some beginner issues as an opportunity to learn. I've repro'd this bug locally so maybe I can offer a fix.

Looks like lines 122 -- 128 of src/cmd/go/internal/modcmd/vendor.go

Shall I take a stab at this one?

@jcrowgey

This comment has been minimized.

Copy link
Contributor

@jcrowgey jcrowgey commented Jan 26, 2020

Actually, I'm not 100% sure I understand the desired behavior. Do we want to create the vendor directory so that modules.txt can get the ## explicit line? Or do we simply not want to create the vendor dir yet (nor modules.txt) since there are no package to vendor?

If the latter, it seems like we can just check len(modpkgs) instead of the write buffer for modules.txt. If the former, since we can no longer rely on vendor dir being created by a call to vendorPkg, I just need to call MkdirAll on vdir before we try to write "modules.txt" into it.
@jayconrod

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 26, 2020
Now that ## explicit lines may appear in modules.txt, the presence of
bytes in the modules.txt buffer doesn't indicate that we have created a
vendor/ directory, nor that we need a modules.txt.

Updates: golang#36580
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jan 27, 2020

Probably we should create the vendor directory and vendor/modules.txt file: that way go build will default to -mod=vendor for future commands and not try to access the network to load the (unnecessary) dependency.

@jayconrod

This comment has been minimized.

Copy link
Contributor Author

@jayconrod jayconrod commented Jan 27, 2020

+1 to creating vendor/modules.txt. This is an important side effect of go mod vendor.

@jcrowgey You are very welcome to work on this. Make sure to test the new behavior. A small change in src/cmd/go/testdata/script/mod_vendor_nodeps.txt would probably be best.

I'm going to remove the Go1.14 milestone from this issue. The 1.14 branch will be cut very soon (maybe today?), so this should probably go into 1.15. (@bcmills let me know if you disagree though).

@jayconrod jayconrod modified the milestones: Go1.14, Go1.15 Jan 27, 2020
@jcrowgey

This comment has been minimized.

Copy link
Contributor

@jcrowgey jcrowgey commented Jan 28, 2020

Thanks to both of you!

Probably we should create the vendor directory and vendor/modules.txt file: that way go build will default to -mod=vendor for future commands and not try to access the network to load the (unnecessary) dependency.

Following that logic, that running go mod vendor should create the side effect where future builds (and presumably go-gets) would default to vendor mode, would we want to create vendor/ and just touch an empty modules.txt file when there's absolutely no dependencies in the pkg cache (ie, even when no go get command has been run)?

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 28, 2020
Even an an empty build list merits a vendor/modules.txt.

Updates: golang#36580
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jan 28, 2020

Maybe? It's not clear to me what should happen if the module has no dependencies whatsoever, since we won't hit the network even with -mod=mod. I'm inclined to leave that case as-is (i.e. don't create the vendor directory or vendor/modules.txt).

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 29, 2020
`go mod vendor` should create vendor/modules.txt even when the only deps
in go.mod are unused.

Updates: golang#36580
@jcrowgey

This comment has been minimized.

Copy link
Contributor

@jcrowgey jcrowgey commented Jan 29, 2020

Ok, perhaps this is on the right track:

It -relies on os.MkdirAll's idempotency (and, hopefully, efficiently doing nothing on a second call) to- ensures the creation of vdir whenever we process an explicit line from the go.mod file which isn't actually used as a dependency.

I added a regression test based on the use case described in the OP here. It's not clear to me if that regression test invokes unnecessary network bandwith if that particular tools version isn't in the local pkg/cache.

EDIT:

This second commit updates to use sync.Once instead of potentially calling os.MkdirAll many times. Anyway, if this is on the right track, I can go ahead and do the CLA thing and open a PR with a branch. Cheers.

jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 31, 2020
`go mod vendor` should create vendor/modules.txt even when the only deps
in go.mod are unused.

Updates: golang#36580
jcrowgey added a commit to jcrowgey/go that referenced this issue Jan 31, 2020
`go mod vendor` should create vendor/modules.txt even when the only deps
in go.mod are unused.

Fixes: golang#36580
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jan 31, 2020

Change https://golang.org/cl/217135 mentions this issue: cmd/vendor: create modules.txt even for unused deps

jcrowgey added a commit to jcrowgey/go that referenced this issue Feb 11, 2020
`go mod vendor` now attempts to create vendor/modules.txt even when the
only dependencies in go.mod are unused.  This change ensures that
vendor/ is created whenever there is any text to be written into modules
txt.

The previous approach relied on a side effect of a call to vendorPkg to
create the directory.  Since the change described in [1], this is no
longer fully reliable, resolving [2].

[1]: https://golang.org/issue/33848#issuecomment-537222782
[2]: Fixes: golang#36580
jcrowgey added a commit to jcrowgey/go that referenced this issue Feb 13, 2020
'go mod vendor' now attempts to create vendor/modules.txt even when the
only dependencies in go.mod are unused.  This change ensures that
vendor/ is created whenever there is any text to be written into modules
txt.

The previous approach relied on a side effect of a call to vendorPkg to
create the directory.  Since the change described in [1], this is no
longer fully reliable, resolving [2].

[1]: https://golang.org/issue/33848#issuecomment-537222782
[2]: Fixes golang#36580
@gopherbot gopherbot closed this in 26aadfa Feb 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants
You can’t perform that action at this time.