Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "use of vendored package not allowed" when packages in GOPATH/src/vendor overlap with GOROOT/src/vendor #34068

Open
javasgl opened this issue Sep 4, 2019 · 8 comments

Comments

@javasgl
Copy link

commented Sep 4, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

only in go1.13 happened, back to go1.12.9 can fix this

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/username/Library/Caches/go-build"
GOENV="/Users/username/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH=""
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/b1/bhf9j3q96m75223l3hh41ccw0000gn/T/go-build173249759=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

go install -v ./...

What did you expect to see?

no errrors

What did you see instead?

src/vendor/golang.org/x/net/http2/frame.go:17:2: use of vendored package not allowed
src/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:28:2: use of vendored package not allowed
src/vendor/golang.org/x/net/http2/transport.go:35:2: use of vendored package not allowed

@javasgl javasgl changed the title Breaking changes! use of vendored package not allowed 1.13 Breaking changes! use of vendored package not allowed Sep 4, 2019

@javasgl javasgl changed the title 1.13 Breaking changes! use of vendored package not allowed go1.13 Breaking changes! use of vendored package not allowed Sep 4, 2019

@mvdan

This comment has been minimized.

Copy link
Member

commented Sep 4, 2019

Please provide full instructions to show that this worked on 1.12, and broke in 1.13. For example, using the official Docker images.

@mvdan mvdan added the WaitingForInfo label Sep 4, 2019

@bcmills bcmills changed the title go1.13 Breaking changes! use of vendored package not allowed cmd/go: "use of vendored package not allowed" Sep 4, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 4, 2019

@javasgl, to amplify what @mvdan said: go install -v ./... doesn't help us reproduce your problem unless you also include the contents of the module or repo that you were working in.

In particular, do the import statements in your code include the vendor/ prefix?

This change probably relates to CL 164619, but we did add a regression test in that CL to try to avoid any change in behavior for existing code using the correct import paths.

@bcmills bcmills added this to the Go1.14 milestone Sep 4, 2019

@psanford

This comment has been minimized.

Copy link

commented Sep 4, 2019

I think I ran into the same issue. I made an example repo here to demonstrate the issue: https://github.com/psanford/gobug34068

If you checkout that repo and set your GOPATH to the git working directory you can try building package example.com/foo from go 1.12 (works) vs 1.13 (errors). example.com/foo is a grpc client hello world example.

Grpc and its dependencies were vendored in src/vendor via gvt.

@javasgl

This comment has been minimized.

Copy link
Author

commented Sep 5, 2019

@bcmills same project structure with @psanford

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

Found a much smaller reproducer.

-- issue34068/issue34068.go --
package issue34068

import (
	_ "net/http"
	_ "golang.org/x/net/http2/hpack"
)
-- $GOPATH/src/vendor/golang.org/x/net/http2/hpack/hpack.go --
package hpack
@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

I looked into this today, but I'm not sure that it's feasible to fix.

The trouble is that a number of caches within cmd/go/internal/load assume (and have long assumed) that every package has a unique canonical import path, and import paths whose first component does not contain a dot are in general reserved for the standard library.

The simple failure mode is this:

  1. First we load golang.org/x/net/http2/hpack from within GOPATH, and resolve it to the path vendor/golang.org/x/net/http2/hpack with location GOPATH/src/vendor/golang.org/x/net/http2/hpack.
    2 Then we load net/http from within GOROOT, and we also resolve it to vendor/golang.org/x/net/http2/hpack. The cache shows the location of that package to be GOPATH/src/vendor/golang.org/x/net/http2/hpack, but if we loaded it from scratch we would have gotten GOROOT/src/vendor/golang.org/x/net/http2/hpack.

A similar failure mode occurs with the relative ordering of GOPATH and GOROOT swapped.

A long-term fix would be to have all of the caches track packages by directory instead of import path. That change is fairly invasive, and in general we are not pursuing long-term fixes in GOPATH mode (although we do try to avoid regressions).

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

Fortunately, there are (at least) two workarounds available:

  1. You can move the vendor directory from the top level (GOPATH/src/vendor) to one level in (GOPATH/src/example.com/vendor). Then its path no longer conflicts with the vendor subdirectory in GOROOT.

  2. You can convert the project to use module mode instead of GOPATH mode, in which case the vendor prefix is no longer added to the canonical path for the packages contained in the project's vendor directory (because there is only one vendor replacement for any given package path).

@bcmills

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

Finally, I will note that this issue does not reproduce for any project compatible with go get. Since the vendor directory for any such project must be within its repository, go get would have placed that vendor directory within the projects subdirectory in GOPATH/src, not at the root of GOPATH/src.

@bcmills bcmills self-assigned this Sep 12, 2019

@bcmills bcmills changed the title cmd/go: "use of vendored package not allowed" cmd/go: "use of vendored package not allowed" when packages in GOPATH/src/vendor overlap with GOROOT/src/vendor Sep 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.