New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go 1.11.4 does not seem to look into the vendor directory #29670

Closed
cnst opened this Issue Jan 11, 2019 · 13 comments

Comments

Projects
None yet
3 participants
@cnst
Copy link

cnst commented Jan 11, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11.4 linux/amd64

Does this issue reproduce with the latest release?

Running the latest release direct from a golang.org download.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ubuntu/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build981837178=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go build | & head -4

What did you expect to see?

Nothing — expected the build to complete successfully.

What did you see instead?

ubuntu:R {607} go version
go version go1.11.4 linux/amd64
ubuntu:R {608} go build | & head -4
server.go:10:2: cannot find package "example.org/M/B" in any of:
	/usr/local/go/src/example.org/M/B (from $GOROOT)
	/home/ubuntu/go/src/example.org/M/B (from $GOPATH)
R.go:8:2: cannot find package "example.org/M/D" in any of:
ubuntu:R {609} ll vendor/example.org/M/B/
total 28
4 -rw-r--r-- 1 ubuntu ubuntu  156 Jan 10 17:57 .drone.yml
8 -rw-r--r-- 1 ubuntu ubuntu 6207 Jan 10 17:57 Gopkg.lock
4 -rw-r--r-- 1 ubuntu ubuntu  798 Jan 10 17:57 Gopkg.toml
4 -rw-r--r-- 1 ubuntu ubuntu   53 Jan 10 17:57 README.md
8 -rw-r--r-- 1 ubuntu ubuntu 4783 Jan 10 17:57 B.go
ubuntu:R {610} 

Issue

The issue I'm having is that not only is the vendor directory seemingly completely ignored (at least for this specific project, as per the error message), but that the error message seems to indicate that it's not even being looked at. I am not sure of the proper way to debug this, or how to troubleshoot this further. The vendor directory comes straight as part of a pristine git clone of a production package (that contains this vendor directory directly), and nothing in its path is a symlink.

@mvdan

This comment has been minimized.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 11, 2019

Please include a complete (and ideally minimal) source tree that reproduces the problem.

@cnst

This comment has been minimized.

Copy link

cnst commented Jan 12, 2019

@bcmills I do not think it's reasonable to require a minimal source code tree here, before there are any known steps of troubleshooting the issue (in fact, the troubleshooting steps is all I'm asking here for), all coupled with confusing/contradictory/outdated documentation of what's even supposed to happen here.

All I did was install Go with apt-get (got the latest go-1.10 on the latest ubuntu-18.10-server-cloudimg-amd64.img); when that didn't work (whichever the reason, was giving me lots of errors like, cannot find package "bufio" in any of:, /usr/lib/go-1.10/src/bufio (from $GOROOT)), I installed the latest go-1.11 directly from golang.org as per official instructions, which seems to work, except for not picking up stuff from vendor.

  • go build on a production project with a vendor directory doesn't work, as specified in the original bug.

  • env GOFLAGS=-mod=vendor go build, as per the first comment, doesn't work, either, producing the same error messages as without the GOFLAGS.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 16, 2019

I'm sorry, but without knowing at least the directory structure of the code in question, we have no way to diagnose it: we don't know what package you are building, where the relevant vendor subdirectories might be located, or where the file server.go is in relation to those directories.

https://golang.org/cmd/go/#hdr-Vendor_Directories describes the expected directory structure.

You might be able to use go list -deps to investigate that structure, but of course it won't tell you why a given directory is not in the search path.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 16, 2019

I am not surprised that -mod=vendor had no effect: according to your go env output you are building in GOPATH mode, which looks for vendor directories more aggressively than does module mode.

@cnst

This comment has been minimized.

Copy link

cnst commented Jan 16, 2019

This issue stops being reproducible when $PWD within $GOPATH (e.g., can be accomplished by making a symlink, and cd'ing into said symlink). I think there's a lot of room for improvement on documenting this requirement, as, logically, it just makes very little sense for vendor directory to work this way.

https://stackoverflow.com/questions/54140415/go-1-11-ignores-the-vendor-directory-the-errors-give-the-impression-the-direc/54156547#54156547

Just as an example, searching for $GOPATH returns https://github.com/golang/go/wiki/GOPATH as first result, and there's zero mentions of vendor at the time of this comment (I've since updated the wiki). Likewise, your link mentions no such requirement, either.

@bcmills bcmills added this to the Unplanned milestone Jan 16, 2019

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 16, 2019

Given that you have a workaround, that the problem is only reproducible when working outside of GOPATH (which is not well-supported in GOPATH mode in the first place — a significant motivation for the introduction of module mode!), and that we still don't have a clear standalone reproducer, I doubt that we will have the bandwidth to fix it before GOPATH mode is deprecated.

That said, if someone wants to send a fix for 1.13 I'd be glad to review it.

@cnst

This comment has been minimized.

Copy link

cnst commented Jan 16, 2019

Are you saying it's normally supposed to work outside of GOPATH as well, with go1.11.4?

I haven't tried reproducing it, but I don't see how my project would be different from any other one, so, I don't imagine you'd have any issues reproducing this.

@cnst

This comment has been minimized.

Copy link

cnst commented Jan 16, 2019

#14566 — the issue is closed, the follow-to issue is closed as well, giving you the impression it should all work nowadays years later, yet the problem still persists.

There's just too much indirection for all of this, and documentation is clearly inadequate on this simple matter.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 16, 2019

The long-term resolution of #14566 is to use module mode when outside of GOPATH.

In module mode (with Go 1.11.4 or later), -mod=vendor will use the vendor directory at the root of the main module, regardless of where in the filesystem that root happens to be.

If you find that that is not the case, please file a separate issue.

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 16, 2019

Looking more closely, this issue seems to be a near-exact duplicate of #14566. Closing as such.

As noted above, there are at least two possible workarounds: use GOPATH mode with the code appropriately located within GOPATH/src, or use module mode with -mod=vendor.

@bcmills bcmills closed this Jan 16, 2019

@cnst

This comment has been minimized.

Copy link

cnst commented Jan 17, 2019

@bcmills can you explain what you're trying to say?

  1. Why is this being closed towards a bug that itself is a closed duplicate of yet another closed bug?

  2. I thought I was pretty clear that I did try -mod=vendor, and it still didn't work?! See #issuecomment-453701218

@mvdan

This comment has been minimized.

Copy link
Member

mvdan commented Jan 17, 2019

Why is this being closed towards a bug that itself is a closed duplicate of yet another closed bug?

The other bug was invalid. The solution there was to use modules. Since you don't seem to use modules, and are trying to build outside GOPATH, the same solution applies. Either build inside GOPATH, or use modules.

  1. I thought I was pretty clear that I did try -mod=vendor, and it still didn't work?!

Read Bryan's replies carefully - that only makes a difference if you're building a module.

If you're having issues getting set up to use modules, try asking for help in the #modules channel on gophers.slack.com. On the other hand, if you think you've found a bug, open an issue with complete instructions on how to reproduce it. Like Bryan said earlier, we need a full program and full instructions on what to do to reproduce the bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment