Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally #32917

Closed
davecheney opened this issue Jul 3, 2019 · 11 comments

Comments

Projects
None yet
5 participants
@davecheney
Copy link
Contributor

commented Jul 3, 2019

What version of Go are you using (go version)?

% go version
go version devel +c485e8b559 Sun Jun 30 05:48:31 2019 +0000 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
% go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dfc/Library/Caches/go-build"
GOENV="/Users/dfc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS="-ldflags=-w"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dfc"
GOPRIVATE=""
GOPROXY="https://gocenter.io"
GOROOT="/Users/dfc/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/dfc/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dfc/devel/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/by/3gf34_z95zg05cyj744_vhx40000gn/T/go-bui
ld247327765=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

(~/devel) % mkdir mod
(~/devel) % cd mod
/Users/dfc/devel/mod
(~/devel/mod) % go mod init github.com/foo/bar
go: creating new go.mod: module github.com/foo/bar
(~/devel/mod) % go test -x

What did you expect to see?

A message about no test files for the package.

What did you see instead?

The go tool is accessing a network service to try to fetch the source code for the package I am currently in.

# get https://gocenter.io/github.com/@v/list
# get https://gocenter.io/github.com/foo/@v/list
# get https://gocenter.io/github.com/@v/list: 404 Not Found (1.182s)
# get https://gocenter.io/github.com/foo/@v/list: 404 Not Found (1.184s)
can't load package: package github.com/foo/bar: unknown import path "github.com/foo/bar": cannot find module providing package github.com/foo/bar
@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 3, 2019

The underlying cause is likely the same as #28459, but I'm going to leave this one open since the failure mode is somewhat different.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 8, 2019

This certainly needs to be fixed, but a proper fix is fairly invasive (and probably best left for 1.14).

I think it is important that we at least eliminate the network access for 1.13, but the error message will probably be abysmal.

(CC @jayconrod in case he disagrees or can spot a simpler robust fix.)

@bcmills bcmills modified the milestones: Go1.14, Go1.13 Jul 8, 2019

@bcmills bcmills self-assigned this Jul 8, 2019

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

Can we actually avoid going out to the network without breaking existing behavior?

Suppose the main module is example.com/a, and there's another module, example.com/a/b in a different repository (not in the directory tree). There is a package example.com/a/b/c in example.com/a, not in example.com/a/b. That means b must be a directory without .go files in the example.com/a. If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache. We may need to add a new requirement on module example.com/a/b as well.

Sorry this is a contrived example, but as long as we allow packages to be provided by any module whose path is a prefix, I think we'll have this problem.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 8, 2019

If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache.

I disagree. In general, we only allow paths starting with . in module mode to refer to packages within the same module. (Consider, for example, the behavior of go test ./....) Since example.com/a/b is not in module example.com/a, running go build . in within the ./b subdirectory of example.com/a should be an error.

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

We could fix this by requiring relative paths to be packages in the main module. It looks like if a relative path contains a wildcard, we expand it to paths in the main module containing go files, and we won't go out to the network. If a path does not contain a wildcard, we join it with the module path without checking if anything is there, treating it as a full package path thereafter.

So currently, go build . is equivalent to go build example.com/a/b in my example. But we could fix go build . while keeping the current behavior of go build example.com/a/b.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 8, 2019

Yes. The tricky part is the vendor directory: we should probably resolve it to the corresponding module path if -mod=vendor is set and the directory contains .go source files, and refuse to resolve it otherwise.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 8, 2019

I think I've spotted a simple fix for 1.13. Will send a CL tomorrow (assuming it works out).

@gopherbot

This comment has been minimized.

Copy link

commented Jul 9, 2019

Change https://golang.org/cl/185344 mentions this issue: cmd/go/internal/search: record errors in the Match struct

@gopherbot

This comment has been minimized.

Copy link

commented Jul 9, 2019

Change https://golang.org/cl/185343 mentions this issue: cmd/go/internal/modload: make packageNotInModuleError reasonable for the Target module

@gopherbot

This comment has been minimized.

Copy link

commented Jul 9, 2019

Change https://golang.org/cl/185345 mentions this issue: cmd/go: rationalize errors in internal/load and internal/modload

@gopherbot

This comment has been minimized.

Copy link

commented Jul 9, 2019

Change https://golang.org/cl/185417 mentions this issue: cmd/go: check for source files in relative paths before attempting to determine the package path

@gopherbot gopherbot closed this in b9edee3 Jul 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.