Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: dont add dependencies of external tests #32380

Closed
JAicewizard opened this issue Jun 1, 2019 · 12 comments
Closed

cmd/go: dont add dependencies of external tests #32380

JAicewizard opened this issue Jun 1, 2019 · 12 comments

Comments

@JAicewizard
Copy link

@JAicewizard JAicewizard commented Jun 1, 2019

What version of Go are you using (go version)?

go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?

presumably

What operating system and processor architecture are you using (go env)?

go env Output
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jaap/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jaap/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jaap/go-projects/templatebuilder/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build972523621=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go mod tidy

What did you expect to see?

a go.mod that doesnt include dependencies of tests in external packages

What did you see instead?

github.com/Netflix/go-expect v0.0.0-20180928190340-9d1f4485533b // indirect
got added because it is a dependency of some tests in gopkg.in/AlecAivazis/survey.v1

If a test is part of the current module(or submodule thereof) it should be included in the dependencies, but external tests will not be run on building on any platform with any build tag.

@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Jun 1, 2019

Please see this comment from @rsc

The module download is split into two parts: downloading the go.mod and downloading the actual code. If you have dependencies only needed for tests, then they will show up in your go.mod, and go get will download their go.mods, but it will not download their code. The test-only dependencies get downloaded only when you need it, such as the first time you run go test.

So the test dependency will show up in go.mod. But it doesn't get downloaded until you run go test.

@mvdan

This comment has been minimized.

Copy link
Member

@mvdan mvdan commented Jun 2, 2019

There's also other existing issues, like #26955. This should probably be closed as a duplicate.

@JAicewizard

This comment has been minimized.

Copy link
Author

@JAicewizard JAicewizard commented Jun 2, 2019

I get that the tests are included in go.mod
But this dependency is included in the tests of another package and wont be run during go test in the current package.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 3, 2019

this dependency is included in the tests of another package and wont be run during go test in the current package.

go mod tidy is meant to follow the same package import graph as the all pattern in module mode.

If you run go test all, does the resulting build end up running tests within github.com/Netflix/go-expect? If so, this is working as designed. (If not, please provide more detail on the nature of the dependency. If it does not contain sensitive paths, please include the output of go mod why -m github.com/Netflix/go-expect.)

@bcmills bcmills changed the title dont add dependencies of external tests cmd/go: dont add dependencies of external tests Jun 3, 2019
@JAicewizard

This comment has been minimized.

Copy link
Author

@JAicewizard JAicewizard commented Jun 3, 2019

after cleaning up all packages form /src and /pkg/mod I cant run go test at all unless I manually go get a certain package.
I think that is because gitlab.com/golang-commonmark/markdown imports 2 diferent versions of blackfriday for benchmark purposes.
This is the output
A quick ctr-f cant find github.com/Netflix/go-expect or gopkg.in/AlecAivazis/survey.v1(with/without .test)

output of go mod why -m github.com/Netflix/go-expect:

go: downloading gopkg.in/russross/blackfriday.v2 v2.0.1 //I think this is because of the markdown lib, but its not important AFAIK
# github.com/Netflix/go-expect
gitlab.com/antipy/antibuild/cli/cmd
gopkg.in/AlecAivazis/survey.v1
gopkg.in/AlecAivazis/survey.v1.test
github.com/Netflix/go-expect
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 5, 2019

A quick ctr-f cant find github.com/Netflix/go-expect or gopkg.in/AlecAivazis/survey.v1(with/without .test)

GitHub search turned it up pretty readily:
https://github.com/AlecAivazis/survey/blob/3db8b5dd7b2875d268abf7c1feb59bd929f6b838/input_test.go#L10

Edit: never mind, I think I misinterpreted.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 5, 2019

Depending on which version of gitlab.com/antipy/antibuild you're using, the dependency on gopkg.in/AlecAivazis/survey.v1 is here:
https://gitlab.com/antipy/antibuild/cli/blob/modules/cmd/new.go#L18

@JAicewizard

This comment has been minimized.

Copy link
Author

@JAicewizard JAicewizard commented Jun 5, 2019

I know where it is imported, but I cant find that test in go test all
I also already know where the go-expect is coming from (see go why output)
Im working on the modules branch FYI.
But the question was if it is used in a test run by go test all and I cant find it in there

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 5, 2019

Ahhh! I get it now.

What was the exact command that you ran to produce that output, and in what working directory? I don't see any paths from gitlab.com/antipy/antibuild in that output at all.

@JAicewizard

This comment has been minimized.

Copy link
Author

@JAicewizard JAicewizard commented Jun 5, 2019

Oh I dint actually notice, I ran it from the GOPATH but gitlab.com/antipy/antibuild inst in there, thats correct.
I cant run it from inside gitlab.com/antipy/antibuild because I get this:

go: downloading gopkg.in/russross/blackfriday.v2 v2.0.1
go: downloading gopkg.in/russross/blackfriday.v2 v2.0.1
can't load package: package gopkg.in/russross/blackfriday.v2: unknown import path "gopkg.in/russross/blackfriday.v2": cannot find module providing package gopkg.in/russross/blackfriday.v2

This is because gitlab.com/golang-commonmark/markdown imports 2 diferent versions of blackfriday for benchmark purposes (and only one is put in go.mod). one uses gopkg.in and the other the github version. I am not 100% sure this is the cause but I could open an issue about this?

After removing this dependency I can run go test all and I do indeed see that the tests using go-expect are there. Although its not how I would have written it, it follows what its supposed to do so I guess that is correct. Thanks for helping me!

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 5, 2019

The issue with gopkg.in/russross/blackfriday.v2 is probably an instance of #31428.

v2.0.1 includes a go.mod file declaring its module path to be github.com/russross/blackfriday/v2, so you should be getting some sort of module fetch error for that path.

(On the other hand, v2.0.0 should work: it lacks a go.mod file and thus does not enforce import paths.)

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 5, 2019

Sounds like the open questions are all resolved though; closing. (Please do file an issue if you run into any more trouble.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.