Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Closed
Kubuxu opened this issue Jul 17, 2019 · 10 comments
Closed

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Kubuxu opened this issue Jul 17, 2019 · 10 comments
Assignees
Milestone

Comments

@Kubuxu
Copy link

@Kubuxu Kubuxu commented Jul 17, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.6 linux/amd64

I've also partially tested go1.13. The build didn't succeed (because of golang/xerrors#2) but the bug shown up either way.

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kubuxu/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/kubuxu/go/"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kubuxu/go2/test/go-ipfs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build559096814=/tmp/go-build -gno-record-gcc-switches"

What did you do?

> git clone https://github.com/ipfs/go-ipfs
> cd go-ipfs
> go mod tidy
> go build ./cmd/ipfs
> cp go.mod go.mod.after-build
> go mod tidy
> cp go.mod go.mod.after-tidy
> diff go.mod.after-build go.mod.after-tidy

To test Go 1.13 use misc/go1.13 branch.

What did you expect to see?

go build should not pull in dependencies into go.mod that go mod tidy latter removes.

What did you see instead?

 	google.golang.org/appengine v1.4.0 // indirect

google.golang.org/appengine is pulled in by go build and removed by go mod tidy.

go mod why -m google.golang.org/appengine returns

# google.golang.org/appengine
(main module does not need module google.golang.org/appengine)

Unfortunately I was not able to isolate this to one package.

@bcmills bcmills changed the title cmd/go: haisen-dependency on google.golang.org/appengine cmd/go: heisen-dependency on google.golang.org/appengine Jul 17, 2019
@bcmills bcmills self-assigned this Jul 17, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

Neat bug. Still don't know what's going on, but thanks for the clear steps to reproduce!

CC @jayconrod

@bcmills bcmills added this to the Go1.13 milestone Jul 17, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

This also reproduces with go mod graph, so it's probably triggered by merely loading the module graph. I suspect it's related to #29773, but have to evidence to confirm that as of yet.

go-ipfs$ git checkout misc/go1.13
Already on 'misc/go1.13'
Your branch is up to date with 'origin/misc/go1.13'.

go-ipfs$ git diff

go-ipfs$ go mod graph | grep appengine
github.com/ipfs/go-ipfs google.golang.org/appengine@v1.4.0
google.golang.org/appengine@v1.4.0 github.com/golang/protobuf@v1.2.0
google.golang.org/appengine@v1.4.0 golang.org/x/net@v0.0.0-20180724234803-3673e40ba225
google.golang.org/appengine@v1.4.0 golang.org/x/text@v0.3.0
golang.org/x/tools@v0.0.0-20190226205152-f727befe758c google.golang.org/appengine@v1.4.0
google.golang.org/grpc@v1.19.0 google.golang.org/appengine@v1.1.0

go-ipfs$ git diff
diff --git i/go.mod w/go.mod
index 3c3ac7896..f4ae8fa06 100644
--- i/go.mod
+++ w/go.mod
@@ -109,6 +109,7 @@ require (
        go4.org v0.0.0-20190313082347-94abd6928b1d // indirect
        golang.org/x/sync v0.0.0-20190423024810-112230192c58 // indirect
        golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb
+       google.golang.org/appengine v1.4.0 // indirect
        gopkg.in/cheggaaa/pb.v1 v1.0.28
        gotest.tools/gotestsum v0.3.4
 )
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

Here's a more minimal reproducer:

go-ipfs$ cat >./imports.go <<EOF
package imports

import (
        _ "github.com/ipfs/go-unixfs"
        _ "github.com/ipfs/iptb-plugins"
)
EOF

go-ipfs$ cat >./go.mod <<EOF
module github.com/ipfs/go-ipfs

require (
        github.com/ipfs/go-unixfs v0.2.0
        github.com/ipfs/iptb-plugins v0.1.0
)

go 1.12
EOF

go-ipfs$ go mod tidy && grep appengine go.mod

go-ipfs$ go list all >/dev/null && grep appengine go.mod
        google.golang.org/appengine v1.4.0 // indirect

Also works if I rename the main module to example.com.

@Kubuxu

This comment has been minimized.

Copy link
Author

@Kubuxu Kubuxu commented Jul 17, 2019

I have a different project (which, unfortunately, I cannot share) where the bug also manifests. It uses go-unixfs but doesn't use iptb-plugins. It leads me to believe that the core of the issues is deeper in the dependency tree.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

Yes, the core of the problem is definitely deep in the graph.
Still getting more minimal. Smallest reproducer so far:

module example.com

require (
	github.com/smartystreets/assertions v1.0.0 // indirect
	github.com/smartystreets/goconvey v0.0.0-20190710185942-9d28bd7c0945
	go.opencensus.io v0.22.0
)

go 1.12
package imports

import (
	_ "github.com/smartystreets/goconvey"

	_ "go.opencensus.io"
)
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

This is probably a duplicate of #31248.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

It's not an exact repro, but I think I have a case that essentially boils down to the same underlying problem.

env GO111MODULE=on

cd main

go mod graph
stdout i@v0.1.0
! grep 'i v' go.mod

go list all
stdout '^i$'
! grep 'i v' go.mod

go list -m all
stdout '^i v0.1.0$'

-- main/go.mod --
module main

go 1.13

require (
	a v0.0.0
	b v0.0.0
	c v0.0.0
)

replace (
	a => ../a
	b => ../b
	c => ../c
	x v0.1.0 => ../x1
	x v0.2.0 => ../x2
	i => ../i
)
-- main/main.go --
package main

import (
	_ "a"
	_ "b"
	_ "c"
)

func main() {}
-- a/go.mod --
module a
go 1.13
require x v0.1.0
-- a/a.go --
package a
-- b/go.mod --
module b
go 1.13
require x v0.2.0
-- b/b.go --
package b
-- c/go.mod --
module c
go 1.13
-- c/c.go --
package c
import _ "i"
-- x1/go.mod --
module x
go1.13
require i v0.1.0
-- x2/go.mod --
module x
go1.13
-- i/go.mod --
-- i/i.go --
package i

This lists package i in the output of go list all, but fails to list its containing module in go list -m all.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jul 17, 2019

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jul 17, 2019

Change https://golang.org/cl/186557 mentions this issue: cmd/go/internal/mvs: in Req, omit versions implied by older-than-selected versions already in the graph

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 17, 2019

Duplicate of #31248

@bcmills bcmills marked this as a duplicate of #31248 Jul 17, 2019
@bcmills bcmills closed this Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.