Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: mod tidy removes lines that build seems to need #31248

Closed
zombiezen opened this issue Apr 3, 2019 · 19 comments
Closed

cmd/go: mod tidy removes lines that build seems to need #31248

zombiezen opened this issue Apr 3, 2019 · 19 comments

Comments

@zombiezen
Copy link
Contributor

@zombiezen zombiezen commented Apr 3, 2019

(I haven't dug into exactly what dependency is causing this or why, but I've managed to whittle the issue down to a fairly manageable no-code repro case.)

What version of Go are you using (go version)?

$ go version
go 1.12.1

Does this issue reproduce with the latest release?

Yes. I haven't checked against tip.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rlight2/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rlight2"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rlight2/src/mg-repro/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build158725968=/tmp/go-build -gno-record-gcc-switches"

What did you do?

git clone https://gist.github.com/5513fc51a179cfd757dcdd1d7c1e4995.git mod-tidy-issue
cd mod-tidy-issue
go mod tidy
go build -mod=readonly

main.go + go.mod gist

What did you expect to see?

A successful build of the binary.

What did you see instead?

go: updates to go.mod needed, disabled by -mod=readonly

Additional surrounding steps

Running go build will cause go build -mod=readonly to pass. Doing this adds lines (all marked "indirect") for:

  • github.com/golang/mock
  • gopkg.in/check.v1
  • gopkg.in/yaml.v2

However, running go mod tidy on this go.mod will unceremoniously remove these added lines. Most confusingly, go mod why doesn't seem to know why these lines are there either:

$ go mod why -m gopkg.in/yaml.v2
# gopkg.in/yaml.v2
(main module does not need module gopkg.in/yaml.v2)
@andybons
Copy link
Member

@andybons andybons commented Apr 4, 2019

@andybons andybons added this to the Unplanned milestone Apr 4, 2019
@jayconrod jayconrod self-assigned this Apr 4, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Apr 4, 2019

We should check for similarities with #29773. (I don't know whether it will turn out to be related, but it's a known source of missing dependencies.)

@prymitive
Copy link

@prymitive prymitive commented Jun 7, 2019

Hit the same issue - go build and go mod tidy keep fighting over what should be in go.mod
gist for reference - https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1

$ git st
On branch master
[...]
	new file:   go.mod
	new file:   go.sum
	new file:   main.go
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0
$ go mod tidy
$ git diff
[no diff]
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0

prymitive added a commit to prymitive/karma that referenced this issue Jun 7, 2019
go.mod keeps getting updated on build but go mod tidy reverts these changes

See golang/go#31248 (comment)
@bcmills
Copy link
Member

@bcmills bcmills commented Jun 7, 2019

@prymitive, can you provide a more compact example, or at least compare the output of go list -m all and go mod graph? As noted above, it would be good to rule out duplicates of #29773.

@prymitive
Copy link

@prymitive prymitive commented Jun 8, 2019

Narrowed down needed modules to just two:

github.com/prometheus/client_golang v0.9.4
github.com/spf13/viper v1.4.0

https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1 updated to reflect that
Also added the output of these 2 commands

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 17, 2019

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 17, 2019

Change https://golang.org/cl/186557 mentions this issue: cmd/go/internal/mvs: in Req, omit versions implied by older-than-selected versions already in the graph

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 17, 2019

I found a minimal repro. A consistent reproducer requires two modules that depend on some module X. One of the modules must depend on the newer version of X and on the other module. The other module must depend on the older X, which must pull in some indirect dependency I that the newer X does not. Then mvs.Req normally adds I to the build list, but since I is not required by the selected versions and is also not needed for the import graph, go mod tidy removes it.

https://go-review.googlesource.com/c/go/+/186557/2/src/cmd/go/testdata/script/mod_indirect_tidy.txt

@bcmills bcmills assigned bcmills and unassigned jayconrod Jul 17, 2019
@bcmills bcmills removed this from the Unplanned milestone Jul 18, 2019
@bcmills bcmills added this to the Go1.13 milestone Jul 18, 2019
gopherbot pushed a commit that referenced this issue Jul 18, 2019
Fixes #29773
Updates #31248

Change-Id: Ic1923119c8cf3a60c586df1b270c3af0c9095f29
Reviewed-on: https://go-review.googlesource.com/c/go/+/186537
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@gopherbot gopherbot closed this in 52fdd62 Jul 18, 2019
@jgiles
Copy link

@jgiles jgiles commented Aug 6, 2019

I'm seeing an issue that looks like this on go1.12.7 . Is there anything I can do to confirm that I have a repro of this issue, or if I should file a separate bug?

@horgh
Copy link
Contributor

@horgh horgh commented Aug 6, 2019

You could see if the behaviour you're seeing is fixed using gotip as described here: #33435 (comment)

@vibridi
Copy link

@vibridi vibridi commented Oct 23, 2019

@bcmills
Hello, sorry to revive a closed issue. I think I'm observing the same problem described here in Go1.13.3
In particular, go get disagrees with go mod tidy over which package should go into go.mod. In my case:

go get writes github.com/ugorji/go v1.1.7 // indirect, also

$ go mod why github.com/ugorji/go
# github.com/ugorji/go
(main module does not need package github.com/ugorji/go)

go mod tidy writes github.com/ugorji/go/codec v1.1.7 // indirect, also

$ go mod why github.com/ugorji/go/codec
# github.com/ugorji/go/codec
mycompany/mypackage
github.com/gin-gonic/gin
github.com/gin-gonic/gin/binding
github.com/ugorji/go/codec

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 23, 2019

In particular, go get disagrees with go mod tidy over which package should go into go.mod.

That is #34086, fixed at head.

@vibridi
Copy link

@vibridi vibridi commented Oct 24, 2019

Thank you very much

@thepudds
Copy link

@thepudds thepudds commented Oct 24, 2019

@vibridi If you haven’t already done so, would you mind confirming the behavior you reported is indeed fixed by using the gotip command:

https://godoc.org/golang.org/dl/gotip

@vibridi
Copy link

@vibridi vibridi commented Oct 24, 2019

@thepudds confirmed, it is fixed at head (680ed10) 👍

@ceejatec
Copy link

@ceejatec ceejatec commented Jun 11, 2020

@bcmills Sorry to necropost, but I'm seeing a similar issue even with go 1.14.4. I have a package where "go mod tidy" removes an //indirect entry from go.mod, but that then causes "go build -mod=readonly" to fail with "cannot find module providing package github.com/dustin/gojson: import lookup disabled by -mod=readonly". On the other hand, running "go build" without "-mod=readonly" adds the same //indirect entries back into go.mod that "go mod tidy" just took out.

Before I attempt to reduce this to a test case and file a new ticket, could you tell me if the following statement should 100% be true?

If you run "go mod tidy", then immediately running "go build -mod=readonly"
should always succeed. (Well, at least if the code compiles -
it should not fail due to needing to write to go.mod or for other
reasons implied by -mod=readonly.)

If it isn't supposed to be 100% true, is there any statement that can be made about "go mod tidy" vs. "go build -mod=readonly" that is 100% true?

@ceejatec
Copy link

@ceejatec ceejatec commented Jun 11, 2020

FYI, I also confirmed this behaviour running "gotip".

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jun 11, 2020

@ceejatec Your statement should be true. Please open a new issue, and we'll take a look.

@ceejatec
Copy link

@ceejatec ceejatec commented Jun 13, 2020

@jayconrod Thanks for confirming. I've filed a new ticket #39570 with a reproducer.

@golang golang locked and limited conversation to collaborators Jun 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet