cmd/go: go mod tidy noop, even though indirect dep versions are wonky

[karalabe]

What version of Go are you using (go version)?

$ go version
go version go1.21.0 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What did you do?

Our project accumulated a lot of dependencies and indirect dependencies in our go.mod file over the years. Whenever touching the deps, we always did go mod tidy to ensure everything's on a correct version. A few weeks ago we've been receiving bug reports about funky version issues and while investigating, we've realised there are indirect dependencies in our go.mod file... that are not actually used at all in our code, neither in any actual dependency.

Go mod tidy is a noop, it doesn't detect nor remove these. If I manually remove all indirect deps from the go.mod file and do a go mod tidy, it resulted in a huge diff with versions changing all over the place compared to the original go.mod and go.sum file. It seems that either something is not detected by go mod tidy correctly, or perhaps the command has some subtlety to it that is not documented?

A repro:

# Check out the go-ethereum repo and the weird commit where we noticed this
$ git clone https://github.com/ethereum/go-ethereum
$ cd go-ethereum
$ git checkout 41ee96fdfee5924004e8fbf9bbc8aef783893917

# Verify that there's no diff and that go mod tidy is a noop
$ git diff
$ go mod tidy
$ git diff

# DO THIS MANUALLY: delete all the //indirect lines from go.mod, rerun go mod tidy
$ go mod tidy
go: finding module for package gotest.tools/v3/assert
go: finding module for package github.com/tklauser/go-sysconf
go: finding module for package github.com/StackExchange/wmi
go: finding module for package github.com/naoina/go-stringutil
go: found github.com/naoina/go-stringutil in github.com/naoina/go-stringutil v0.1.0
go: found github.com/StackExchange/wmi in github.com/StackExchange/wmi v1.2.1
go: found github.com/tklauser/go-sysconf in github.com/tklauser/go-sysconf v0.3.12
go: found gotest.tools/v3/assert in gotest.tools/v3 v3.5.0

$ git diff
[... 400 lines of diff ...]

What did you expect to see?

My expectation is for go mod tidy to generate the same indirect dependency versions, no matter how it is run and with what input as far as indirect deps are concerned.

What did you see instead?

Seems that the indirect dependencies listed in the go.mod file play a role in the result of go mod tidy, which IMO they should not.

[seankhliao]

you can upgrade indirect dependencies, and the result of is recorded in your go.mod.
that won't be replicated if you remove any record of your upgrades by deleting the go.mod lines.