cmd/go: get doesn't select dependencies limited to go@version

[alexander-bachmann]

Go version

go version go1.21.8 darwin/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/aoeu/Library/Caches/go-build'
GOENV='/Users/aoeu/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/aoeu/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/aoeu/go'
GOPRIVATE=''
GOPROXY=''
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.21.8'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/aoeu/Desktop/just-a-test/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/sm/14wxd09941l27xkmt31qrz3h0000gq/T/go-build2401627713=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

we would like to stay on golang version 1.21 and still be able to update our dependencies using go get -u ./...

the problem is that go get doesn’t respect the local toolchain version and attempts to pull in module versions that are newer than the local toolchain

we found this solution #59886 (comment) but it doesn’t seem to work as seen in the example below:

% go version
go version go1.21.8 darwin/amd64

% export GOTOOLCHAIN=local

% go get k8s.io/client-go go@1.21.8
go: k8s.io/client-go@v0.30.1 requires go >= 1.22.0 (running go 1.21.8; GOTOOLCHAIN=local)

I also attempted with go get k8s.io/client-go go@1.21 and go get k8s.io/client-go go@1.20

What did you see happen?

not download compatible module version

What did you expect to see?

download compatible module version

[seankhliao]

cc @matloob @samthanawalla @hyangah

[thediveo]

the current behavior also breaks tools like go-mod-upgrade completely that use go get.

[matloob]

I can reproduce this. We should fix it. I'm not totally sure if it justifies getting in after the freeze or to be backported, but it might?

[alexander-bachmann]

hey, @matloob just following up. Have you had any luck attempting to fix this issue?

[matloob]

Hi sorry I haven't had a chance to start work on this yet.

[rostidev]

$ go get -u ./...
go: github.com/ory/client-go@v1.17.2 requires go >= 1.24.0 (running go 1.23.6; GOTOOLCHAIN=local)

I use Fedora 41 on my work laptop. Currently this is the latest version of Fedora and it will never get Go 1.24.x but in the future Fedora 42 that is planned for release after about two months from now. I'm also not planing to upgrade my Fedora to version 42 at the day 1 after it will be released, so it will happen even later.

I can run go get -u github.com/ory/client-go@v1.15.17 now, but what about the rest of the dependencies? I'm blocked and will not be able to upgrade all other dependencies automatically for several months. Upgrading them one by one is very inconvenient. Please fix this bug ASAP!

[thediveo]

For me, I install latest stable from the Go project anyway, using dev containers and Github actions. Works and keeps me perfectly covered security-wise. If there is anything that doesn't work anymore, then that's the distro packers for technology that rapidly evolves.

[matloob]

Summary: Our suggestion was not correct and the current behavior is working as intended. We would have to change the semantics of go get to allow this use case and it's not clear what the semantics should be. Until we come up with a solution that doesn't add too much complexity, you'll have to allow the use (for example ,using GOTOOLCHAIN=auto) of a newer toolchain so your go command can understand the newer dependencies.

I'll have to admit that we were mistaken on this: the suggestion in #59886 (comment) was not correct and is not supposed to work, even when the Go command used to run it is new enough.

We got confused by this because while go get dep1 dep2 dep3 go@1.21 looks like it is not pinning the versions of dep1, dep2, and dep3, @upgrade is implicitly used when a version is missing. (See the explanation a couple of paragraphs down in the go get section of the Go modules reference)

The meaning of @upgrade is to use the latest version of a module, unless it's older than the current version (See the version queries section of the Go modules reference)

The meaning of the command is then to pin all the deps at specific (either the current required or latest) versions. If any of these depend on the others at a different version, there will be a conflict, resulting in an error saying that the versions conflict with each other. These conflicts can also occur with the minimum required go versions, which are treated similarly to module requirements. You'll get the conflict error if your go command is at or above the go version in the go.mod file for the upgrade versions of the modules.

When the go command is older than the minimum version of go required by the upgrade version of the module, we don't even get to the point of detecting the conflict. The go command needs to be newer than the upgrade version of the module you're getting to properly understand its go.mod file, which is why it reports the requires go >= 1.22.0 error.

So there are two issues with using go get dep1 dep2 dep3 go@1.21 to limit get to the dependencies compatible with go1.21: the first is that that's not what the semantics of the command are and the second is that the go command has to be new enough.

To address the first issue, if your go command is new enough, you should get each of the dependencies with a separate go command, and then use a final go command to downgrade your go version. That would look like

go get dep1
go get dep2
go get dep3
go get go@1.21

The first three commands will add the requirements on dep1, dep2, and dep3, potentially adding requirements on newer versions of go. The final command will cause Go to make the necessary downgrades to the work module's dependencies so that the module's requirement can be set to go1.21.

The second issue is a bit harder to address. Our recommendation to users is to use GOTOOLCHAIN=auto or GOTOOLCHAIN=<version>+auto to allow the go command to upgrade itself while running the updates. You can keep your default GOTOOLCHAIN to local while you're doing the builds and only set it to auto while running go get.

GOTOOLCHAIN=auto go get dep1
GOTOOLCHAIN=auto go get dep2
GOTOOLCHAIN=auto go get dep3
GOTOOLCHAIN=auto go get go@1.21

We understand that that is not something that users are able to or want to do. But if it is possible, it will have the best results.

@alexander-bachmann Is that something you can do? Can you use newer versions of go only to manage dependencies while continuing to use the previous version of go for your development and builds?

@rosti-il If you can, I'd recommend allowing upgrades to later versions of go. Fedora sets the value of GOTOOLCHAIN to local by default so that the version used can be completely controlled by the package manager. This means the go command can't understand newer go.mod files. If you don't have a strict requirement to stay on older versions of go, you can allow the go version to update when necessary by globally updating the value of your GOTOOLCHAIN: go env -w GOTOOLCHAIN=auto. See the Fedora documentation on their Go package in the GOTOOLCHAIN section near the bottom of the page. If you can't globally set GOTOOLCHAIN, you can set it just when running the go get command.

To allow an older version of Go to be able to update dependencies with an upper bound go requirement lesser than or equal to its own version, we'd have to add another option or mode to the Go command. And given the complexity of the options that already exist (for example the @upgrade semantics we got confused by) it would only make sense to add the mode if the benefit outweighed the added complexity.

[rostidev]

@matloob This is not my personal project, and I can't decide on my own which version of Go to use or if GOTOOLCHAIN can be re-defined as auto. We have a policy that requires using the Go available in the latest Fedora. And we don't even start using the latest Fedora from day one. For example, when I started working on this project in early November, we were using the minimum Go version 1.22 in go.mod files because Fedora 41 (that have Go 1.23) was too new and still wasn't adopted by all project members. Now we use Go 1.23 and can't use Go 1.24 yet.