Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go get -u && go mod tidy behaves differently starting with Go 1.21.0, leading to unknown directive toolchain errors #62409

Closed
gaby opened this issue Sep 1, 2023 · 6 comments
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@gaby
Copy link

gaby commented Sep 1, 2023

What version of Go are you using (go version)?

$ go version
go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/ubuntu/.cache/go-build'
GOENV='/home/ubuntu/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/ubuntu/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/ubuntu/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/snap/go/10319'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/snap/go/10319/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/ubuntu/Desktop/git/fix/contrib/websocket/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2504710228=/tmp/go-build -gno-record-gcc-switches'

What did you do?

This is a continuation of #57001

Lets say, I have a library made with Go that supports go version 1.17, 1.18, 1.19, and 1.20.

My go.mod file looks as follow:

module github.com/my/lib

go 1.18

require (
	github.com/some/other/lib v1.0.0
)

Normally when getting updates, you would do go get -u -v then go mod tidy. Since go1.21 running go mod tidy now changes your go.mod file to the following:

module github.com/my/lib

go 1.21

toolchain go1.21.0

require (
	github.com/some/other/lib v1.0.0
)

This a breaking change behavior, given that now if I commit that code? Any previous CI/CD process, or any other developer in the team that's using a different version of go can no longer build/test/run the code because of the toolchain directive.

What did you expect to see?

I expected that running go mod tidy would only cleanup imports and the go.sum file.

What did you see instead?

Running go mod tidy changes the go.mod go version and adds a toolchain directive that's not available in any previous go version.

Suggested solution

There's two possible solutions here:

  • Make go commands use the local install by default instead of auto.
  • The go mod tidy command shouldn't change the version/add a directive to go.mod

Example running with go 1.20.7 in Docker:

root@31579b87a9bd:/data# cat go.mod 
module github.com/my/lib

go 1.21

toolchain go1.21.0

require (
	github.com/some/other/lib v1.0.0
)

root@31579b87a9bd:/data# go mod tidy
go: errors parsing go.mod:
/data/go.mod:5: unknown directive: toolchain
root@31579b87a9bd:/data# go version 
go version go1.20.7 linux/amd64

This bring the question, how can we write libraries that are backward compatible with previous go versions?

@dmitshur dmitshur changed the title go: Unknown directive toolchain cmd/go: go get -u && go mod tidy behaves differently starting with Go 1.21.0, leading to unknown directive toolchain errors Sep 1, 2023
@dmitshur dmitshur added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Sep 1, 2023
@dmitshur
Copy link
Contributor

dmitshur commented Sep 1, 2023

Thanks for reporting this. CC @bcmills, @matloob.

I wanted to clarify a part of the "What did you do?" section. Specifically, when you start out with a go.mod file with support for Go 1.17 through 1.20 and a go 1.18 directive, how does it end up with go 1.21?

I tried to reproduce it as follows (using go1.21.0):

$ cd $(mktemp -d)
$ cat <<EOF >go.mod
module github.com/my/lib

go 1.18

require github.com/google/uuid v1.2.0
EOF
$ cat <<EOF >p.go
package p

import _ "github.com/google/uuid"
EOF

At that point, doing go get -u -v updated the uuid to its latest version v1.3.1, but left the go directive at its starting 1.18 value:

$ go get -u -v
go: trying upgrade to github.com/google/uuid@v1.3.1
go: upgraded github.com/google/uuid v1.2.0 => v1.3.1

$ cat go.mod
module github.com/my/lib

go 1.18

require github.com/google/uuid v1.3.1

Similarly, running go mod tidy did clean up unused entries from go.sum, but otherwise left go.mod as it was before, with go 1.18:

$ go mod tidy
$ echo $?
0

$ cat go.mod
module m

go 1.18

require github.com/google/uuid v1.3.1

Can you clarify what the reproduction steps are missing that's causing the problem? Thanks.

@dmitshur dmitshur added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 1, 2023
@dmitshur dmitshur added this to the Backlog milestone Sep 1, 2023
@gaby
Copy link
Author

gaby commented Sep 2, 2023

@dmitshur Interesting, could this be triggered by a dependency having a toolchain directive on their go.mod file?

Example CI that is currently failing using 1.21: https://github.com/gofiber/contrib/actions/runs/6047961443/job/16412414494

The CI file is located here: https://github.com/gofiber/contrib/blob/main/.github/workflows/govulncheck.yml

The go.mod for when the failure happens is here: https://github.com/gofiber/contrib/blob/main/paseto/go.mod

This also happens with all the go.mod files in each directory of https://github.com/gofiber/contrib/

@dmitshur
Copy link
Contributor

dmitshur commented Sep 2, 2023

Yes, that appears to be what's happening.

As far as I can see, the root problem is that as of gofiber/fiber#2588, the go.mod of the module github.com/gofiber/fiber/v2 states "go 1.21", which means that module requires Go language 1.21 or higher to be used. So when go mod tidy is run in the github.com/gofiber/contrib/paseto module which requires github.com/gofiber/fiber/v2@v2.49.0, it upgrades said module's go line to be 1.21 too. Notably, if you run go get go@1.18 in paseto, it downgrades to github.com/gofiber/fiber/v2@v2.48.0.

(Please refer to https://go.dev/ref/mod#go-mod-file-go and https://go.dev/blog/toolchain for more detailed information on this topic.)

@gaby
Copy link
Author

gaby commented Sep 2, 2023

@dmitshur I just literally realize that too... Sending a message to the Fiber maintainers as I am typing this.

My guess is that the only fix is to roll back those changes in gofiber/fiber.

@gaby
Copy link
Author

gaby commented Sep 2, 2023

@dmitshur Thanks for your help, this can be closed. It's a Fiber issue, not Go.

@gaby gaby closed this as completed Sep 2, 2023
@dmitshur
Copy link
Contributor

dmitshur commented Sep 2, 2023

Glad that we figured out what was causing this. Thanks for the detailed report.

@dmitshur dmitshur closed this as not planned Won't fix, can't repro, duplicate, stale Sep 2, 2023
scbizu added a commit to ezbuy/ezorm that referenced this issue Oct 24, 2023
sunby added a commit to sunby/milvus that referenced this issue Jan 31, 2024
apimachinery use go 1.21 in v0.29.0
reference issue: golang/go#62409

Signed-off-by: sunby <sunbingyi1992@gmail.com>
sunby added a commit to sunby/milvus that referenced this issue Feb 1, 2024
apimachinery use go 1.21 in v0.29.0
reference issue: golang/go#62409

Signed-off-by: sunby <sunbingyi1992@gmail.com>
sre-ci-robot pushed a commit to milvus-io/milvus that referenced this issue Feb 1, 2024
apimachinery use go 1.21 in v0.29.0
reference issue: golang/go#62409

Signed-off-by: sunby <sunbingyi1992@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

2 participants