Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: install -mod=readonly pkg@version uses -lang=go1.16 #54908

Closed
ldez opened this issue Sep 6, 2022 · 12 comments
Closed

cmd/go: install -mod=readonly pkg@version uses -lang=go1.16 #54908

ldez opened this issue Sep 6, 2022 · 12 comments
Assignees
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@ldez
Copy link

ldez commented Sep 6, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build19723161=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I created a repository to illustrate the problem.

ldez/go-install@5fca274

go install -mod=readonly . and go install -mod=readonly <module> don't have the same behavior.

The go version seems badly guessed.

The problem has been detected here golangci/golangci-lint#3177

What did you expect to see?

The same behavior locally and with a "remote" module.

What did you see instead?

$ docker run --rm -it golang:1.19 sh
# go install -mod=readonly github.com/ldez/go-install@5fca274bdce3e0aac69eefcc3dcaf867c7c8d9dc
go: downloading github.com/ldez/go-install v0.0.0-20220906210438-5fca274bdce3
go: downloading golang.org/x/exp v0.0.0-20220906200021-fcb1a314c389
# golang.org/x/exp/constraints
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:13:2: embedding interface element ~int|~int8|~int16|~int32|~int64 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:20:2: embedding interface element ~uint|~uint8|~uint16|~uint32|~uint64|~uintptr requires go1.18 or later (-lang was set to go1.16; check go.mod)
    pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:27:2: embedding interface element Signed|Unsigned requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:34:2: embedding interface element ~float32|~float64 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:41:2: embedding interface element ~complex64|~complex128 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:49:2: embedding interface element Integer|Float|~string requires go1.18 or later (-lang was set to go1.16; check go.mod)
# 
$ docker run --rm -it golang:1.19 sh
# git clone https://github.com/ldez/go-install.git
Cloning into 'go-install'...
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 8 (delta 0), reused 8 (delta 0), pack-reused 0
Receiving objects: 100% (8/8), done.
# cd go-install
# go install -mod=readonly .
go: downloading golang.org/x/exp v0.0.0-20220906200021-fcb1a314c389
#
@ldez ldez changed the title cmd/go: go version seems badly guessed when doing "go install <remote>" cmd/go: go version seems badly guessed when doing "go install -mod=readonly <remote>" Sep 6, 2022
@seankhliao seankhliao changed the title cmd/go: go version seems badly guessed when doing "go install -mod=readonly <remote>" cmd/go: install -mod=readonly pkg@version uses -go=go1.16 Sep 6, 2022
@seankhliao seankhliao added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Sep 6, 2022
@mknyszek mknyszek added this to the Backlog milestone Sep 6, 2022
@mknyszek
Copy link
Contributor

mknyszek commented Sep 6, 2022

CC @bcmills

@seankhliao seankhliao changed the title cmd/go: install -mod=readonly pkg@version uses -go=go1.16 cmd/go: install -mod=readonly pkg@version uses -lang=go1.16 Sep 6, 2022
@bcmills
Copy link
Member

bcmills commented Sep 6, 2022

This is a side-effect of #36876.

Per https://go.dev/ref/mod#go-mod-file-go, go mod vendor only records the go versions for dependencies when the main module's go directive specifies go 1.17 or higher.

@ldez, try running go mod tidy -go=1.17 in your module, followed by go mod vendor; please let us know if that doesn't fix the problem.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 6, 2022
@ldez
Copy link
Author

ldez commented Sep 6, 2022

I ran the commands:

$ go mod tidy -go=1.17
$ git st
## main...origin/main
M go.mod
M go.sum
$ git add .
$ git commit -m "go mod tidy -go=1.17"
[main 19d85af] go mod tidy -go=1.17
2 files changed, 25 insertions(+), 1 deletion(-)
$ go mod vendor                       
$ git add .                           
$ git commit -m "go mod vendor"
$ git push                     
Enumerating objects: 23, done.
Counting objects: 100% (23/23), done.
Delta compression using up to 16 threads
Compressing objects: 100% (17/17), done.
Writing objects: 100% (20/20), 13.20 KiB | 13.20 MiB/s, done.
Total 20 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 1 local object.
To github.com:ldez/go-install.git
5fca274..83193a3  main -> main

https://github.com/ldez/go-install/tree/83193a350e20b2a2cd1a73b16e48509f001ae926

$ docker run --rm -it golang:1.19 sh
# go install -mod=readonly github.com/ldez/go-install@83193a350e20b2a2cd1a73b16e48509f001ae926
go: downloading github.com/ldez/go-install v0.0.0-20220906213845-83193a350e20
go: downloading golang.org/x/exp v0.0.0-20220906200021-fcb1a314c389
# golang.org/x/exp/constraints
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:13:2: embedding interface element ~int|~int8|~int16|~int32|~int64 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:20:2: embedding interface element ~uint|~uint8|~uint16|~uint32|~uint64|~uintptr requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:27:2: embedding interface element Signed|Unsigned requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:34:2: embedding interface element ~float32|~float64 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:41:2: embedding interface element ~complex64|~complex128 requires go1.18 or later (-lang was set to go1.16; check go.mod)
pkg/mod/golang.org/x/exp@v0.0.0-20220906200021-fcb1a314c389/constraints/constraints.go:49:2: embedding interface element Integer|Float|~string requires go1.18 or later (-lang was set to go1.16; check go.mod)
# 
$ docker run --rm -it golang:1.19 sh
# git clone https://github.com/ldez/go-install.git
Cloning into 'go-install'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 28 (delta 4), reused 27 (delta 3), pack-reused 0
Receiving objects: 100% (28/28), 13.95 KiB | 3.49 MiB/s, done.
Resolving deltas: 100% (4/4), done.
# cd go-install
# go install -mod=readonly .
go: downloading golang.org/x/exp v0.0.0-20220906200021-fcb1a314c389
# github.com/ldez/go-install
embedding interface element ~[]string requires go1.18 or later (-lang was set to go1.17; check go.mod)
./main.go:11:20: implicit function instantiation requires go1.18 or later (-lang was set to go1.17; check go.mod)
# 

@bcmills The behavior is still the wrong one and now the problem is also here locally.

Maybe I missed something but this cannot fix the problem. The dependency module (golang.org/x/exp) has the right Go version in the go.mod and my test module too (not after your suggestions indeed), and the local Go version is also right.

Maybe my report was not clear about the expectations 🤔.

@ldez
Copy link
Author

ldez commented Sep 7, 2022

@bcmills did I have to clarify something or provide more explanations?

@bcmills
Copy link
Member

bcmills commented Sep 7, 2022

@ldez, the error message is different (and correct) after the update.

In ldez/go-install@83193a3, the error is saying that it can't build golang.org/x/exp with -lang=go1.16, which is correct because vendor/modules.txt does not have the necessary version annotations.

In ldez/go-install@f93b9d8, the error is saying that it can't build github.com/ldez/go-install with -lang=go1.17, which is correct because generics require Go 1.18 and github.com/ldez/go-install/go.mod says go 1.17. (That is: the compiler for Go 1.17 isn't supposed to know how to use generics, even if the generic function is declared in a package that uses a newer language version.)

If you want to actually use the generic API in x/exp, you need to set your own module version high enough to enable generics:

go mod tidy -go=1.18 && go mod vendor

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 7, 2022
@ldez
Copy link
Author

ldez commented Sep 7, 2022

Maybe I don't understand something:

-mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
https://go.dev/ref/mod#build-commands

So I don't understand why the dependencies need to be vendored to be used with -mod=readonly if the flag is here to ignore the vendor directory.

If I follow your explanation every application that uses a lib with generics must vendor the dependencies to be installed with go install -mod=readonly, maybe I don't really understand the goal of go install -mod=readonly.

@bcmills
Copy link
Member

bcmills commented Sep 7, 2022

Ahhh, I get it. Somehow I had missed the -mod=readonly part when I read the steps to reproduce. 🤦‍♂️

I'll have to look into this again.

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 7, 2022
@bcmills bcmills self-assigned this Sep 7, 2022
@bcmills bcmills added the modules label Sep 7, 2022
@frioux
Copy link

frioux commented Sep 29, 2022

If it's not clear, I think this means no one can install gopls today, following the documented instructions. #55084 was called a duplicate of this, but I sorta think gopls should change deps to allow users to install today.

@seankhliao
Copy link
Member

gopls can be installed without the -mod=readonly flag, it should be vim-go that needs to use the correct command

@bcmills bcmills modified the milestones: Backlog, Go1.20 Oct 4, 2022
@gopherbot
Copy link

Change https://go.dev/cl/439855 mentions this issue: cmd/go: set dependency go versions for go install --mod=readonly mod@vers

@frioux
Copy link

frioux commented Oct 11, 2022 via email

gopherbot pushed a commit that referenced this issue Oct 11, 2022
…vers

When running go install --mod=readonly module@version. modfetch.GoSumFile
was not set, so the checksumOk check that's done when checking whether
we need to set the GoVersion from the go mod file was failing. Bypass
the checksumOk check when there's no GoSumFile.

For #54908

Change-Id: I56cf9d36a505b1223e6bf82a7d455746e2f09849
Reviewed-on: https://go-review.googlesource.com/c/go/+/439855
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
@matloob
Copy link
Contributor

matloob commented Nov 2, 2022

This should be fixed by CL 439855

@matloob matloob closed this as completed Nov 2, 2022
romaindoumenc pushed a commit to TroutSoftware/go that referenced this issue Nov 3, 2022
…vers

When running go install --mod=readonly module@version. modfetch.GoSumFile
was not set, so the checksumOk check that's done when checking whether
we need to set the GoVersion from the go mod file was failing. Bypass
the checksumOk check when there's no GoSumFile.

For golang#54908

Change-Id: I56cf9d36a505b1223e6bf82a7d455746e2f09849
Reviewed-on: https://go-review.googlesource.com/c/go/+/439855
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Michaelhobo added a commit to Michaelhobo/mender-artifact-1 that referenced this issue Dec 27, 2022
Newest version of klauspost/compress uses go1.17. When compiling changes for adding zstd, the vendored klauspost/compress library failed to build because it expected go1.17, but go mod vendor defaults it to go1.16 instead, due to golang/go#54908 (comment)

Signed-off-by: Michael Ho <callmemikeh@gmail.com>
@golang golang locked and limited conversation to collaborators Nov 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants