Panic in csi-lib-utils CI

[jsafrane]

logcheck v0.8.1 panics when checking this PR: kubernetes-csi/csi-lib-utils#165

It could be related to k8s libraries bump and go.mod version bump to 1.22 in the PR.

hack/verify-logcheck.sh
Installing logcheck to temp dir: sigs.k8s.io/logtools/logcheck@v0.8.1
go: downloading sigs.k8s.io/logtools v0.8.1
go: downloading golang.org/x/tools v0.16.1
go: downloading golang.org/x/exp v0.0.0-[202](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_csi-lib-utils/165/pull-kubernetes-csi-csi-lib-utils/1787827549770878976#1:build-log.txt%3A202)30807204917-050eac23e9de
go: downloading golang.org/x/mod v0.14.0
Verifing logcheck: /tmp/tmp.e43WUlXgDw/logcheck -check-contextual /home/prow/go/src/github.com/kubernetes-csi/csi-lib-utils/...
panic: nil object
goroutine 9689 [running]:
golang.org/x/tools/go/analysis/internal/checker.(*action).importObjectFact(0xc037b34770?, {0x0?, 0x0?}, {0x7c6340?, 0xc038f3c530?})
	/home/prow/go/pkg/mod/golang.org/x/tools@v0.16.1/go/analysis/internal/checker/checker.go:915 +0x231
sigs.k8s.io/logtools/logcheck/pkg.checkForFunctionExpr(0xc000706940, 0xc038f40f70, 0xc000014f80)
	/home/prow/go/pkg/mod/sigs.k8s.io/logtools@v0.8.1/logcheck/pkg/logcheck.go:181 +0x2ff
sigs.k8s.io/logtools/logcheck/pkg.run.func1({0x7c7610?, 0xc000706940?})
	/home/prow/go/pkg/mod/sigs.k8s.io/logtools@v0.8.1/logcheck/pkg/logcheck.go:148 +0x52
go/ast.inspector.Visit(0xc038f3a258, {0x7c7610?, 0xc000706940?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:386 +0x2b
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7610?, 0xc000706940?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:51 +0x5c
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7c50?, 0xc000a15110?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:154 +0x86a
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7c50?, 0xc000a15140?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:153 +0x829
go/ast.walkExprList(...)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:26
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7ed0?, 0xc00037b000?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:226 +0x1f92
go/ast.walkStmtList(...)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:32
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7ac0?, 0xc000a15170?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:234 +0x1f13
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c7688?, 0xc000a151a0?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:357 +0x1907
go/ast.walkDeclList(...)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:38
go/ast.Walk({0x7c63a0?, 0xc038f3a258?}, {0x7c75e8?, 0xc000a2c1e0?})
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:366 +0x1b05
go/ast.Inspect(...)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/go/ast/walk.go:397
sigs.k8s.io/logtools/logcheck/pkg.run(0xc038f40f70, 0xc000014f80)
	/home/prow/go/pkg/mod/sigs.k8s.io/logtools@v0.8.1/logcheck/pkg/logcheck.go:143 +0x65
sigs.k8s.io/logtools/logcheck/pkg.Analyser.func1(0x0?)
	/home/prow/go/pkg/mod/sigs.k8s.io/logtools@v0.8.1/logcheck/pkg/logcheck.go:125 +0x17
golang.org/x/tools/go/analysis/internal/checker.(*action).execOnce(0xc01055c460)
	/home/prow/go/pkg/mod/golang.org/x/tools@v0.16.1/go/analysis/internal/checker/checker.go:775 +0x9f7
sync.(*Once).doSlow(0xc001a017b8?, 0xc001a017d0?)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/sync/once.go:74 +0xbf
sync.(*Once).Do(...)
	/home/prow/go/pkg/csiprow.oTp1vGSaHp/go-1.21.5/src/sync/once.go:65
golang.org/x/tools/go/analysis/internal/checker.(*action).exec(...)
	/home/prow/go/pkg/mod/golang.org/x/tools@v0.16.1/go/analysis/internal/checker/checker.go:691
golang.org/x/tools/go/analysis/internal/checker.execAll.func1(0x0?)
	/home/prow/go/pkg/mod/golang.org/x/tools@v0.16.1/go/analysis/internal/checker/checker.go:679 +0x45
created by golang.org/x/tools/go/analysis/internal/checker.execAll in goroutine 9557
	/home/prow/go/pkg/mod/golang.org/x/tools@v0.16.1/go/analysis/internal/checker/checker.go:685 +0x15a

[k8s-ci-robot]

This issue is currently awaiting triage.

If logtools contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bells17]

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_csi-lib-utils/165/pull-kubernetes-csi-csi-lib-utils/1787827549770878976

Error location:

• https://github.com/kubernetes-sigs/logtools/blob/v0.8.1/logcheck/pkg/logcheck.go#L181C12-L181C28
• https://cs.opensource.google/go/x/tools/+/refs/tags/v0.16.1:go/analysis/internal/checker/checker.go;l=913-916

[bells17]

Since merely upgrading the Go version to 1.22 in go.mod caused a panic as shown below, it seems like an issue related to the Go version.

kubernetes-csi/csi-lib-utils#167
https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_csi-lib-utils/167/pull-kubernetes-csi-csi-lib-utils/1787882479147814912

[pohly]

We might have to bump the logcheck dependencies to a newer version to deal with Go 1.22. Just a guess, though.

[bells17]

It seems that just upgrading the Go modules version doesn't resolve the issue.
(Although the panic no longer occurs, a different error has started to appear.)

kubernetes-csi/csi-lib-utils#167
bells17@36ac3c7

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_csi-lib-utils/167/pull-kubernetes-csi-csi-lib-utils/1787891931343228928
https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_csi-lib-utils/167/pull-kubernetes-csi-csi-lib-utils/1787890240338268160

The error seems to be occurring in the following location:
https://cs.opensource.google/go/x/tools/+/master:go/analysis/internal/checker/checker.go;l=727?q=failed%20prerequisites&ss=go%2Fx%2Ftools

[pohly]

When I build logcheck v0.8.1 in the logcheck repo and run the binary against the csi-lib-tools PR, I get a crash. When I build inside the csi-lib-tools repo, I don't. Odd.

Updating dependencies fixed the crash that I saw, so it seems worthwhile. It's just not clear whether it's solving the CI crash.

[pohly]

A test build with my update in kubernetes-csi/csi-lib-utils#168 leads to a different weird failure:

Verifing logcheck: /tmp/tmp.x6fgvfL64i/logcheck -check-contextual /home/prow/go/src/github.com/kubernetes-csi/csi-lib-utils/...
/home/prow/go/src/github.com/kubernetes-csi/csi-lib-utils/vendor/k8s.io/apimachinery/pkg/api/resource/amount.go:17:1: package requires newer Go version go1.22

I verified earlier that Go go1.22.2 is in the PATH both when building logcheck and when invoking it. Perhaps it is referring to the Go version in the logtools go.mod?

[pohly]

I brought this up on the Gophers Slack and the feedback was that this might be a bug in Go, so I'll file an issue.

I have found through experimentation that it works when I put "go 1.22.3" into the go.mod of logtools, so I'll put that into #29 even though it doesn't look like a proper fix.

[pohly]

I turned out that the odd behavior in the CI came about because the installed Go was 1.21.5 (curl --fail --location https://dl.google.com/go/go1.21.5.linux-amd64.tar.gz) which is not recent enough anymore for csi-lib-utils after the Kubernetes 1.30 update.

The good news is that this manual download of the Go toolchain probably is not needed anymore. With Go in the container >= 1.21, it will download a newer release automatically as needed by the code that is being built.