go/types: invalid memory address or nil pointer dereference on ppc64le

[Rajalakshmi-Girish]

Go version

go1.26-devel_39fd61ddb0

Output of go env in your module/workspace:

[root@raji-ws ~]# go env
AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='ppc64le'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1568121291=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='ppc64le'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE=''
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH=''
GOPPC64='power8'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_ppc64le'
GOVCS=''
GOVERSION='go1.26-devel_39fd61ddb0 Tue Oct 21 13:51:31 2025 -0700'
GOWORK=''
PKG_CONFIG='pkg-config'
[root@raji-ws ~]#

What did you do?

Below k8s Unit tests are failing on ppc64le after 39fd61d with panic: runtime error: invalid memory address or nil pointer dereference

k8s.io/code-generator/cmd/client-gen
k8s.io/code-generator/cmd/client-gen/generators
k8s.io/code-generator/cmd/client-gen: types
k8s.io/code-generator/cmd/deepcopy-gen: generators
k8s.io/code-generator/cmd/go-to-protobuf: protobuf
k8s.io/code-generator/cmd/prerelease-lifecycle-gen: prerelease-lifecycle-generators
k8s.io/code-generator/cmd: validation-gen
k8s.io/code-generator/cmd/validation-gen: util
k8s.io/code-generator/cmd/validation-gen: validators
k8s.io/code-generator/pkg: util
k8s.io/kubernetes/cmd: import-boss
k8s.io/kubernetes/test: typecheck

What did you see happen?

[root@raji-ws kubernetes]# go test k8s.io/code-generator/cmd/client-gen/args -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xeee28]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/client-gen/args       0.015s
FAIL
[root@raji-ws kubernetes]# go test k8s.io/code-generator/cmd/client-gen/generators -race
?       k8s.io/code-generator/cmd/client-gen/generators [no test files]
[root@raji-ws kubernetes]# go test k8s.io/code-generator/cmd/client-gen/generators/util -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xeee38]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc00022a000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc00022a000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/client-gen/generators/util    0.015s
FAIL
[root@raji-ws kubernetes]# go test k8s.io/code-generator/cmd/client-gen/types -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xee1b8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc000228000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc000228000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/client-gen/types      0.015s
FAIL
[root@raji-ws kubernetes]#
[root@raji-ws kubernetes]#  go test k8s.io/code-generator/cmd/deepcopy-gen/generators -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xef4a8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001ee000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001ee000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/deepcopy-gen/generators       0.016s
FAIL
[root@raji-ws kubernetes]# go test k8s.io/code-generator/cmd/go-to-protobuf/protobuf -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xeeed8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/go-to-protobuf/protobuf       0.016s
FAIL
[root@raji-ws kubernetes]# go version
go version go1.26-devel_7b506d106f Fri Oct 24 18:10:58 2025 -0700 linux/ppc64le
[root@raji-ws kubernetes]#
[root@raji-ws kubernetes]#  go test -race k8s.io/code-generator/cmd/prerelease-lifecycle-gen/prerelease-lifecycle-generators
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf5718]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001ae000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001ae000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/prerelease-lifecycle-gen/prerelease-lifecycle-generators      0.016s
FAIL
[root@raji-ws kubernetes]# go test -race k8s.io/code-generator/cmd/validation-gen
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf27d8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc000202000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc000202000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/validation-gen        0.016s
FAIL
[root@raji-ws kubernetes]# go test -race k8s.io/api/extensions/v1beta1
ok      k8s.io/api/extensions/v1beta1   1.018s
[root@raji-ws kubernetes]# go test -race k8s.io/code-generator/cmd/validation-gen/util
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xee1b8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001a8000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001a8000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/validation-gen/util   0.015s
FAIL
[root@raji-ws kubernetes]#
[root@raji-ws kubernetes]# go test -race k8s.io/code-generator/cmd/validation-gen/validators
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf27d8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001fa000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001fa000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/validation-gen/validators     0.016s
FAIL
[root@raji-ws kubernetes]# go test -race k8s.io/code-generator/pkg/util
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf5678]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001f4000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001f4000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/pkg/util  0.017s
FAIL
[root@raji-ws kubernetes]# go test -race k8s.io/kubernetes/cmd/import-boss
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf0228]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc0001aa000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/kubernetes/cmd/import-boss       0.016s
FAIL
[root@raji-ws kubernetes]#
[root@raji-ws kubernetes]# go test -race k8s.io/kubernetes/test/typecheck
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xef648]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc000170000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc000170000)
        /usr/local/go/src/go/types/named.go:557 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/kubernetes/test/typecheck        0.015s
FAIL
[root@raji-ws kubernetes]#

What did you expect to see?

The tests have to PASS.

[gabyhelp]

Related Issues

• reflect: k8s unit tests on ppc64le sees SEGV fault with tip #48238 (closed)
• cmd/link,runtime: k8s unit tests on ppc64le panics with index out of range #48837 (closed)
• k8s UT typecheck/TestVerify: crashes when built with Go at tip due to go/types.Sizes change #62247 (closed)
• go/types: `panic: runtime error: invalid memory address or nil pointer dereference` with go1.22rc1 #64812 (closed)
• cmd/compile: test failure in k8s after prove pass rewrite #68809 (closed)
• hash/crc32: commit 18dbe5b broke kubernetes unit tests #74767 (closed)
• runtime: panic: runtime error: index out of range on ppc64x when accessing ftab #17854 (closed)
• cmd/compile: internal compiler error: bad live variable at entry #55254 (closed)
• runtime: systemOOM occurs when running kubelet on ppc64le cluster #50048 (closed)
• runtime: unexpected nil pointer dereferences since map dynamic entry changes #31782 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

we need a better bug report.
both the code and the actual panic output.

[jkrishmys]

@seankhliao
go test -race k8s.io/kubernetes/test/typecheck
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xef9d8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
/usr/local/go/src/go/types/named.go:292
go/types.(*Named).resolve(0xc0001f0000)
/usr/local/go/src/go/types/named.go:280 +0x490
go/types.(*Named).Underlying(0xc0001f0000)
/usr/local/go/src/go/types/named.go:556 +0x3c
go/types.defPredeclaredTypes()
/usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
/usr/local/go/src/go/types/universe.go:278 +0x284
FAIL k8s.io/kubernetes/test/typecheck 0.017s
FAIL

After commit 39fd61d (CL 712720: “go/types, types2: guard Named.underlying with Named.mu”), running go test -race confirms concurrent access to (*Named).underlying and state_ between under() and resolve() / SetUnderlying().

This appears to occur because under() still writes:
t.underlying = u
t.setState(underlying)

for all entries in seen without holding each t.mu.
These pointer writes race with other goroutines reading or updating underlying, and the subsequent atomic.OrUint32 in setState can observe partially initialized memory.

This failure reproduces consistently on Power (ppc64le) but not on x86 (amd64).
On architectures like Power, memory writes are not guaranteed to be visible to other goroutines without explicit synchronization (e.g. a mutex or happens-before edge), as a result, the “underlying” bit in state_ may become visible before the corresponding pointer write, leading to a nil-pointer dereference or data-race detection.

I believe a fix would be to guard each assignment in under() with t.mu, or move this logic into a helper that applies double-checked locking to synchronize both underlying and state_ updates.

[Rajalakshmi-Girish]

we need a better bug report. both the code and the actual panic output.

I now added the stacktrace to bug description.

[randall77]

@mrkfrmn

[adonovan]

Possible dup of the locking problem that was either introduced or fixed in https://go.dev/cl/713521. (I can't find an issue number for that stack of go/types CLs).

[mrkfrmn]

Quite likely this is the same root cause as 75963 which was tripping up our race builders briefly — this should be fixed past CL cf826bf.

@Rajalakshmi-Girish @jkrishmys can you confirm if this reproduces past there?

[jkrishmys]

Quite likely this is the same root cause as 75963 which was tripping up our race builders briefly — this should be fixed past CL cf826bf.

@Rajalakshmi-Girish @jkrishmys can you confirm if this reproduces past there?

Thanks, we will confirm.

[Rajalakshmi-Girish]

@Rajalakshmi-Girish @jkrishmys can you confirm if this reproduces past there?

@mrkfrmn Yes it is still reproducible on go version go1.26-devel_041f564b3e Tue Oct 28 13:58:12 2025 -0700 linux/ppc64le

[root@alpha-enabled-1761677903-worker-0 kubernetes]# go test k8s.io/code-generator/cmd/client-gen/args -race
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xeef18]

goroutine 1 [running]:
go/types.(*Named).setState(...)
        /usr/local/go/src/go/types/named.go:293
go/types.(*Named).unpack(0xc000150000)
        /usr/local/go/src/go/types/named.go:281 +0x490
go/types.(*Named).Underlying(0xc000150000)
        /usr/local/go/src/go/types/named.go:584 +0x3c
go/types.defPredeclaredTypes()
        /usr/local/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
        /usr/local/go/src/go/types/universe.go:278 +0x284
FAIL    k8s.io/code-generator/cmd/client-gen/args       0.014s
FAIL
[root@alpha-enabled-1761677903-worker-0 kubernetes]#  go version
go version go1.26-devel_041f564b3e Tue Oct 28 13:58:12 2025 -0700 linux/ppc64le
[root@alpha-enabled-1761677903-worker-0 kubernetes]#

[jkrishmys]

@mrkfrmn. Thanks for the CL update. After verifying with Go commit (go1.26-devel_cf826bfcb4), the original concurrency *issue in go/types.(Named).resolveUnderlying appears fixed — there is no panic without -race on ppc64le.

However, when running with the race detector, reproducible crashes persist during Go’s universe initialization (defPredeclaredTypes) as reported above by @Rajalakshmi-Girish . So, Without -race: All tests (k8s.io/code-generator/cmd/client-gen/args, test/typecheck) pass cleanly. With -race: Crashes reproducibly even with GOMAXPROCS=1, so it is not a data race.

Just to ensure did a small test with this code

package repro
import _ "go/types"
import "testing"
func TestNoop(t *testing.T) {}

go test -race -count=1 -v with -race

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedbf8]

goroutine 1 [running]:
go/types.(*Named).setState(...)
/home/jkrishna/godevel/go/src/go/types/named.go:292
go/types.(*Named).resolve(0xc000378000)
/home/jkrishna/godevel/go/src/go/types/named.go:280 +0x490
go/types.(*Named).Underlying(0xc000378000)
/home/jkrishna/godevel/go/src/go/types/named.go:556 +0x3c
go/types.defPredeclaredTypes()
/home/jkrishna/godevel/go/src/go/types/universe.go:135 +0x1688
go/types.init.2()
/home/jkrishna/godevel/go/src/go/types/universe.go:278 +0x284
exit status 2
FAIL test 0.019s

and without -race

go test -count=1 -v

=== RUN TestNoop
--- PASS: TestNoop (0.00s)