runtime: packages tests crash on AIX with "makeslice len out of range"

[ayappanec]

##### Test execution environment.
# GOARCH: ppc64
# CPU: POWER9
# GOOS: aix
# OS Version: AIX 2 7 00FAC25F4B00

...
ok  	crypto/sha512	0.049s
ok  	crypto/subtle	1.349s
--- FAIL: TestFIPSCertAlgs (5.09s)
panic: runtime error: makeslice: len out of range [recovered, repanicked]

goroutine 361 gp=0xa0000004a8701c0 m=10 mp=0xa0000004a94c008 [running]:
panic({0x110087c28?, 0x110002140?})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/panic.go:879 +0x1b8 fp=0xa0000004ab589c0 sp=0xa0000004ab58900 pc=0x100099578
testing.tRunner.func1.2({0x110087c28, 0x110002140})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/testing/testing.go:2127 +0x210 fp=0xa0000004ab58a80 sp=0xa0000004ab589c0 pc=0x100163ab0
testing.tRunner.func1()
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/testing/testing.go:2130 +0x350 fp=0xa0000004ab58c20 sp=0xa0000004ab58a80 pc=0x1001634c0
panic({0x110087c28?, 0x110002140?})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/panic.go:860 +0x17c fp=0xa0000004ab58ce0 sp=0xa0000004ab58c20 pc=0x10009953c
runtime.panicmakeslicelen(...)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/slice.go:30
runtime.makeslice(0x2955f0d80699f74b?, 0x57f427c39e459b29?, 0xb79cf0d56f0c5a30?)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/slice.go:112 +0xcc fp=0xa0000004ab58d18 sp=0xa0000004ab58ce0 pc=0x10009c01c
math/big.(*Int).Bytes(0xa0000004ab390c0)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/math/big/int.go:531 +0x54 fp=0xa0000004ab58d58 sp=0xa0000004ab58d18 pc=0x10023ddb4
crypto/rsa.bigIntEqual(0x41?, 0xa0000004ab390c0)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/crypto/rsa/rsa.go:150 +0x48 fp=0xa0000004ab58d90 sp=0xa0000004ab58d58 pc=0x10029f088
crypto/rsa.(*PublicKey).Equal(0xa0000004a700a10, {0x110031070?, 0xa0000004a9be6a0})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/crypto/rsa/rsa.go:88 +0x7c fp=0xa0000004ab58dc0 sp=0xa0000004ab58d90 pc=0x10029ee2c
crypto/x509.CreateCertificate({0x1100d4e60, 0x1101853c8}, 0xa0000004ab59628, 0xa0000004a966c08, {0x11001d7b8, 0xa0000004a97ede0}, {0x110030f30?, 0xa0000004a700a10})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/crypto/x509/x509.go:1773 +0x858 fp=0xa0000004ab59558 sp=0xa0000004ab58dc0 pc=0x10031c0c8
crypto/tls.fipsCert(0xa0000004a73afc8, {0x10045ce7d, 0x5}, {0x11001d600, 0xa0000004a97ede0}, 0xa0000004a700af0, 0x80)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/crypto/tls/fips140_test.go:630 +0x4d4 fp=0xa0000004ab59bd0 sp=0xa0000004ab59558 pc=0x1003e6074
crypto/tls.TestFIPSCertAlgs(0xa0000004a73afc8)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/crypto/tls/fips140_test.go:392 +0x150 fp=0xa0000004ab59f30 sp=0xa0000004ab59bd0 pc=0x1003e39d0
testing.tRunner(0xa0000004a73afc8, 0x1100d8ca8)
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/testing/testing.go:2189 +0x114 fp=0xa0000004ab59f90 sp=0xa0000004ab59f30 pc=0x100163084
testing.(*T).Run.gowrap1()
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/testing/testing.go:2254 +0x2c fp=0xa0000004ab59fc0 sp=0xa0000004ab59f90 pc=0x1001641cc
runtime.goexit({})
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/asm_ppc64x.s:1067 +0x4 fp=0xa0000004ab59fc0 sp=0xa0000004ab59fc0 pc=0x1000a1d44
created by testing.(*T).Run in goroutine 1
	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/testing/testing.go:2254 +0x4b0

https://build.golang.org/log/3f7a82a42ebe4a985c84ccbbfe749b85c37fbc84

The failure seems to happen after this change a409055

[gabyhelp]

Related Issues

• runtime: "index out of range" in netpoll on aix-ppc64 #35282 (closed)
• runtime: package tests crash on AIX with "name offset out of range" #77400 (closed)
• os: TestStdPipe fails in AIX #76081 (closed)
• runtime: aix-ppc64 failing with "fatal error: index out of range" as of CL 146719 #38929 (closed)
• runtime: crash on AIX #75574
• runtime/cgo: runtime crash on AIX #77753 (closed)
• math/big: TestAliasing/Exp-XZ panic in math/big.(*stack).nat #72043 (closed)
• crypto/x509: occasional failure on FreeBSD builders #14688 (closed)
• crypto/ecdsa: buffers are too small, can cause panic on s390x #34927 (closed)
• cmd/asm: aes related tests fail with SIGILL on z13 #63387 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dr2chase]

@golang/compiler

[randall77]

@golang/aix

[randall77]

@ianlancetaylor Seems to happen after your CL?

[ianlancetaylor]

Apologies, I missed that this was due to my CL. Although that CL revealed the problem, the actual change causing it was https://go.dev/cl/729880, which was also my CL. Sent https://go.dev/cl/764301 to fix this.

[gopherbot]

Change https://go.dev/cl/764301 mentions this issue: cmd/link: set the size of runtime.gcmask.* on AIX