Skip to content

crypto/internal/fips140/sha256: sporadic panics when calling "(crypto/x509).SystemCertPool()" #76169

@leonklingele

Description

@leonklingele

Go version

go1.25.3

Output of go env in your module/workspace:

This was observed on a prod system. `go version -m $BINARY_PATH` yields the following:

[BINARY_NAME]: go1.25.3
[.. REDACTED ..]
	build	-buildmode=exe
	build	-compiler=gc
	build	-ldflags="-w -s -X [REDACTED]"
	build	CGO_ENABLED=0
	build	GOARCH=arm64
	build	GOOS=linux
	build	GOARM64=v8.0
[.. REDACTED ..]

What did you do?

We observed the following panic on one of our prod systems:

panic: runtime error: slice bounds out of range [3616445671586817383:64]
goroutine 1 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:26 +0x5e
main.main.func1()
	/go/src/[REDACTED]/main.go:306 +0x39
panic({0x1027e80?, 0xc0003817e8?})
	/usr/local/go/src/runtime/panic.go:783 +0x132
crypto/internal/fips140/sha256.(*Digest).Write(0xc0005667f0, {0xc000229500?, 0x33c, 0x870?})
	/usr/local/go/src/crypto/internal/fips140/sha256/sha256.go:176 +0x274
crypto/sha256.Sum224({0xc000229500, 0x33c, 0x348})
	/usr/local/go/src/crypto/sha256/sha256.go:70 +0xa7
crypto/x509.(*CertPool).AppendCertsFromPEM(0xc0004ec360, {0xc000600000?, 0x0?, 0x10ac6c8?})
	/usr/local/go/src/crypto/x509/cert_pool.go:239 +0x179
crypto/x509.loadSystemRoots()
	/usr/local/go/src/crypto/x509/root_unix.go:72 +0x3a5
crypto/x509.initSystemRoots()
	/usr/local/go/src/crypto/x509/root.go:40 +0x56
sync.(*Once).doSlow(0xc00012c1a8?, 0xc000022b10?)
	/usr/local/go/src/sync/once.go:78 +0xac
sync.(*Once).Do(...)
	/usr/local/go/src/sync/once.go:69
crypto/x509.systemRootsPool()
	/usr/local/go/src/crypto/x509/root.go:31 +0x45
crypto/x509.SystemCertPool()
	/usr/local/go/src/crypto/x509/cert_pool.go:118 +0x25
[REDACTED]/pkg/auth.TLSConfigFromFile({0x10ba74e?, 0x11}, {0x0, 0x0})
	/go/src/[REDACTED]/pkg/auth/token.go:380 +0x66
main.main()
	/go/src/[REDACTED]/main.go:532 +0x386c

Where [REDACTED]/pkg/auth/token.go includes the following:

import (
	"crypto/x509"
)

cp, err := x509.SystemCertPool() // Line 380

We did not investigate this yet but the lower bound in slice bounds out of range [3616445671586817383:64] seems very incorrect.

The panic itself occurs here: https://github.com/golang/go/blob/go1.25.3/src/crypto/internal/fips140/sha256/sha256.go#L176

Find attached the system root store in use on that system (/etc/ssl/certs/ca-certificates.crt which was taken from arm64v8/debian:bookworm-slim):
ca-certificates.crt

What did you see happen?

See above.

What did you expect to see?

Panic should not occur.

Metadata

Metadata

Assignees

No one assigned

    Labels

    NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions