crypto/x509: macos 10.14 SIGSEGV in crypto/x509._Cfunc_FetchPEMRoots

[akamensky]

What version of Go are you using (go version)?

go version go1.11.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/username/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/username/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/j6/vhm757px2zz3jgvxy1w3gfzr0000gn/T/go-build447402874=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Just running (and debugging) the code that was working perfectly fine on 1.10 (since 1.11 release a lot of weird issues like debugger cannot kill process sometimes, and runaway CPU usage randomly etc etc)

What did you expect to see?

No crashing

What did you see instead?

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8c72e3e pc=0x8c57013]

runtime stack:
runtime.throw(0x1bea596, 0x2a)
	/usr/local/go/src/runtime/panic.go:608 +0x72
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:374 +0x288

goroutine 130 [syscall]:
runtime.cgocall(0x1001ba0, 0xc00071aa30, 0x1be9723)
	/usr/local/go/src/runtime/cgocall.go:128 +0x5e fp=0xc00071a9f0 sp=0xc00071a9b8 pc=0x1004afe
crypto/x509._Cfunc_FetchPEMRoots(0xc000193ef8, 0xc000193f20, 0xc000000000)
	_cgo_gotypes.go:108 +0x4d fp=0xc00071aa30 sp=0xc00071a9f0 pc=0x128c7cd
crypto/x509.loadSystemRoots.func1(0xc000193ef8, 0xc000193f20, 0x0)
	/usr/local/go/src/crypto/x509/root_cgo_darwin.go:172 +0x1b8 fp=0xc00071aac0 sp=0xc00071aa30 pc=0x1292148
crypto/x509.loadSystemRoots(0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/x509/root_cgo_darwin.go:172 +0xbf fp=0xc00071abb0 sp=0xc00071aac0 pc=0x128c98f
crypto/x509.initSystemRoots()
	/usr/local/go/src/crypto/x509/root.go:21 +0x37 fp=0xc00071abf0 sp=0xc00071abb0 pc=0x1272227
sync.(*Once).Do(0x25992d8, 0x1bff2c8)
	/usr/local/go/src/sync/once.go:44 +0xd1 fp=0xc00071ac28 sp=0xc00071abf0 pc=0x107e511
crypto/x509.systemRootsPool(0x0)
	/usr/local/go/src/crypto/x509/root.go:16 +0x42 fp=0xc00071ac48 sp=0xc00071ac28 pc=0x12721d2
crypto/x509.(*Certificate).Verify(0xc00015e100, 0xc000632b60, 0x10, 0xc0007931a0, 0x0, 0xbee74173c72b9df8, 0xc24227121, 0x257b7a0, 0x0, 0x0, ...)
	/usr/local/go/src/crypto/x509/verify.go:746 +0xb61 fp=0xc00071b030 sp=0xc00071ac48 pc=0x1278901
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc00071be58, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/handshake_client.go:340 +0x28a1 fp=0xc00071b988 sp=0xc00071b030 pc=0x12bd2f1
crypto/tls.(*clientHandshakeState).handshake(0xc00071be58, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/handshake_client.go:247 +0xa01 fp=0xc00071bcd0 sp=0xc00071b988 pc=0x12ba291
crypto/tls.(*Conn).clientHandshake(0xc0000a3180, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/handshake_client.go:168 +0x7ab fp=0xc00071bf10 sp=0xc00071bcd0 pc=0x12b96cb
crypto/tls.(*Conn).Handshake(0xc0000a3180, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1272 +0x14d fp=0xc00071bf78 sp=0xc00071bf10 pc=0x12b7a7d
net/http.(*persistConn).addTLS.func2(0x0, 0xc0000a3180, 0xc00036b810, 0xc0004cdbc0)
	/usr/local/go/src/net/http/transport.go:1171 +0x61 fp=0xc00071bfc0 sp=0xc00071bf78 pc=0x13ffac1
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc00071bfc8 sp=0xc00071bfc0 pc=0x105f961
created by net/http.(*persistConn).addTLS
	/usr/local/go/src/net/http/transport.go:1167 +0x21d

goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x27a9e58, 0x72, 0x8)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b698, 0x72, 0x2527800, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b698, 0xffffffffffffff00, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Accept(0xc00053b680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:384 +0x383
net.(*netFD).accept(0xc00053b680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:238 +0x7d
net.(*TCPListener).accept(0xc0004c22f8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x56
net.(*TCPListener).AcceptTCP(0xc0004c22f8, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock.go:247 +0x79
net/http.tcpKeepAliveListener.Accept(0xc0004c22f8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:3232 +0x53
net/http.(*Server).Serve(0xc000335c70, 0x1ecc120, 0xc0004c22f8, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2826 +0x350
net/http.(*Server).ListenAndServe(0xc000335c70, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:2764 +0x1ed
net/http.ListenAndServe(0x1bd1719, 0xe, 0x1ec6b60, 0xc00033ed00, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:3004 +0xd6
main.main()
	/Users/username/go/src/bitbucket.org/{{redacted}}/main.go:39 +0x43f

goroutine 7 [chan receive]:
github.com/go-redis/redis/internal/pool.(*ConnPool).reaper(0xc00013a160, 0xdf8475800)
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:449 +0xb4
created by github.com/go-redis/redis/internal/pool.NewConnPool
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:99 +0x2a3

goroutine 80 [chan receive]:
github.com/go-redis/redis/internal/pool.(*ConnPool).reaper(0xc000377a20, 0xdf8475800)
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:449 +0xb4
created by github.com/go-redis/redis/internal/pool.NewConnPool
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:99 +0x2a3

goroutine 51 [IO wait]:
internal/poll.runtime_pollWait(0x27a9d88, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b718, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b718, 0xc000177000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b700, 0xc000177000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b700, 0xc000177000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc0004c2308, 0xc000177000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0001b5140, 0xc000177000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc0000a8840)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc0000a8840, 0xc0006a550a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc0000a8840, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc0006f2de0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc0006f2de0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc0000a8840, 0x101000, 0xc0008cd100, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001ec460, 0x1ecc5e0, 0xc000216300, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001ec460, 0x1ecc5e0, 0xc000216300)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 52 [IO wait]:
internal/poll.runtime_pollWait(0x27a9cb8, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b798, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b798, 0xc000644000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b780, 0xc000644000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b780, 0xc000644000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc0004c2310, 0xc000644000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0006369c0, 0xc000644000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc000081200)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc000081200, 0xc00068550a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc000081200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc0006369f0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc0006369f0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc000081200, 0x101000, 0xc00074a500, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001ec5a0, 0x1ecc5e0, 0xc000339f80, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001ec5a0, 0x1ecc5e0, 0xc000339f80)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 37 [chan receive]:
github.com/go-redis/redis/internal/pool.(*ConnPool).reaper(0xc000912000, 0xdf8475800)
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:449 +0xb4
created by github.com/go-redis/redis/internal/pool.NewConnPool
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:99 +0x2a3

goroutine 57 [chan receive]:
github.com/go-redis/redis/internal/pool.(*ConnPool).reaper(0xc000376f20, 0xdf8475800)
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:449 +0xb4
created by github.com/go-redis/redis/internal/pool.NewConnPool
	/Users/username/go/src/github.com/go-redis/redis/internal/pool/pool.go:99 +0x2a3

goroutine 74 [IO wait]:
internal/poll.runtime_pollWait(0x27a9638, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b318, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b318, 0xc0007ac000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b300, 0xc0007ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b300, 0xc0007ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc00033b8a0, 0xc0007ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0007aa360, 0xc0007ac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc00076e7e0)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc00076e7e0, 0xc00068950a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc00076e7e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc000703fb0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc000703fb0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc00076e7e0, 0x101000, 0xc0008dec00, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001badc0, 0x1ecc5e0, 0xc000774580, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001badc0, 0x1ecc5e0, 0xc000774580)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 72 [IO wait]:
internal/poll.runtime_pollWait(0x27a97d8, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b218, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b218, 0xc0007ba000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b200, 0xc0007ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b200, 0xc0007ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc00033b890, 0xc0007ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0007aa840, 0xc0007ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc00076e960)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc00076e960, 0xc0008dd50a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc00076e960, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc00086dad0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc00086dad0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc00076e960, 0x101000, 0xc0008ccc00, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001ba8c0, 0x1ecc5e0, 0xc000774800, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001ba8c0, 0x1ecc5e0, 0xc000774800)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 73 [IO wait]:
internal/poll.runtime_pollWait(0x27a9708, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b298, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b298, 0xc0007bc000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b280, 0xc0007bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b280, 0xc0007bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc00033b898, 0xc0007bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0007aaab0, 0xc0007bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc00076ea20)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc00076ea20, 0xc0007c750a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc00076ea20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc00086cd50, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc00086cd50, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc00076ea20, 0x101000, 0xc00074b400, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001bab40, 0x1ecc5e0, 0xc000774a00, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001bab40, 0x1ecc5e0, 0xc000774a00)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 71 [IO wait]:
internal/poll.runtime_pollWait(0x27a98a8, 0x72, 0x23)
	/usr/local/go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc00053b198, 0x72, 0x1ec7000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x96
internal/poll.(*pollDesc).waitRead(0xc00053b198, 0xc0007ae000, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x45
internal/poll.(*FD).Read(0xc00053b180, 0xc0007ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x36a
net.(*netFD).Read(0xc00053b180, 0xc0007ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x7a
net.(*conn).Read(0xc00033b888, 0xc0007ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0xb0
net/http.(*connReader).Read(0xc0007aa5d0, 0xc0007ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:786 +0x2c4
bufio.(*Reader).fill(0xc00076e8a0)
	/usr/local/go/src/bufio/bufio.go:100 +0x2d2
bufio.(*Reader).ReadSlice(0xc00076e8a0, 0xc0007b950a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:341 +0x3d6
bufio.(*Reader).ReadLine(0xc00076e8a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:370 +0x91
net/textproto.(*Reader).readLineSlice(0xc000890030, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0xaf
net/textproto.(*Reader).ReadLine(0xc000890030, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x6f
net/http.readRequest(0xc00076e8a0, 0x101000, 0xc0008de400, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:958 +0xb0
net/http.(*conn).readRequest(0xc0001ba640, 0x1ecc5e0, 0xc0007746c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:966 +0x4b0
net/http.(*conn).serve(0xc0001ba640, 0x1ecc5e0, 0xc0007746c0)
	/usr/local/go/src/net/http/server.go:1788 +0xa31
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2851 +0x7b0

goroutine 97 [select]:
net/http.(*Transport).getConn(0x256f3c0, 0xc0009524b0, 0xc000331d00, 0xc000680780, 0x5, 0xc000632b60, 0x14, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1004 +0xa32
net/http.(*Transport).roundTrip(0x256f3c0, 0xc0008de900, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:451 +0xd0f
net/http.(*Transport).RoundTrip(0x256f3c0, 0xc0008de900, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/roundtrip.go:17 +0x5e
net/http.send(0xc0008de900, 0x1ec5a00, 0x256f3c0, 0x0, 0x0, 0x0, 0x0, 0x1c00850, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:250 +0x4ae
net/http.(*Client).send(0x257b460, 0xc0008de900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0x1e2
net/http.(*Client).do(0x257b460, 0xc0008de900, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:641 +0x10af
net/http.(*Client).Do(0x257b460, 0xc0008de900, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:509 +0x5e
github.com/aws/aws-sdk-go/aws/corehandlers.sendFollowRedirects(0xc000045400, 0x0, 0x0, 0x0)
	/Users/username/go/src/github.com/aws/aws-sdk-go/aws/corehandlers/handlers.go:120 +0x72
github.com/aws/aws-sdk-go/aws/corehandlers.glob..func3(0xc000045400)
	/Users/username/go/src/github.com/aws/aws-sdk-go/aws/corehandlers/handlers.go:112 +0x246
github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc0000455a8, 0xc000045400)
	/Users/username/go/src/github.com/aws/aws-sdk-go/aws/request/handlers.go:213 +0xf2
github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc000045400, 0x0, 0x0)
	/Users/username/go/src/github.com/aws/aws-sdk-go/aws/request/request.go:493 +0x553
github.com/aws/aws-sdk-go/service/s3.(*S3).GetBucketLocation(0xc0004c2530, 0xc0004c2538, 0x0, 0x0, 0x0)
	/Users/username/go/src/github.com/aws/aws-sdk-go/service/s3/api.go:2141 +0x92
bitbucket.org/{{redacted}}(0xc000199940, 0x0)
	/Users/username/go/src/bitbucket.org/{{redacted}}:199 +0x10b
bitbucket.org/{{redacted}}(0xc000199940, 0x0, 0x0)
	/Users/username/go/src/bitbucket.org/{{redacted}}:109 +0x4e
bitbucket.org/{{redacted}}(0xc000199940, 0x0, 0x0)
	/Users/username/go/src/bitbucket.org/{{redacted}}:94 +0x37
bitbucket.org/{{redacted}}(0xc000874900)
	/Users/username/go/src/bitbucket.org/{{redacted}}:60 +0xa1
bitbucket.org/{{redacted}}(0xc000874900)
	/Users/username/go/src/bitbucket.org/{{redacted}}:54 +0x49
created by bitbucket.org/{{redacted}}
	/Users/username/go/src/bitbucket.org/{{redacted}}:45 +0x3af

goroutine 108 [chan receive]:
net/http.(*persistConn).addTLS(0xc0008fb560, 0xc000632b60, 0x10, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1177 +0x246
net/http.(*Transport).dialConn(0x256f3c0, 0x1ecc620, 0xc00009c008, 0xc000331d00, 0xc000680780, 0x5, 0xc000632b60, 0x14, 0x0, 0x0, ...)
	/usr/local/go/src/net/http/transport.go:1322 +0xf31
net/http.(*Transport).getConn.func4(0x256f3c0, 0x1ecc620, 0xc00009c008, 0xc0009524e0, 0xc00034ac60)
	/usr/local/go/src/net/http/transport.go:999 +0x8d
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:998 +0x83d
fatal error: unexpected signal during runtime execution
panic during panic
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8c96b98 pc=0x8c167e7]

runtime stack:
runtime.throw(0x1bea596, 0x2a)
	/usr/local/go/src/runtime/panic.go:608 +0x72
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:374 +0x288
fatal error: unexpected signal during runtime execution
stack trace unavailable

[ianlancetaylor]

You say that the crash is random. Is it always in crypto/x509.loadSystemRoots, or do you have different stack traces?

CC @randall77

[akamensky]

@ianlancetaylor when I said random means that it happens at random, not that it happens with random functions. The same code works fine until I just get this stack trace on the process that has been running for quite a bit, all with same compiled binary.

I did not compare stack traces so far. I will catch a couple more and report back on this if they all crash in same function or not.

[akamensky]

Sorry for the delay, confirmed that it breaks in same place for me, always same stack trace. This happens when running anything that loads SSL/TLS related things, and as noted before it does not happen consistently, but rather at random.

For now I am falling back to 1.10.4 which works fine (I guess something that has not been backported to 1.10 branch is causing this?)

[aclements]

This traceback looks to me like the C code invoked by crypto/x509 had a bad memory reference. Re-titling to reflect that this is most likely an issue in crypto/x509, not in the runtime.

[FiloSottile]

@akamensky Can you check if this is still an issue at tip with the new macOS roots code?

[akamensky]

@FiloSottile I currently have no way to verify whether this issue is resolved or not. Meanwhile after doing completely clean install of 1.11.4 (and now 1.11.5) with removing entire $GOPATH/* and then rebuilding everything I had not seen this issue for awhile. Which possibly suggests it may have been caused by using different go compiler version / different standard library version to compile one of dependencies and then linking it with binary(?)

Since I have not seen this issue for awhile, I think okay to close (I guess can reopen if starts appearing again?)