Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/x509: frequent panics when doing HTTPS requests on Darwin #30763

Closed
leonklingele opened this issue Mar 12, 2019 · 18 comments

Comments

Projects
None yet
6 participants
@leonklingele
Copy link
Contributor

commented Mar 12, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/USER/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/USER/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/jv/n5rf3drx1td1ljkt6xk2_60w0000gn/T/go-build096456041=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

EDIT:
It turned out this not only affects go get, but any HTTPS requests on Darwin. See the full conversation in this thread.

Run the following command to get get -u a bunch of apps and libraries (doesn't really matter which ones you get, just make sure you get a lot so this reproduces quicker):

$ bash -x -c 'go get -u golang.org/x/tools/cmd/goimports ; go get -u golang.org/x/tools/cmd/gorename ; go get -u github.com/sqs/goreturns ; go get -u github.com/mdempsky/gocode ; go get -u github.com/alecthomas/gometalinter ; go get -u github.com/mgechev/revive ; go get -u github.com/zmb3/gogetdoc ; go get -u github.com/zmb3/goaddimport ; go get -u github.com/rogpeppe/godef ; go get -u golang.org/x/tools/cmd/guru ; go get -u github.com/fatih/gomodifytags ; go get -u github.com/tpng/gopkgs ; go get -u github.com/ramya-rao-a/go-outline'

What did you expect to see?

go get -u should succeed for every one of them.

What did you see instead?

A panic from time to time. Example output when running above command, two panic were issued:

+ go get -u golang.org/x/tools/cmd/goimports
+ go get -u golang.org/x/tools/cmd/gorename
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1f19e0 pc=0x7fff9b6914dd]

runtime stack:
runtime.throw(0x15ee5ed, 0x2a)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
	/usr/local/Cellar/go/1.12/libexec/src/runtime/signal_unix.go:374 +0x4a9

goroutine 13 [syscall]:
runtime.cgocall(0x1001740, 0xc000479470, 0xc0000272d0)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc000479440 sp=0xc000479408 pc=0x1004b8b
crypto/x509._Cfunc_FetchPEMRoots(0xc0000272c8, 0xc0000272d0, 0xc00011cf00, 0x0)
	_cgo_gotypes.go:110 +0x4d fp=0xc000479470 sp=0xc000479440 pc=0x12fea5d
crypto/x509.loadSystemRoots.func1(0xc0000272c8, 0xc0000272d0, 0x1b)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root_cgo_darwin.go:281 +0x12d fp=0xc0004794b0 sp=0xc000479470 pc=0x1302b7d
crypto/x509.loadSystemRoots(0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root_cgo_darwin.go:281 +0xec fp=0xc000479580 sp=0xc0004794b0 pc=0x12febcc
crypto/x509.initSystemRoots()
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root.go:21 +0x26 fp=0xc0004795a8 sp=0xc000479580 pc=0x12f20b6
sync.(*Once).Do(0x1a69fd8, 0x1618780)
	/usr/local/Cellar/go/1.12/libexec/src/sync/once.go:44 +0xb3 fp=0xc0004795d8 sp=0xc0004795a8 pc=0x106e6a3
crypto/x509.systemRootsPool(...)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root.go:16
crypto/x509.(*Certificate).Verify(0xc0000d0c00, 0xc000406f20, 0xa, 0xc0004d9620, 0x0, 0xbf19f60847452520, 0x799b344, 0x1a4cd00, 0x0, 0x0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/verify.go:744 +0x6b8 fp=0xc0004796f0 sp=0xc0004795d8 pc=0x12f5c58
crypto/tls.(*Conn).verifyServerCertificate(0xc00034c000, 0xc0004d8f60, 0x2, 0x2, 0x145b, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:838 +0x26a fp=0xc000479888 sp=0xc0004796f0 pc=0x1327d1a
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc000479dc8, 0xc0000ee840, 0x155)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:454 +0x1872 fp=0xc000479bc0 sp=0xc000479888 pc=0x1325e82
crypto/tls.(*clientHandshakeState).handshake(0xc000479dc8, 0xc00016a7e0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:399 +0x3fb fp=0xc000479cc8 sp=0xc000479bc0 pc=0x132423b
crypto/tls.(*Conn).clientHandshake(0xc00034c000, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:208 +0x2cd fp=0xc000479f38 sp=0xc000479cc8 pc=0x132288d
crypto/tls.(*Conn).Handshake(0xc00034c000, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/conn.go:1343 +0xef fp=0xc000479f78 sp=0xc000479f38 pc=0x1320eaf
net/http.(*persistConn).addTLS.func2(0x0, 0xc00034c000, 0xc00022ddb0, 0xc00042d500)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1190 +0x42 fp=0xc000479fc0 sp=0xc000479f78 pc=0x13cd282
runtime.goexit()
	/usr/local/Cellar/go/1.12/libexec/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000479fc8 sp=0xc000479fc0 pc=0x10597b1
created by net/http.(*persistConn).addTLS
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1186 +0x1ab

goroutine 1 [select]:
net/http.(*Transport).getConn(0x1a44220, 0xc0004d8e40, 0x0, 0xc000302340, 0x5, 0xc000406f20, 0xe, 0x0, 0x0, 0x0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1020 +0x63d
net/http.(*Transport).roundTrip(0x1a44220, 0xc00012ce00, 0x20, 0x50, 0xc00036ade0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:467 +0x76e
net/http.(*Transport).RoundTrip(0x1a44220, 0xc00012ce00, 0x1a44220, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc00012ce00, 0x16b6fa0, 0x1a44220, 0x0, 0x0, 0x0, 0xc0000c2240, 0xc00036b008, 0x1, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:250 +0x202
net/http.(*Client).send(0x1a4cb00, 0xc00012ce00, 0x0, 0x0, 0x0, 0xc0000c2240, 0x0, 0x1, 0xc00007a000)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:174 +0xfb
net/http.(*Client).do(0x1a4cb00, 0xc00012ce00, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:641 +0x279
net/http.(*Client).Do(...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:509
net/http.(*Client).Get(0x1a4cb00, 0xc000302340, 0x30, 0x0, 0x0, 0x7fff5fbff962)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:398 +0x9e
cmd/go/internal/web.GetMaybeInsecure.func1(0x15d8458, 0x5, 0xffffffff0000006f, 0xc00036b2d0, 0x106e74a, 0x1a4c1d0, 0x10dfadd)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/web/http.go:90 +0x13f
cmd/go/internal/web.GetMaybeInsecure(0x7fff5fbff962, 0x1f, 0x0, 0x1, 0x6, 0x114c400, 0xc000100210, 0x40000c000100210, 0x16c40c0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/web/http.go:99 +0x9d
cmd/go/internal/get.repoRootForImportDynamic(0x7fff5fbff962, 0x1f, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/vcs.go:764 +0x105
cmd/go/internal/get.RepoRootForImportPath(0x7fff5fbff962, 0x1f, 0x0, 0x0, 0x21, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/vcs.go:650 +0x3b1
cmd/go/internal/get.downloadPackage(0xc000170480, 0xc00011eed0, 0x7fff5fbff962)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:441 +0x13df
cmd/go/internal/get.download(0x7fff5fbff962, 0x1f, 0x0, 0xc000202c60, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:285 +0xd1b
cmd/go/internal/get.runGet(0x1a41c60, 0xc0000b80b0, 0x1, 0x1)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:167 +0x194
main.main()
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/main.go:219 +0x837

goroutine 34 [syscall]:
os/signal.signal_recv(0x0)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/sigqueue.go:139 +0x9f
os/signal.loop()
	/usr/local/Cellar/go/1.12/libexec/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/usr/local/Cellar/go/1.12/libexec/src/os/signal/signal_unix.go:29 +0x41

goroutine 21 [chan receive]:
net/http.(*persistConn).addTLS(0xc00032ec60, 0xc000406f20, 0xa, 0x0, 0xc000406f2b, 0x3)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1196 +0x1d4
net/http.(*Transport).dialConn(0x1a44220, 0x16c2b40, 0xc0000b6010, 0x0, 0xc000302340, 0x5, 0xc000406f20, 0xe, 0x0, 0x0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1269 +0x1cba
net/http.(*Transport).getConn.func4(0x1a44220, 0x16c2b40, 0xc0000b6010, 0xc0004d8e70, 0xc0001a5c20)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1015 +0xa6
created by net/http.(*Transport).getConn
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1014 +0x455
+ go get -u github.com/sqs/goreturns
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4c1c78 pc=0x7fff9b6914dd]

runtime stack:
runtime.throw(0x15ee5ed, 0x2a)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
	/usr/local/Cellar/go/1.12/libexec/src/runtime/signal_unix.go:374 +0x4a9

goroutine 62 [syscall]:
runtime.cgocall(0x1001740, 0xc0005ab470, 0xc0000b7e40)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc0005ab440 sp=0xc0005ab408 pc=0x1004b8b
crypto/x509._Cfunc_FetchPEMRoots(0xc0000b7e38, 0xc0000b7e40, 0xc000069e00, 0x0)
	_cgo_gotypes.go:110 +0x4d fp=0xc0005ab470 sp=0xc0005ab440 pc=0x12fea5d
crypto/x509.loadSystemRoots.func1(0xc0000b7e38, 0xc0000b7e40, 0x1b)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root_cgo_darwin.go:281 +0x12d fp=0xc0005ab4b0 sp=0xc0005ab470 pc=0x1302b7d
crypto/x509.loadSystemRoots(0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root_cgo_darwin.go:281 +0xec fp=0xc0005ab580 sp=0xc0005ab4b0 pc=0x12febcc
crypto/x509.initSystemRoots()
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root.go:21 +0x26 fp=0xc0005ab5a8 sp=0xc0005ab580 pc=0x12f20b6
sync.(*Once).Do(0x1a69fd8, 0x1618780)
	/usr/local/Cellar/go/1.12/libexec/src/sync/once.go:44 +0xb3 fp=0xc0005ab5d8 sp=0xc0005ab5a8 pc=0x106e6a3
crypto/x509.systemRootsPool(...)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/root.go:16
crypto/x509.(*Certificate).Verify(0xc000309080, 0xc0000d6a60, 0xa, 0xc000423e90, 0x0, 0xbf19f608b70f46a0, 0x658c14ae, 0x1a4cd00, 0x0, 0x0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/x509/verify.go:744 +0x6b8 fp=0xc0005ab6f0 sp=0xc0005ab5d8 pc=0x12f5c58
crypto/tls.(*Conn).verifyServerCertificate(0xc0003c6e00, 0xc0004238f0, 0x2, 0x2, 0x145b, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:838 +0x26a fp=0xc0005ab888 sp=0xc0005ab6f0 pc=0x1327d1a
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc0005abdc8, 0xc0001042c0, 0x155)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:454 +0x1872 fp=0xc0005abbc0 sp=0xc0005ab888 pc=0x1325e82
crypto/tls.(*clientHandshakeState).handshake(0xc0005abdc8, 0xc0003e46c0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:399 +0x3fb fp=0xc0005abcc8 sp=0xc0005abbc0 pc=0x132423b
crypto/tls.(*Conn).clientHandshake(0xc0003c6e00, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/handshake_client.go:208 +0x2cd fp=0xc0005abf38 sp=0xc0005abcc8 pc=0x132288d
crypto/tls.(*Conn).Handshake(0xc0003c6e00, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/crypto/tls/conn.go:1343 +0xef fp=0xc0005abf78 sp=0xc0005abf38 pc=0x1320eaf
net/http.(*persistConn).addTLS.func2(0x0, 0xc0003c6e00, 0xc0004c6780, 0xc000137f20)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1190 +0x42 fp=0xc0005abfc0 sp=0xc0005abf78 pc=0x13cd282
runtime.goexit()
	/usr/local/Cellar/go/1.12/libexec/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc0005abfc8 sp=0xc0005abfc0 pc=0x10597b1
created by net/http.(*persistConn).addTLS
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1186 +0x1ab

goroutine 1 [select]:
net/http.(*Transport).getConn(0x1a44220, 0xc0004233b0, 0x0, 0xc0003d8ec0, 0x5, 0xc0000d6a60, 0xe, 0x0, 0x0, 0x0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1020 +0x63d
net/http.(*Transport).roundTrip(0x1a44220, 0xc000108400, 0x20, 0x50, 0xc00035abf0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:467 +0x76e
net/http.(*Transport).RoundTrip(0x1a44220, 0xc000108400, 0x1a44220, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc000108400, 0x16b6fa0, 0x1a44220, 0x0, 0x0, 0x0, 0xc0000c41d8, 0xc00035ae18, 0x1, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:250 +0x202
net/http.(*Client).send(0x1a4cb00, 0xc000108400, 0x0, 0x0, 0x0, 0xc0000c41d8, 0x0, 0x1, 0x1a4e020)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:174 +0xfb
net/http.(*Client).do(0x1a4cb00, 0xc000108400, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:641 +0x279
net/http.(*Client).Do(...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:509
net/http.(*Client).Get(0x1a4cb00, 0xc0003d8ec0, 0x2b, 0x0, 0x0, 0xc0001111c1)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/client.go:398 +0x9e
cmd/go/internal/web.GetMaybeInsecure.func1(0x15d8458, 0x5, 0xffffffff0000006f, 0xc00035b0e0, 0x106e74a, 0x1a4c1d0, 0x10dfadd)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/web/http.go:90 +0x13f
cmd/go/internal/web.GetMaybeInsecure(0xc0001111c1, 0x1a, 0x0, 0x1, 0x6, 0x114c400, 0xc000184630, 0x40000c000184630, 0x16c40c0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/web/http.go:99 +0x9d
cmd/go/internal/get.repoRootForImportDynamic(0xc0001111c1, 0x1a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/vcs.go:764 +0x105
cmd/go/internal/get.RepoRootForImportPath(0xc0001111c1, 0x1a, 0x0, 0x0, 0x21, 0x0, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/vcs.go:650 +0x3b1
cmd/go/internal/get.downloadPackage(0xc00046e000, 0xc0000a75c0, 0xc0001111c1)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:441 +0x13df
cmd/go/internal/get.download(0xc0001111c1, 0x1a, 0xc000505200, 0xc0000aef20, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:285 +0xd1b
cmd/go/internal/get.download(0x7fff5fbff96a, 0x18, 0x0, 0xc0000aef20, 0x0)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:381 +0x70d
cmd/go/internal/get.runGet(0x1a41c60, 0xc0000c8030, 0x1, 0x1)
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/internal/get/get.go:167 +0x194
main.main()
	/usr/local/Cellar/go/1.12/libexec/src/cmd/go/main.go:219 +0x837

goroutine 18 [syscall]:
os/signal.signal_recv(0x0)
	/usr/local/Cellar/go/1.12/libexec/src/runtime/sigqueue.go:139 +0x9f
os/signal.loop()
	/usr/local/Cellar/go/1.12/libexec/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/usr/local/Cellar/go/1.12/libexec/src/os/signal/signal_unix.go:29 +0x41

goroutine 56 [chan receive]:
net/http.(*persistConn).addTLS(0xc000376120, 0xc0000d6a60, 0xa, 0x0, 0xc0000d6a6b, 0x3)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1196 +0x1d4
net/http.(*Transport).dialConn(0x1a44220, 0x16c2b40, 0xc0000d6000, 0x0, 0xc0003d8ec0, 0x5, 0xc0000d6a60, 0xe, 0x0, 0x10544a0, ...)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1269 +0x1cba
net/http.(*Transport).getConn.func4(0x1a44220, 0x16c2b40, 0xc0000d6000, 0xc0004233e0, 0xc0000951a0)
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1015 +0xa6
created by net/http.(*Transport).getConn
	/usr/local/Cellar/go/1.12/libexec/src/net/http/transport.go:1014 +0x455
+ go get -u github.com/mdempsky/gocode
+ go get -u github.com/alecthomas/gometalinter
+ go get -u github.com/mgechev/revive
+ go get -u github.com/zmb3/gogetdoc
+ go get -u github.com/zmb3/goaddimport
+ go get -u github.com/rogpeppe/godef
+ go get -u golang.org/x/tools/cmd/guru
+ go get -u github.com/fatih/gomodifytags
+ go get -u github.com/tpng/gopkgs
+ go get -u github.com/ramya-rao-a/go-outline
@agnivade

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

Does this happen in 1.11 too ?

@agnivade agnivade added this to the Go1.13 milestone Mar 12, 2019

@agnivade agnivade changed the title cmd/go: go get -u panics from time to time cmd/go: get -u panics from time to time on darwin Mar 12, 2019

@agnivade

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

/cc @FiloSottile @agl for crypto. @ianlancetaylor for cgo.

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented Mar 12, 2019

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2019

Sorry for the late response. I was unable to reproduce this with Go 1.11.5 on OS X 10.11.6 (15G22010).

Go 1.12 makes using HTTPS requests quite useless as just about every third one produces a panic.

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2019

I bisected it down to f6be1cf which apparently introduced the issue.
Please escalate this ticket.

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2019

Here's the output of the original go get -u bash script with a GODEBUG="x509roots=1" env var:

+ go get -u golang.org/x/tools/cmd/goimports
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: dlv-cert returned 1
crypto/x509: Equifax Secure Certificate Authority returned 4
crypto/x509: GTE CyberTrust Global Root returned 4
crypto/x509: Thawte Premium Server CA returned 4
crypto/x509: Thawte Server CA returned 4
crypto/x509: Class 3 Public Primary Certification Authority returned 4
crypto/x509: mail.leonklingele.de returned 4

[SOME MORE -- REDACTED BY LEON]

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1f1da8 pc=0x7fff9b6914dd]

runtime stack:
runtime.throw(0x15e22bb, 0x2a)
	/path/to/go/src/runtime/panic.go:608 +0x72
runtime.sigpanic()
	/path/to/go/src/runtime/signal_unix.go:374 +0x49c

goroutine 45 [syscall]:
runtime.cgocall(0x1001740, 0xc0003575f0, 0xc0004d9e80)
	/path/to/go/src/runtime/cgocall.go:128 +0x5b fp=0xc0003575c0 sp=0xc000357588 pc=0x1004b6b
crypto/x509._Cfunc_FetchPEMRoots(0xc0004d9e78, 0xc0004d9e80, 0xc000069a01, 0x0)
	_cgo_gotypes.go:110 +0x4d fp=0xc0003575f0 sp=0xc0003575c0 pc=0x12f8c2d
crypto/x509.loadSystemRoots.func1(0xc0004d9e78, 0xc0004d9e80, 0x0)
	/path/to/go/src/crypto/x509/root_cgo_darwin.go:281 +0x12d fp=0xc000357630 sp=0xc0003575f0 pc=0x12fc37d
crypto/x509.loadSystemRoots(0x0, 0x0, 0x0)
	/path/to/go/src/crypto/x509/root_cgo_darwin.go:281 +0xeb fp=0xc000357700 sp=0xc000357630 pc=0x12f8d9b
crypto/x509.initSystemRoots()
	/path/to/go/src/crypto/x509/root.go:21 +0x26 fp=0xc000357728 sp=0xc000357700 pc=0x12ebb76
sync.(*Once).Do(0x1a40dd8, 0x160c158)
	/path/to/go/src/sync/once.go:44 +0xb3 fp=0xc000357758 sp=0xc000357728 pc=0x106d293
crypto/x509.systemRootsPool(...)
	/path/to/go/src/crypto/x509/root.go:16
crypto/x509.(*Certificate).Verify(0xc0000e5b80, 0xc0004d8180, 0xa, 0xc00054ecc0, 0x0, 0xbf1aa65496247cd0, 0x8454ee8, 0x1a23b20, 0x0, 0x0, ...)
	/path/to/go/src/crypto/x509/verify.go:744 +0x705 fp=0xc0003579a8 sp=0xc000357758 pc=0x12ef745
crypto/tls.(*Conn).verifyServerCertificate(0xc0003cd880, 0xc00054e7e0, 0x2, 0x2, 0x1555, 0x0)
	/path/to/go/src/crypto/tls/handshake_client.go:838 +0x268 fp=0xc000357b40 sp=0xc0003579a8 pc=0x1321028
crypto/tls.(*clientHandshakeStateTLS13).readServerCertificate(0xc000357e70, 0x0, 0x0)
	/path/to/go/src/crypto/tls/handshake_client_tls13.go:435 +0x1d5 fp=0xc000357c78 sp=0xc000357b40 pc=0x1324a95
crypto/tls.(*clientHandshakeStateTLS13).handshake(0xc000357e70, 0xc000258000, 0x0)
	/path/to/go/src/crypto/tls/handshake_client_tls13.go:85 +0x21b fp=0xc000357cc8 sp=0xc000357c78 pc=0x13224eb
crypto/tls.(*Conn).clientHandshake(0xc0003cd880, 0x0, 0x0)
	/path/to/go/src/crypto/tls/handshake_client.go:198 +0x4a4 fp=0xc000357f38 sp=0xc000357cc8 pc=0x131bd94
crypto/tls.(*Conn).Handshake(0xc0003cd880, 0x0, 0x0)
	/path/to/go/src/crypto/tls/conn.go:1343 +0xee fp=0xc000357f78 sp=0xc000357f38 pc=0x131a1fe
net/http.(*persistConn).addTLS.func2(0x0, 0xc0003cd880, 0xc0004ba910, 0xc000556960)
	/path/to/go/src/net/http/transport.go:1190 +0x42 fp=0xc000357fc0 sp=0xc000357f78 pc=0x13c5072
runtime.goexit()
	/path/to/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000357fc8 sp=0xc000357fc0 pc=0x10588c1
created by net/http.(*persistConn).addTLS
	/path/to/go/src/net/http/transport.go:1186 +0x1a9

goroutine 1 [select]:
net/http.(*Transport).getConn(0x1a1b200, 0xc00054e3f0, 0x0, 0xc0002fc2c0, 0x5, 0xc0004d8180, 0xe, 0x0, 0x0, 0x0, ...)
	/path/to/go/src/net/http/transport.go:1020 +0x63a
net/http.(*Transport).roundTrip(0x1a1b200, 0xc00043e000, 0x20, 0x50, 0xc00037ee48)
	/path/to/go/src/net/http/transport.go:467 +0x764
net/http.(*Transport).RoundTrip(0x1a1b200, 0xc00043e000, 0x1a1b200, 0x0, 0x0)
	/path/to/go/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc00043e000, 0x169bc20, 0x1a1b200, 0x0, 0x0, 0x0, 0xc0000b00b0, 0x0, 0x1, 0x0)
	/path/to/go/src/net/http/client.go:250 +0x1fd
net/http.(*Client).send(0x1a23920, 0xc00043e000, 0x0, 0x0, 0x0, 0xc0000b00b0, 0x0, 0x1, 0xc0004a6000)
	/path/to/go/src/net/http/client.go:174 +0xfa
net/http.(*Client).do(0x1a23920, 0xc00043e000, 0x0, 0x0, 0x0)
	/path/to/go/src/net/http/client.go:641 +0x279
net/http.(*Client).Do(...)
	/path/to/go/src/net/http/client.go:509
net/http.(*Client).Get(0x1a23920, 0xc0002fc2c0, 0x31, 0x0, 0x0, 0x7fff5fbff88a)
	/path/to/go/src/net/http/client.go:398 +0x9d
cmd/go/internal/web.GetMaybeInsecure.func1(0x15cc25c, 0x5, 0xffffffff0000006f, 0xc00037f338, 0x106d33a, 0x1a22ff0, 0x10dd415)
	/path/to/go/src/cmd/go/internal/web/http.go:90 +0x13f
cmd/go/internal/web.GetMaybeInsecure(0x7fff5fbff88a, 0x20, 0x0, 0x1, 0x6, 0x1148f00, 0xc0002ae370, 0x40000c0002ae370, 0x16a8c60)
	/path/to/go/src/cmd/go/internal/web/http.go:99 +0x9d
cmd/go/internal/get.repoRootForImportDynamic(0x7fff5fbff88a, 0x20, 0x0, 0x0, 0x0, 0x0, 0x0)
	/path/to/go/src/cmd/go/internal/get/vcs.go:770 +0xfb
cmd/go/internal/get.RepoRootForImportPath(0x7fff5fbff88a, 0x20, 0x0, 0x0, 0x21, 0x0, 0x0)
	/path/to/go/src/cmd/go/internal/get/vcs.go:657 +0x434
cmd/go/internal/get.downloadPackage(0xc00017e900, 0xc0001fb710, 0x7fff5fbff88a)
	/path/to/go/src/cmd/go/internal/get/get.go:424 +0x12fc
cmd/go/internal/get.download(0x7fff5fbff88a, 0x20, 0x0, 0xc00020f2e0, 0x0)
	/path/to/go/src/cmd/go/internal/get/get.go:285 +0xd17
cmd/go/internal/get.runGet(0x1a18c40, 0xc0000b4030, 0x1, 0x1)
	/path/to/go/src/cmd/go/internal/get/get.go:167 +0x194
main.main()
	/path/to/go/src/cmd/go/main.go:219 +0x82d

goroutine 34 [syscall]:
os/signal.signal_recv(0x0)
	/path/to/go/src/runtime/sigqueue.go:139 +0x9f
os/signal.loop()
	/path/to/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/path/to/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 39 [chan receive]:
net/http.(*persistConn).addTLS(0xc0001786c0, 0xc0004d8180, 0xa, 0x0, 0xc0004d818b, 0x3)
	/path/to/go/src/net/http/transport.go:1196 +0x1d2
net/http.(*Transport).dialConn(0x1a1b200, 0x16a7720, 0xc0000c4000, 0x0, 0xc0002fc2c0, 0x5, 0xc0004d8180, 0xe, 0x0, 0x0, ...)
	/path/to/go/src/net/http/transport.go:1269 +0x1c9e
net/http.(*Transport).getConn.func4(0x1a1b200, 0x16a7720, 0xc0000c4000, 0xc00054e420, 0xc0001b0300)
	/path/to/go/src/net/http/transport.go:1015 +0xa6
created by net/http.(*Transport).getConn
	/path/to/go/src/net/http/transport.go:1014 +0x453
@agnivade

This comment has been minimized.

Copy link
Member

commented Mar 14, 2019

Thanks for the bisect ! Tentatively mark it as a release blocker because this seems like a basic behavior. Also probably warrants a backport. @FiloSottile, feel free to decide otherwise.

@FiloSottile FiloSottile self-assigned this Mar 14, 2019

marques-work added a commit to gocd-contrib/docker-golang-image that referenced this issue Mar 19, 2019

@marques-work

This comment has been minimized.

Copy link

commented Apr 2, 2019

@agnivade @FiloSottile Will a fix for this be going on in the next release?

@abligh

This comment has been minimized.

Copy link

commented Apr 8, 2019

Note that per dupe of #30889 this appears to affect any https connection initiated from golang (reproduced here with a clean install of go1.12.2 darwin/amd64 on OS-X 10.11.6 (15G22010)). Suggest changing the title to note the rather broader effect?

@leonklingele leonklingele changed the title cmd/go: get -u panics from time to time on darwin net/http: frequent panics when doing HTTPS requests on Darwin Apr 8, 2019

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented Apr 15, 2019

Hey, is anyone trying to fix HTTPS on OS X in any Go stable release? :)

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented May 10, 2019

I suggest to revert f6be1cf until someone familiar with Objective-C finds the time to fix whatever went wrong with that change.

@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented May 10, 2019

Minimal reproducible:

package main

import (
	"net/http"
)

func main() {
	resp, err := http.Get("https://golang.org/")
	if err != nil {
		panic(err)
	}

	println(resp.StatusCode, http.StatusText(resp.StatusCode))
}
$ go run main.go
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x173e20 pc=0x7fff9dcad4dd]

runtime stack:
runtime.throw(0x12c375a, 0x2a)
	/usr/local/Cellar/go/1.12.5/libexec/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
	/usr/local/Cellar/go/1.12.5/libexec/src/runtime/signal_unix.go:374 +0x4a9
[etc. see above]

@bradfitz bradfitz changed the title net/http: frequent panics when doing HTTPS requests on Darwin crypto/x509: frequent panics when doing HTTPS requests on Darwin May 16, 2019

@FiloSottile

This comment has been minimized.

Copy link
Member

commented May 21, 2019

It's hard to tell without symbols, but it should be a duplicate of #28092 (comment).

@gopherbot

This comment has been minimized.

Copy link

commented May 22, 2019

Change https://golang.org/cl/178537 mentions this issue: crypto/x509: fix value ownership in isSSLPolicy on macOS

@gopherbot gopherbot closed this in a3d4655 May 22, 2019

@FiloSottile

This comment has been minimized.

Copy link
Member

commented May 22, 2019

This should be now fixed at tip. Please test it with https://golang.org/dl/gotip and report back. If it works, I am going to file this for cherry-picking.

$ go get golang.org/dl/gotip
$ gotip download
$ GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
$ gotip run [YOUR_PROGRAM]
@leonklingele

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2019

Thanks for the fix, @FiloSottile! https://golang.org/cl/178537 has resolved the issue! 🎉 Tests also pass.

@gopherbot

This comment has been minimized.

Copy link

commented May 29, 2019

Change https://golang.org/cl/179339 mentions this issue: [release-branch.go1.12] crypto/x509: fix value ownership in isSSLPolicy on macOS

@gopherbot

This comment has been minimized.

Copy link

commented May 29, 2019

Change https://golang.org/cl/179340 mentions this issue: [release-branch.go1.11] crypto/x509: fix value ownership in isSSLPolicy on macOS

gopherbot pushed a commit that referenced this issue Jun 7, 2019

[release-branch.go1.11] crypto/x509: fix value ownership in isSSLPoli…
…cy on macOS

CFDictionaryGetValueIfPresent does not take ownership of the value, so
releasing the properties dictionary before passing the value to CFEqual
can crash. Not really clear why this works most of the time.

See https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFMemoryMgmt/Concepts/Ownership.html

Fixes #32281
Updates #28092
Updates #30763

Change-Id: I5ee7ca276b753a48abc3aedfb78b8af68b448dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/178537
Reviewed-by: Adam Langley <agl@golang.org>
(cherry picked from commit a3d4655)
Reviewed-on: https://go-review.googlesource.com/c/go/+/179340
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

gopherbot pushed a commit that referenced this issue Jun 7, 2019

[release-branch.go1.12] crypto/x509: fix value ownership in isSSLPoli…
…cy on macOS

CFDictionaryGetValueIfPresent does not take ownership of the value, so
releasing the properties dictionary before passing the value to CFEqual
can crash. Not really clear why this works most of the time.

See https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFMemoryMgmt/Concepts/Ownership.html

Fixes #32282
Updates #28092
Updates #30763

Change-Id: I5ee7ca276b753a48abc3aedfb78b8af68b448dd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/178537
Reviewed-by: Adam Langley <agl@golang.org>
(cherry picked from commit a3d4655)
Reviewed-on: https://go-review.googlesource.com/c/go/+/179339
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.