crypto/x509: honor OS X certificate trust settings #18141
Comments
|
CL https://golang.org/cl/33721 mentions this issue. |
Darwin separately stores bits indicating whether a root certificate should be trusted; this changes Go to read and use those when initializing SystemCertPool. Unfortunately, the trust API is very slow. To avoid a delay of up to 0.5s in initializing the system cert pool, we assume that the trust settings found in kSecTrustSettingsDomainSystem will always indicate trust. (That is, all root certs Apple distributes are trusted.) This is not guaranteed by the API but is true in practice. In the non-cgo codepath, we do not have that benefit, so we must check the trust status of every certificate. This causes about 0.5s of delay in initializing the SystemCertPool. On OS X 10.11 and older, the "security" command requires a certificate to be provided in a file and not on stdin, so the non-cgo codepath creates temporary files for each certificate, further slowing initialization. Updates #18141. Change-Id: If681c514047afe5e1a68de6c9d40ceabbce54755 Reviewed-on: https://go-review.googlesource.com/33721 Run-TryBot: Quentin Smith <quentin@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
|
CL https://golang.org/cl/33727 mentions this issue. |
|
CL https://golang.org/cl/33728 mentions this issue. |
…ot CAs Darwin separately stores bits indicating whether a root certificate should be trusted; this changes Go to read and use those when initializing SystemCertPool. Unfortunately, the trust API is very slow. To avoid a delay of up to 0.5s in initializing the system cert pool, we assume that the trust settings found in kSecTrustSettingsDomainSystem will always indicate trust. (That is, all root certs Apple distributes are trusted.) This is not guaranteed by the API but is true in practice. In the non-cgo codepath, we do not have that benefit, so we must check the trust status of every certificate. This causes about 0.5s of delay in initializing the SystemCertPool. On OS X 10.11 and older, the "security" command requires a certificate to be provided in a file and not on stdin, so the non-cgo codepath creates temporary files for each certificate, further slowing initialization. Updates #18141. Change-Id: If681c514047afe5e1a68de6c9d40ceabbce54755 Reviewed-on: https://go-review.googlesource.com/33721 Run-TryBot: Quentin Smith <quentin@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-on: https://go-review.googlesource.com/33727
…ot CAs Darwin separately stores bits indicating whether a root certificate should be trusted; this changes Go to read and use those when initializing SystemCertPool. Unfortunately, the trust API is very slow. To avoid a delay of up to 0.5s in initializing the system cert pool, we assume that the trust settings found in kSecTrustSettingsDomainSystem will always indicate trust. (That is, all root certs Apple distributes are trusted.) This is not guaranteed by the API but is true in practice. In the non-cgo codepath, we do not have that benefit, so we must check the trust status of every certificate. This causes about 0.5s of delay in initializing the SystemCertPool. On OS X 10.11 and older, the "security" command requires a certificate to be provided in a file and not on stdin, so the non-cgo codepath creates temporary files for each certificate, further slowing initialization. Updates #18141. Change-Id: If681c514047afe5e1a68de6c9d40ceabbce54755 Reviewed-on: https://go-review.googlesource.com/33721 Run-TryBot: Quentin Smith <quentin@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-on: https://go-review.googlesource.com/33728
bradfitz
changed the title
|
I'm using terraform which has included this fix in it's latest version. The problem I'm having on my mac is that I'm getting |
|
@micahlmartin The changes for this bug should not be able to affect TLS handshake timeouts. Please open a new bug. |
|
This change appears to be causing intermittent "x509 certificate not trusted" errors for our non-cgo built Darwin binaries in Terraform: hashicorp/terraform#10779 (comment) Starting with the release where we switched to 1.7.4, a number of users (not just the above issue) have reported that they intermittently get "x509 certificate not trusted" errors. As you can see from the issue, running it in a loop one user is seeing a 10% failure rate. I'm not 100% sure this change is causing it, but in the version that this started happening, we had no changes related to the code where users are reporting this issue and we updated to Go 1.7.4. Our builds do not use cgo since we cross-compile Darwin builds from Linux. |
|
@mitchellh, can you verify whether https://go-review.googlesource.com/c/34389/ reduces or eliminates the flakiness? As a speed optimization we tried piping into the macOS "security" tool, which works on Sierra, but it seems flaky, so the CL above passes it via a temp file always which is slower but seems to actually work. I hope that's what you're seeing. |
|
Hey @bradfitz I'll try to test that later today. In the mean time I can report that users with my identical-Terraform-commit build on top of Go 1.7.3 are not seeing any issues, so I'm more confident that this issue and your CL are on the path to success. |
|
Unfortunately that CL by itself adds a 3.5 second delay to Go processes using TLS, so it needs some work. |
|
Let's move discussion to #18203 |
|
I am seeing this in terraform 0.8.8, running on mac OS sierra 10.12.3, with go version go1.8 darwin/amd64 installed via homebrew. I don't see any difference when trying with terraform 0.8.8 downloaded from hasicorp directly as opposed to installed via homebrew. Could we have a regression with go 1.8, since this seemed to start with go 1.7.4? |
|
@dharmapunk82, we don't track comments on closed issues. If you have a reproducible problem, please file a new bug that can be tracked. I have questions about your statement, but I won't ask them here at risk of this turning into a discussion on a closed issue. Please file a new bug with details. |
On Darwin, user's trust preferences for root certificates were not honored. If the user had a root certificate loaded in their Keychain that was explicitly not trusted, a Go program would still verify a connection using that root certificate.
Thanks to Xy Ziemba for identifying and reporting this issue.
The text was updated successfully, but these errors were encountered: