crypto/x509: root_cgo_darwin omits certs with number of trust settings 0 erroneously when CGO_ENABLED=1

[dlamotte]

What version of Go are you using (go version)?

go version go1.10.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

Ran

package main

import (
	"crypto/x509"
	"fmt"
	"time"
)

func main() {
	start := time.Now()
	certs, err := x509.SystemCertPool()
	end := time.Now()
	if err != nil {
		panic(err)
	}
	fmt.Printf("found %d certs in %v\n", len(certs.Subjects()), end.Sub(start))
}

With patch from https://go-review.googlesource.com/changes/97801/revisions/1/patch?download (#24147) to inspect certs.

Looking for a very specific cert to be trusted on connections passing through Zscaler. This specific cert is coming up as "untrusted" with the patch when the OSX Keychain Access UI says it is in fact "Always Trust" in the cert info.

Example output from go run main.go of the above program:

roots=176, trustedRoots=175, untrustedRoots=16
...
crypto/x509: untrusted root CN=Zscaler Root CA,OU=Zscaler Inc.,O=Zscaler Inc.,L=San Jose,ST=California,C=US
...
found 175 certs in 538.654509ms

I can "fix" the issue by using Keychain Access to update the Trust from Always Trust to Never Trust and back to Always Trust (close Get Info window between changes to persist them). The actual difference appears by querying the system with the security command as follows:

% sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/Downloads/Zscaler\ Root\ CA.cer
% security dump-trust-settings -d > not-working.txt
... update trust in Keychain Access from `Always Trust` to `Never Trust` and back to `Always Trust` (close `Get Info` window between changes to persist them)
% security dump-trust-settings -d > working.txt
% diff -u not-working.txt working.txt
--- not-working.txt	2018-05-30 17:16:22.000000000 -0500
+++ working.txt	2018-05-30 17:17:18.000000000 -0500
@@ -59,4 +59,43 @@
    Trust Setting 0:
       Policy OID            : SSL
 Cert 15: Zscaler Root CA
-   Number of trust settings : 0
+   Number of trust settings : 10
+   Trust Setting 0:
+      Policy OID            : SSL
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 1:
+      Policy OID            : SSL
+      Allowed Error         : Host name mismatch
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 2:
+      Policy OID            : SMIME
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 3:
+      Policy OID            : SMIME
+      Allowed Error         : S/MIME Email address mismatch
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 4:
+      Policy OID            : EAP
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 5:
+      Policy OID            : IPSec
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 6:
+      Policy OID            : Code Signing
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 7:
+      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 8:
+      Policy OID            : Apple X509 Basic
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot
+   Trust Setting 9:
+      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
+      Result Type           : kSecTrustSettingsResultTrustRoot

Note: when importing the cert manually using the UI, the above issue is not seen as it appears the UI automatically adds trust settings.

What did you expect to see?

I expect that a cert added via security command would work without updating trust settings via the Keychain Access UI.

What did you see instead?

Commands (like helm) that attempt to make secured connections fail.

I started down this rabbit hole by noticing that CGO_ENABLED=0 compiled binaries didn't see this issue whereas CGO_ENABLED=1 compiled binaries did. I finally decided to dig into it and discovered this.

Possibly related to #24652

[ianlancetaylor]

CC @FiloSottile

[adamdecaf]

#24652 proposed a fix https://golang.org/cl/104735

[dlamotte]

Thanks @adamdecaf for the link to the fix. It does indeed look like it'll fix my issue. Closing this as a dupe of #24652