Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/x509: root_cgo_darwin and root_nocgo_darwin omit some system certs #24652

Closed
jdhenke opened this issue Apr 2, 2018 · 78 comments
Closed

crypto/x509: root_cgo_darwin and root_nocgo_darwin omit some system certs #24652

jdhenke opened this issue Apr 2, 2018 · 78 comments

Comments

@jdhenke
Copy link

@jdhenke jdhenke commented Apr 2, 2018

Please answer these questions before submitting your issue. Thanks!

What did you do?

$ cat main.go
package main

import (
	"crypto/x509"
	"fmt"
	"log"
)

func main() {
	certs, err := x509.SystemCertPool()
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Num System Certs: %d\n", len(certs.Subjects()))
}
$ CGO_ENABLED=0 go run main.go
Num System Certs: 188
$ CGO_ENABLED=1 go run main.go
Num System Certs: 168

What did you expect to see?

I expected to see the same number of certificates regardless of whether I used cgo.

What did you see instead?

The implementation using CGO resulted in fewer system certificates, which causes problems for our tooling that relies on one of those missing certificates to be in the SystemCertPool.

System details

go version go1.10.1 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jhenke/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jhenke"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_b/gz_w_nfj0_33f5y3s_0pg8xs080pym/T/go-build925272903=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.10.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.10.1
uname -v: Darwin Kernel Version 16.7.0: Mon Nov 13 21:56:25 PST 2017; root:xnu-3789.72.11~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G1114
lldb --version: lldb-900.0.64
  Swift-4.0
@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 3, 2018

Could you tell us more about the certificates not found in the cgo path? Are they set with specific trust policies? Could you paste a certificate that isn't found, but should be?

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 3, 2018

Do you know of a way to print/examine the trust policies of a certificate? After some experimentation, we have found that the certificates that aren't showing up seem to say "This certificate has custom trust settings" in the Keychain UI. However, expanding the "Trust" section doesn't reveal any specifics:

image

I'm not sure how the certificate was added/got to this state. It seems that if we manually update the state to "Always Trust" in the Keychain UI, then the certificate is returned. However, we'd like to understand this further, since most apps seem to trust the certificate even with these "Custom" trust settings but Go with CGo does not, which is causing issues for us.

*To clarify, I suspect that the issue has to do with the trust settings marked for the certificate rather than with the certificate itself

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 3, 2018

(CC: @FiloSottile)

@bcmills bcmills added this to the Go1.11 milestone Apr 3, 2018
@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 3, 2018

@jdhenke @nmiyake I've had to add the certificates into either the login keychain or /Library/Keychains/System.keychain. Doing this with the security tool looks like:

# One of these two should work
$ security add-trusted-cert -p ssl -k ~/Library/Keychains/login.keychain cert.pem
$ security add-trusted-cert -p ssl -k /Library/Keychains/System.keychain cert.pem

How are you adding the certs?

The Keychain UI is pretty much the easiest way to view trust, but it's not always clear. I've been working on better ways to parse the security tool output.

You can inspect the certs with the tool by running something like, which is what non-cgo Go does:

security find-certificate -c <name> -a ~/Library/Keychains/login.keychain /Library/Keychains/System.keychain /System/Library/Keychains/SystemRootCertificates.keychain 

I've talked about the inverse of this problem (explicit distrust of certificates) at #24084, which has the same confusion/problem.

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 3, 2018

Unfortunately, I'm not sure about the provenance of how the cert was added. However, I do suspect that how it was added/how it was upgraded from a previous add is at the root of the issue here.

I have a certificate in my System store called "MSCA-ROOT-01-CA". I'm not sure how it was added, but the screenshot earlier in the issue shows that it displays as "This certificate has custom trust settings", although the UI shows "Always Trust" for all the parts.

I added some debugging code to root_cgo_darwin.go (edits at the end of this post), and the resulting output for this cert is:

Processing cert: MSCA-ROOT-01-CA
getting domain 1 for trust settings
using domain 1 for trust settings
getting domain 2 for trust settings
trustSettings is not NULL
len of trustSettings: 0
done. untrusted: 0, trustAsRoot: 0, trustRoot: 0

Based my reading of the code, this certificate is in the "Admin" domain and has a non-NULL but empty trust setting. Because the trust setting is empty and it isn't a system cert, it decides not to trust it.

This logically makes sense to me, but I guess the resulting behavior isn't consistent with other applications (other applications seem willing to use this certificate for verification).

Modified FetchPEMRoots in crypto/x509/root_cgo_darwin.go:

int FetchPEMRoots(CFDataRef *pemRoots, CFDataRef *untrustedPemRoots) {
	if (useOldCode()) {
		return FetchPEMRoots_MountainLion(pemRoots);
	}

	// Get certificates from all domains, not just System, this lets
	// the user add CAs to their "login" keychain, and Admins to add
	// to the "System" keychain
	SecTrustSettingsDomain domains[] = { kSecTrustSettingsDomainSystem,
					     kSecTrustSettingsDomainAdmin,
					     kSecTrustSettingsDomainUser };

	int numDomains = sizeof(domains)/sizeof(SecTrustSettingsDomain);
	if (pemRoots == NULL) {
		return -1;
	}
	printf("numDomains: %d\n", numDomains);

	// kSecTrustSettingsResult is defined as CFSTR("kSecTrustSettingsResult"),
	// but the Go linker's internal linking mode can't handle CFSTR relocations.
	// Create our own dynamic string instead and release it below.
	CFStringRef policy = CFStringCreateWithCString(NULL, "kSecTrustSettingsResult", kCFStringEncodingUTF8);

	CFMutableDataRef combinedData = CFDataCreateMutable(kCFAllocatorDefault, 0);
	CFMutableDataRef combinedUntrustedData = CFDataCreateMutable(kCFAllocatorDefault, 0);
	for (int i = 0; i < numDomains; i++) {
		CFArrayRef certs = NULL;
		OSStatus err = SecTrustSettingsCopyCertificates(domains[i], &certs);
		if (err != noErr) {
			continue;
		}

		CFIndex numCerts = CFArrayGetCount(certs);
		for (int j = 0; j < numCerts; j++) {
			CFDataRef data = NULL;
			CFErrorRef errRef = NULL;
			CFArrayRef trustSettings = NULL;
			SecCertificateRef cert = (SecCertificateRef)CFArrayGetValueAtIndex(certs, j);
			if (cert == NULL) {
				continue;
			}
			// We only want trusted certs.
			int untrusted = 0;
			int trustAsRoot = 0;
			int trustRoot = 0;
			if (i == 0) {
				trustAsRoot = 1;
			} else {
			    CFStringRef commonNameRef = NULL;
				SecCertificateCopyCommonName(cert, &commonNameRef);
				printf("Processing cert: %s\n",  CFStringGetCStringPtr(commonNameRef, kCFStringEncodingMacRoman));

				// Certs found in the system domain are always trusted. If the user
				// configures "Never Trust" on such a cert, it will also be found in the
				// admin or user domain, causing it to be added to untrustedPemRoots. The
				// Go code will then clean this up.

				// Trust may be stored in any of the domains. According to Apple's
				// SecTrustServer.c, "user trust settings overrule admin trust settings",
				// so take the last trust settings array we find.
				// Skip the system domain since it is always trusted.
				for (int k = i; k < numDomains; k++) {
					printf("getting domain %d for trust settings\n", k);
					CFArrayRef domainTrustSettings = NULL;
					err = SecTrustSettingsCopyTrustSettings(cert, domains[k], &domainTrustSettings);
					if (err == errSecSuccess && domainTrustSettings != NULL) {
						if (trustSettings) {
							CFRelease(trustSettings);
						}
						printf("using domain %d for trust settings\n", k);
						trustSettings = domainTrustSettings;
					}
				}
				if (trustSettings == NULL) {
				    printf("this certificate must be verified to a known trusted certificate\n");
					// "this certificate must be verified to a known trusted certificate"; aka not a root.
					continue;
				}
				printf("trustSettings is not NULL\n");
				printf("len of trustSettings: %d\n", CFArrayGetCount(trustSettings));
				for (CFIndex k = 0; k < CFArrayGetCount(trustSettings); k++) {
					CFNumberRef cfNum;
					CFDictionaryRef tSetting = (CFDictionaryRef)CFArrayGetValueAtIndex(trustSettings, k);
					if (CFDictionaryGetValueIfPresent(tSetting, policy, (const void**)&cfNum)){
						SInt32 result = 0;
						CFNumberGetValue(cfNum, kCFNumberSInt32Type, &result);
						printf("resultNum: %d at index k=%d\n", result, k);
						// TODO: The rest of the dictionary specifies conditions for evaluation.
						if (result == kSecTrustSettingsResultDeny) {
							untrusted = 1;
						} else if (result == kSecTrustSettingsResultTrustAsRoot) {
							trustAsRoot = 1;
						} else if (result == kSecTrustSettingsResultTrustRoot) {
							trustRoot = 1;
						}
					}
				}
				printf("done. untrusted: %d, trustAsRoot: %d, trustRoot: %d\n", untrusted, trustAsRoot, trustRoot);
				CFRelease(trustSettings);
			}

			if (trustRoot) {
				// We only want to add Root CAs, so make sure Subject and Issuer Name match
				CFDataRef subjectName = SecCertificateCopyNormalizedSubjectContent(cert, &errRef);
				if (errRef != NULL) {
					CFRelease(errRef);
					continue;
				}
				CFDataRef issuerName = SecCertificateCopyNormalizedIssuerContent(cert, &errRef);
				if (errRef != NULL) {
					CFRelease(subjectName);
					CFRelease(errRef);
					continue;
				}
				Boolean equal = CFEqual(subjectName, issuerName);
				CFRelease(subjectName);
				CFRelease(issuerName);
				if (!equal) {
					continue;
				}
			}

			// Note: SecKeychainItemExport is deprecated as of 10.7 in favor of SecItemExport.
			// Once we support weak imports via cgo we should prefer that, and fall back to this
			// for older systems.
			err = SecKeychainItemExport(cert, kSecFormatX509Cert, kSecItemPemArmour, NULL, &data);
			if (err != noErr) {
				continue;
			}

			if (data != NULL) {
				if (!trustRoot && !trustAsRoot) {
					untrusted = 1;
				}
				CFMutableDataRef appendTo = untrusted ? combinedUntrustedData : combinedData;
				CFDataAppendBytes(appendTo, CFDataGetBytePtr(data), CFDataGetLength(data));
				CFRelease(data);
			}
		}
		CFRelease(certs);
	}
	CFRelease(policy);
	*pemRoots = combinedData;
	*untrustedPemRoots = combinedUntrustedData;
	return 0;
}
@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 3, 2018

@nmiyake Could you rebase that change off the latest commit on master? There's another change which mixes up the diff a bit.

I'm not sure if trusting a certificate without any policies from the user/admin domain would cause problems. If an attacker is modifying your trust policies they can already install a root CA.

There is a kSecTrustSettingsResultDeny policy I'd expect to be set (to explicitly distrust), but I don't know how widespread that's used. Keychain does use that.

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 3, 2018

Sure. Here's the modified code on master:

int FetchPEMRoots(CFDataRef *pemRoots, CFDataRef *untrustedPemRoots) {
	int i;

	if (useOldCode()) {
		return FetchPEMRoots_MountainLion(pemRoots);
	}

	// Get certificates from all domains, not just System, this lets
	// the user add CAs to their "login" keychain, and Admins to add
	// to the "System" keychain
	SecTrustSettingsDomain domains[] = { kSecTrustSettingsDomainSystem,
					     kSecTrustSettingsDomainAdmin,
					     kSecTrustSettingsDomainUser };

	int numDomains = sizeof(domains)/sizeof(SecTrustSettingsDomain);
	if (pemRoots == NULL) {
		return -1;
	}
	printf("numDomains: %d\n", numDomains);

	// kSecTrustSettingsResult is defined as CFSTR("kSecTrustSettingsResult"),
	// but the Go linker's internal linking mode can't handle CFSTR relocations.
	// Create our own dynamic string instead and release it below.
	CFStringRef policy = CFStringCreateWithCString(NULL, "kSecTrustSettingsResult", kCFStringEncodingUTF8);

	CFMutableDataRef combinedData = CFDataCreateMutable(kCFAllocatorDefault, 0);
	CFMutableDataRef combinedUntrustedData = CFDataCreateMutable(kCFAllocatorDefault, 0);
	for (i = 0; i < numDomains; i++) {
		int j;
		CFArrayRef certs = NULL;
		OSStatus err = SecTrustSettingsCopyCertificates(domains[i], &certs);
		if (err != noErr) {
			continue;
		}

		CFIndex numCerts = CFArrayGetCount(certs);
		for (j = 0; j < numCerts; j++) {
			CFDataRef data = NULL;
			CFErrorRef errRef = NULL;
			CFArrayRef trustSettings = NULL;
			SecCertificateRef cert = (SecCertificateRef)CFArrayGetValueAtIndex(certs, j);
			if (cert == NULL) {
				continue;
			}
			// We only want trusted certs.
			int untrusted = 0;
			int trustAsRoot = 0;
			int trustRoot = 0;
			if (i == 0) {
				trustAsRoot = 1;
			} else {
				CFStringRef commonNameRef = NULL;
				SecCertificateCopyCommonName(cert, &commonNameRef);
				printf("Processing cert: %s\n",  CFStringGetCStringPtr(commonNameRef, kCFStringEncodingMacRoman));

				int k;
				CFIndex m;

				// Certs found in the system domain are always trusted. If the user
				// configures "Never Trust" on such a cert, it will also be found in the
				// admin or user domain, causing it to be added to untrustedPemRoots. The
				// Go code will then clean this up.

				// Trust may be stored in any of the domains. According to Apple's
				// SecTrustServer.c, "user trust settings overrule admin trust settings",
				// so take the last trust settings array we find.
				// Skip the system domain since it is always trusted.
				for (k = i; k < numDomains; k++) {
					printf("getting domain %d for trust settings\n", k);
					CFArrayRef domainTrustSettings = NULL;
					err = SecTrustSettingsCopyTrustSettings(cert, domains[k], &domainTrustSettings);
					if (err == errSecSuccess && domainTrustSettings != NULL) {
						if (trustSettings) {
							CFRelease(trustSettings);
						}
						printf("using domain %d for trust settings\n", k);
						trustSettings = domainTrustSettings;
					}
				}
				if (trustSettings == NULL) {
					// "this certificate must be verified to a known trusted certificate"; aka not a root.
					printf("this certificate must be verified to a known trusted certificate\n");
					continue;
				}
				printf("trustSettings is not NULL\n");
				printf("len of trustSettings: %d\n", CFArrayGetCount(trustSettings));
				for (m = 0; m < CFArrayGetCount(trustSettings); m++) {
					CFNumberRef cfNum;
					CFDictionaryRef tSetting = (CFDictionaryRef)CFArrayGetValueAtIndex(trustSettings, m);
					if (CFDictionaryGetValueIfPresent(tSetting, policy, (const void**)&cfNum)){
						SInt32 result = 0;
						CFNumberGetValue(cfNum, kCFNumberSInt32Type, &result);
						printf("resultNum: %d at index k=%d\n", result, k);
						// TODO: The rest of the dictionary specifies conditions for evaluation.
						if (result == kSecTrustSettingsResultDeny) {
							untrusted = 1;
						} else if (result == kSecTrustSettingsResultTrustAsRoot) {
							trustAsRoot = 1;
						} else if (result == kSecTrustSettingsResultTrustRoot) {
							trustRoot = 1;
						}
					}
				}
				printf("done. untrusted: %d, trustAsRoot: %d, trustRoot: %d\n", untrusted, trustAsRoot, trustRoot);
				CFRelease(trustSettings);
			}

			if (trustRoot) {
				// We only want to add Root CAs, so make sure Subject and Issuer Name match
				CFDataRef subjectName = SecCertificateCopyNormalizedSubjectContent(cert, &errRef);
				if (errRef != NULL) {
					CFRelease(errRef);
					continue;
				}
				CFDataRef issuerName = SecCertificateCopyNormalizedIssuerContent(cert, &errRef);
				if (errRef != NULL) {
					CFRelease(subjectName);
					CFRelease(errRef);
					continue;
				}
				Boolean equal = CFEqual(subjectName, issuerName);
				CFRelease(subjectName);
				CFRelease(issuerName);
				if (!equal) {
					continue;
				}
			}

			// Note: SecKeychainItemExport is deprecated as of 10.7 in favor of SecItemExport.
			// Once we support weak imports via cgo we should prefer that, and fall back to this
			// for older systems.
			err = SecKeychainItemExport(cert, kSecFormatX509Cert, kSecItemPemArmour, NULL, &data);
			if (err != noErr) {
				continue;
			}

			if (data != NULL) {
				if (!trustRoot && !trustAsRoot) {
					untrusted = 1;
				}
				CFMutableDataRef appendTo = untrusted ? combinedUntrustedData : combinedData;
				CFDataAppendBytes(appendTo, CFDataGetBytePtr(data), CFDataGetLength(data));
				CFRelease(data);
			}
		}
		CFRelease(certs);
	}
	CFRelease(policy);
	*pemRoots = combinedData;
	*untrustedPemRoots = combinedUntrustedData;
	return 0;
}

Output was the same:

Processing cert: MSCA-ROOT-01-CA
getting domain 1 for trust settings
using domain 1 for trust settings
getting domain 2 for trust settings
trustSettings is not NULL
len of trustSettings: 0
done. untrusted: 0, trustAsRoot: 0, trustRoot: 0

From what I can tell, I have 3 certificates in my keychain that fit this criteria.

Adding this logic fixes the specific issue that we're seeing:

if (CFArrayGetCount(trustSettings) == 0) {
	trustAsRoot = 1;
}

(if this were to be done, I would presume it should probably be done for the case where trustSettings == NULL as well for consistency)

Interestingly, this doesn't fully resolve the diff for the number of certificates between CGO_ENABLED=0 and 1. My breakdown is:

  • master, CGO_ENABLED=1: 172 certificates
  • Trust certs with trustSettings ==0, CGO_ENABLED=1: 175 certificates
  • master, CGO_ENABLED=0: 196 certificates
@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 4, 2018

@nmiyake Cool. If you want to submit that if (CFArrayGetCount(trustSettings) == 0) { change we can get it reviewed.

There's probably something up with the trust policies on those remaining certificates. Can you run the following?

$ security trust-settings-export user-trust.plist
$ security trust-settings-export -d admin-trust.plist

This dumps plist (xml) files of your certificate trust. The best way to find a specific cert is by the sha1 hash. It's the <key>...</key> in the following snippet.

Can you find a certificate that's added into a keychain, but isn't showing up in Go? I'm curious what <key>trustSettings</key> is saying.

The values there are mapped to SecTrustSettingsResult.

                <key>0563B8630D62D75ABBC8AB1E4BDFB5A899B24D43</key>
                <dict>
                        <key>issuerName</key>
                        <data>
                        MGUxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMx
                        GTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20xJDAiBgNVBAMTG0Rp
                        Z2lDZXJ0IEFzc3VyZWQgSUQgUm9vdCBDQQ==
                        </data>
                        <key>modDate</key>
                        <date>2018-02-24T19:37:04Z</date>
                        <key>serialNumber</key>
                        <data>
                        DOfg5RfYRv6P5WD8G/AwOQ==
                        </data>
                        <key>trustSettings</key>
                        <array>
                                <dict>
                                        <key>kSecTrustSettingsResult</key>
                                        <integer>4</integer>
                                </dict>
                        </array>
                </dict>

I'm building a quick tool to help debug these files. You can run it over the exported plist files and get something that's a bit easier to parse. https://github.com/adamdecaf/plist-parser

hash=0563B863 issuer="CN=DigiCert Assured ID Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US" trustSettings=map[string]string{"kSecTrustSettingsResult":"4"}
@nmiyake
Copy link

@nmiyake nmiyake commented Apr 4, 2018

OK, performed more digging and diagnostics.

Here's the overview of my state with the current Go:

No CGo certs: 176
CGo certs: 161

0 CGo only, 15 non-CGo only, 161 common

After running with my modification proposed above, I get:

No CGo certs: 176
CGo certs: 164

1 CGo only, 13 non-CGo only, 163 common

I ran your parsing tool on my output, and indeed for some reason there are 3 certificates that explicitly have an empty trust settings set (only showing one here):

hash=00C337EA issuer="CN=MSCA-ROOT-01-CA" trustSettings=map[string]string{}

I'm not sure how this entry was created for me, but that's clearly the issue. Two of these certificates are valid and are added, and thus increment the "common" count by 2. One of these certificates is expired. This one appears only in the CGo code, which accounts for the "1 CGo only".

Here are some of the certificates that show up only for non-CGo (out of the 13):

        Apple Worldwide Developer Relations Certification Authority
        Developer ID Certification Authority
        DigiCert Assured ID CA-1
        DigiCert SHA2 Assured ID CA
        DigiCert SHA2 High Assurance Server CA

2 of these have an entry in my user-trust.plist with trustSettings=map[string]string{"kSecTrustSettingsResult":"-2147409654"}:

hash=20744DE6 issuer="CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US" trustSettings=map[string]string{"kSecTrustSettingsResult":"-2147409654"}

None of the other entries show up in my plist. In the keychain, these show up as "no value specified" for trust:

image

The one curious thing is the extra certificate that shows up for CGo only after the local modification. That certificate is an old CA certificate I have that is expired:

image

It's showing up because before the modification it was in my admin-trust.plist with map[string]string{} as the trustSettings, so the change now includes it. I'm guessing the non-CGo code automatically prunes it based on expiration.

I don't think this should be an issue since even if the cert is added as a root cert, any code that does validation should properly check the expiration status.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 4, 2018

Change https://golang.org/cl/104735 mentions this issue: crypto/x509: add certs with empty trust settings for cgo_darwin

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 4, 2018

@nmiyake Cool on that CL. You'll probably want to assign @FiloSottile as a reviewer.

As far as the certificates with "kSecTrustSettingsResult":"-2147409654". I think this is why they're not marked as trusted. -2147409654 is not a valid value. The relevant Go code only checks a couple of the values (see SecTrustSettingsResult).

if (CFDictionaryGetValueIfPresent(tSetting, policy, (const void**)&cfNum)){
	SInt32 result = 0;
	CFNumberGetValue(cfNum, kCFNumberSInt32Type, &result);
	printf("resultNum: %d at index k=%d\n", result, k);
	// TODO: The rest of the dictionary specifies conditions for evaluation.
	if (result == kSecTrustSettingsResultDeny) {
		untrusted = 1;
	} else if (result == kSecTrustSettingsResultTrustAsRoot) {
		trustAsRoot = 1;
	} else if (result == kSecTrustSettingsResultTrustRoot) {
		trustRoot = 1;
	}
}

Are your trust settings managed by an enterprise or tool by chance? It looks like some tool generated partially invalid policy settings.

I've had to work around this and seen it before. Here's an example plist I've seen in the wild. (Note: it's the same -2147409654 value)

				<dict>
					<key>kSecTrustSettingsAllowedError</key>
					<integer>-2147409654</integer>
					<key>kSecTrustSettingsPolicy</key>
					<data>
					KoZIhvdjZAEC
					</data>
					<key>kSecTrustSettingsPolicyName</key>
					<string>basicX509</string>
					<key>kSecTrustSettingsResult</key>
					<integer>3</integer>
				</dict>

On those 13-2 certificates I wonder again if the plist/trust policies were generated properly. Just search for DigiCert SHA2 High Assurance Server CA gives a different cert fingerprint than what your policy has. See: https://crt.sh/?id=2900424 (SHA1 prefix A031C467) https://crt.sh/?id=2900424.

Can you find any of those certificates on crt.sh? https://crt.sh/?a=1

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 4, 2018

Yes, this behavior is on a machine that's managed by a company and uses tools to do so -- something along the way there writing an invalid policy entry is definitely a possibility.

I guess the difference in observed behavior is that most macOS applications (or Apple's cert API itself?) are more lenient on their verification here? Even though strict validation may technically be correct, if it results in an observable difference in behavior between native macOS apps and Go apps with CGo enabled that stills seems like it could be an issue (and I don't really have any good way of knowing how common or uncommon this scenario may be more broadly).

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 4, 2018

I guess the difference in observed behavior is that most macOS applications (or Apple's cert API itself?) are more lenient on their verification here?

I don't think we've determined that quite yet. Are any of those 13-2 certs not CA's?

I'm reading through the code paths and noticing only the cgo path checks the certificate is a Root CA (by checking Issuer == Subject).

Edit: Yep. I verified a non-ca certificate would show up in a non-cgo call to x509.SystemCertPool().

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 4, 2018

Sorry you're right -- the only place where I know that to be true is for the trustSettings being an empty map (one of those certificates was a CA that was verifying all connections, but wasn't picked up by CGo before the proposed change).

Yes, I think that theory makes sense -- as shows in the UI for the certificates (the last screenshot), all of those certificates seem to be intermediate CAs.

If that's the case, I should probably update my PR to set trustRoot=1 (rather than trustAsRoot=1) so that the post-filtering step still happens -- I think that should eliminate that case of the one extra certificate being present only in CGo after the change (will verify locally now)

Edit: it didn't fix the issue I was seeing because that cert was a root cert (but just expired). However, functionally I think the change is correct, so updated PR.

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 5, 2018

Yes, I think that theory makes sense -- as shows in the UI for the certificates (the last screenshot), all of those certificates seem to be intermediate CAs.

Good. I think we've tracked down the issues. This shows a difference in cgo and non-cgo paths in that cgo only captures the Subject == Issuer certificates.

Are you seeing any connection errors from being unable to build a chain? (From these 11 intermediates not being captured?)

Looking at the Unix implementation we aren't checking Subject == Issuer or .IsCA. Perhaps we need to decide if SystemCertPool() should return these intermediate certificates?

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 5, 2018

@nmiyake Looking through the old CL's I found https://go-review.googlesource.com/c/go/+/64851 (from #16532).

@nmiyake
Copy link

@nmiyake nmiyake commented Apr 5, 2018

Ah interesting -- so that approach does use trustAsRoot for the certificates with empty entries (rather than `trustRoot), so it does add intermediate certs, which this one doesn't after my last update.

The only added certificate that I regularly use with Go is a custom root certificate, so I would not be impacted by this either way. I don't have a strong opinion on whether or not intermediates should be considered as root -- however, I do think it would be nice for this behavior to be consistent (at a minimum between CGo and non-CGo on Darwin, and in an ideal world across all platforms).

If there's a particular test you want me to run around validation with intermediate certs, if you outline it I can give it a stab and update with results.

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented Apr 5, 2018

cc @FiloSottile Do you have thoughts on SystemCertPool() returning intermediate certificates?

@nmiyake
Copy link

@nmiyake nmiyake commented May 7, 2018

What's the process to get more eyes on this? Whether it's via my PR or another mechanism, it would be nice to determine a path forward as there are many people on our team who seem to be encountering this issue

@dlamotte
Copy link

@dlamotte dlamotte commented May 30, 2018

I might have opened a dupe here #25649. Was having a hard time following this issue thread.

@adamdecaf
Copy link
Contributor

@adamdecaf adamdecaf commented May 31, 2018

@dlamotte There were a few issues we had to track down in this thread. #25649 does look like a duplicate as you also expected the cert to be added to the x509.CertPool.

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jun 25, 2018

There are multiple issues with our macOS root discovery.

The cgo path is unaware of defaults, documented at https://developer.apple.com/documentation/security/1400261-sectrustsettingscopytrustsetting, so it will omit the following certificate.

Cert 1: mkcert development CA
   Number of trust settings : 0

CL 104735 is an incomplete fix, because if trustSettings are present but don't have a kSecTrustSettingsResult value, it defaults to trustRoot. So it will omit the following certificate.

Cert 1: mkcert development CA
   Number of trust settings : 1
   Trust Setting 0:
      Policy OID            : SSL

The nocgo path, on the other hand, asks security verify-cert to use the default verification policy, basic, so it will omit the following certificate.

Cert 1: mkcert development CA
   Number of trust settings : 1
   Trust Setting 0:
      Policy OID            : SSL
      Result Type           : kSecTrustSettingsResultTrustRoot

Finally, the cgo path is checking if any policy (ssl or any other explicitly set) has a kSecTrustSettingsResult value (ignoring the defaults, see above), with the last one in the array winning, omitting the following certificate (!!).

Cert 1: mkcert development CA
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : SSL
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : Code Signing
      Result Type           : kSecTrustSettingsResultDeny

And I didn't even get into allowed errors.

It's fairly late in the freeze, but I'm inclined to fix these, and maybe even backport them, because ignoring the policy types can lead to inclusion of roots that are not supposed to be trusted for TLS, and although crypto/x509 is not TLS-specific, it is meant to serve the WebPKI. @agl agree?

@gopherbot please open the backport tracking issues.

@FiloSottile FiloSottile changed the title crypto/x509: root_cgo_darwin omits some system certs crypto/x509: root_cgo_darwin and root_nocgo_darwin omit some system certs Jun 25, 2018
@cvigo
Copy link

@cvigo cvigo commented Dec 3, 2018

If the test fails, please copy the whole output in a comment here. Do check that there's nothing you consider sensitive in it (it only lists names of certificates in your keychain, which might however include names of S/MIME senders) and if you'd prefer to report privately email filippo at golang.org.

Another test fail

Suprisingly I two similar private Root CAs that return different results (see Global Root CA vs. Global Root CA Work)

Test Results
```
crypto/x509: verify-cert approved CN=Apple Worldwide Developer Relations Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
crypto/x509: verify-cert rejected CN=com.apple.systemdefault,O=System Identity: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=com.apple.kerberos.kdc,O=System Identity: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert approved CN=AutoFirma ROOT
crypto/x509: verify-cert approved CN=127.0.0.1
crypto/x509: verify-cert approved CN=BBVA Autoridad de Certificacion Digital,O=Banco Bilbao Vizcaya Argentaria
crypto/x509: verify-cert approved CN=BBVA CA Servidores,O=BBVA
crypto/x509: verify-cert approved CN=BBVA CA Raiz,O=BBVA
crypto/x509: verify-cert approved CN=BBVA Servidores Autoridad de Certificacion Digital,OU=Para Uso Interno BBVA,O=Banco Bilbao Vizcaya Argentaria
crypto/x509: verify-cert approved CN=Global Root CA,OU=Security Architecture Cryptography,O=BBVA,C=ES
crypto/x509: verify-cert approved CN=Global Root CA Work,OU=Security Architecture Cryptography,O=BBVA,C=ES
crypto/x509: verify-cert approved CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
crypto/x509: verify-cert approved CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
crypto/x509: verify-cert approved CN=wifiaccess.grupobbva.com,OU=Comunicaciones,O=BBVA,L=Bilbao,C=ES
crypto/x509: verify-cert rejected CN=vpnaas_live.es.nextgen.igrupobbva,OU=Architecture Security,O=BBVA,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=vpnaas_live.es.nextgen.igrupobbva,OU=Security Architecture,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=vpnaas.es.nextgen.igrupobbva,OU=Security Architecture,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=isepsncorpeditc2.igrupobbva,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=link.live.es.platform.bbva.com,OU=SECURITY,O=BBVA,L=MADRID,ST=MADRID,C=ES
crypto/x509: verify-cert rejected CN=armadillo.smlb.secaas-live-es.ext.es.iaas.igrupobbva,OU=Dyd,O=BBVA,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert approved CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
crypto/x509: verify-cert approved CN=Apple Worldwide Developer Relations Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
crypto/x509: verify-cert approved CN=Developer Authentication Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
crypto/x509: verify-cert rejected CN=Adobe Content Certificate 10-6,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Intermediate CA 10-4,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Intermediate CA 10-3,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Content Certificate 10-5,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Xcode Server Builder (05/11/2018\, 09:57:44): "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-87654321K,CN=NAME REMOVED FOR PRIVACY - 87654321K,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-12345678K,CN=NAME REMOVED FOR PRIVACY - 12345678K,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
crypto/x509: verify-cert approved CN=Ether R3 ES Issuing CA Work,OU=Security Architecture Cryptography,O=BBVA,C=ES
crypto/x509: verify-cert approved CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
crypto/x509: verify-cert approved CN=wifiaccess.grupobbva.com,OU=Comunicaciones,O=BBVA,L=Bilbao,C=ES
crypto/x509: verify-cert rejected CN=vpnaas_live.es.nextgen.igrupobbva,OU=Architecture Security,O=BBVA,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=vpnaas_live.es.nextgen.igrupobbva,OU=Security Architecture,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=vpnaas.es.nextgen.igrupobbva,OU=Security Architecture,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=isepsncorpeditc2.igrupobbva,O=BBVA,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=link.live.es.platform.bbva.com,OU=SECURITY,O=BBVA,L=MADRID,ST=MADRID,C=ES
crypto/x509: verify-cert approved CN=Apple Worldwide Developer Relations Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
crypto/x509: verify-cert rejected CN=armadillo.smlb.secaas-live-es.ext.es.iaas.igrupobbva,OU=Dyd,O=BBVA,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert approved CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
crypto/x509: verify-cert approved CN=Developer Authentication Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
crypto/x509: verify-cert rejected CN=Adobe Content Certificate 10-6,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Intermediate CA 10-4,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Content Certificate 10-5,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Adobe Intermediate CA 10-3,OU=Cloud Technology,O=Adobe Systems,L=San Jose,ST=California,C=US: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Xcode Server Builder (05/11/2018\, 09:57:44): "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-87654321K,CN=NAME REMOVED FOR PRIVACY - 87654321K,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-12345678K,CN=NAME REMOVED FOR PRIVACY - 12345678K,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=Ether R3 ES Issuing CA Work,OU=Security Architecture Cryptography,O=BBVA,C=ES
crypto/x509: ran security verify-cert 51 times
    cgo sys roots: 366.462356ms
non-cgo sys roots: 671.314532ms
signed certificate only present in non-cgo pool (acceptable): CN=Apple Worldwide Developer Relations Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
signed certificate only present in non-cgo pool (acceptable): CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US
signed certificate only present in non-cgo pool (acceptable): CN=wifiaccess.grupobbva.com,OU=Comunicaciones,O=BBVA,L=Bilbao,C=ES
signed certificate only present in non-cgo pool (acceptable): CN=vpnaas_live.es.nextgen.igrupobbva,OU=Security Architecture,O=BBVA,L=Madrid,ST=Madrid,C=ES
signed certificate only present in non-cgo pool (acceptable): CN=isepsncorpeditc2.igrupobbva,O=BBVA,L=Madrid,ST=Madrid,C=ES
signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
signed certificate only present in non-cgo pool (acceptable): CN=Developer Authentication Certification Authority,OU=Apple Worldwide Developer Relations,O=Apple Inc.,C=US
signed certificate only present in non-cgo pool (acceptable): CN=Ether R3 ES Issuing CA Work,OU=Security Architecture Cryptography,O=BBVA,C=ES
certificate only present in cgo pool: SERIALNUMBER=IDCES-87654321K,CN=NAME REMOVED FOR PRIVACY - 87654321K,C=ES
certificate only present in cgo pool: CN=Xcode Server Builder (05/11/2018\, 09:57:44)
certificate only present in cgo pool: CN=armadillo.smlb.secaas-live-es.ext.es.iaas.igrupobbva,OU=Dyd,O=BBVA,L=Madrid,ST=Madrid,C=ES
certificate only present in cgo pool: SERIALNUMBER=IDCES-12345678K,CN=NAME REMOVED FOR PRIVACY - 12345678K,C=ES
Number of trusted certs = 11
Cert 0: wifiaccess.grupobbva.com
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 1: vpnaas.es.nextgen.igrupobbva
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 2: isepsncorpeditc2.igrupobbva
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 3: link.live.es.platform.bbva.com
   Number of trust settings : 3
   Trust Setting 0:
      Policy OID            : SSL
      Policy String         : 185.24.6.15
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Policy String         : 185.24.6.15
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : Apple X509 Basic
      Policy String         : 185.24.6.15
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 4: armadillo.smlb.secaas-live-es.ext.es.iaas.igrupobbva
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : SSL
      Policy String         : atenea.live.global.ether.igrupobbva
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Policy String         : atenea.live.global.ether.igrupobbva
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 5: Xcode Server Builder (05/11/2018, 09:57:44)
   Number of trust settings : 9
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 6: NAME REMOVED FOR PRIVACY - 87654321K
   Number of trust settings : 9
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 7: NAME REMOVED FOR PRIVACY - 12345678K
   Number of trust settings : 9
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 8: BBVA Autoridad de Certificacion Digital
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 9: BBVA CA Raiz
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 10: Global Root CA Work
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Number of trusted certs = 5
Cert 0: AutoFirma ROOT
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 1: 127.0.0.1
   Number of trust settings : 9
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 2: BBVA CA Servidores
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 3: BBVA Servidores Autoridad de Certificacion Digital
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 2:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 3:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 4:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 5:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 6:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 7:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 8:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 4: Global Root CA
   Number of trust settings : 0

!!! The test failed!

Please report the whole output at #24652 wrapping it in a code block
Thank you!
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: AutoFirma ROOT returned 1
crypto/x509: 127.0.0.1 returned 2
crypto/x509: BBVA CA Servidores returned 2
crypto/x509: BBVA Servidores Autoridad de Certificacion Digital returned 2
crypto/x509: Global Root CA returned 1
crypto/x509: wifiaccess.grupobbva.com returned 4
crypto/x509: vpnaas.es.nextgen.igrupobbva returned 1
crypto/x509: isepsncorpeditc2.igrupobbva returned 4
crypto/x509: link.live.es.platform.bbva.com returned 2
crypto/x509: armadillo.smlb.secaas-live-es.ext.es.iaas.igrupobbva returned 1
crypto/x509: Xcode Server Builder (05/11/2018, 09:57:44) returned 2
crypto/x509: NAME REMOVED FOR PRIVACY - 87654321K returned 2
crypto/x509: NAME REMOVED FOR PRIVACY - 12345678K returned 2
crypto/x509: BBVA Autoridad de Certificacion Digital returned 1
crypto/x509: BBVA CA Raiz returned 1
crypto/x509: Global Root CA Work returned 1

</details>
@cvigo
Copy link

@cvigo cvigo commented Dec 4, 2018

Unfortunately this change is hard to test and interacts with a very complex API, so I did not feel safe shipping it in 1.11 at the last minute. We will be targeting the next minor release after more testing.

As a temporary workaround, in some cases you can make the trust settings explicit by opening Keychain Access, finding the releavnt certificate, and marking it as Always Trust. It might even already show as such in the UI, because empty trust settings are the same as Always Trust, but Go doesn't understand that. Just toggling it in the UI back and forth should make the settings explicit.

It did not work for me unfortunately, although the trust settings are now printed by @FiloSottile test:

Test results
Cert 4: Global Root CA
   Number of trust settings : 10
   Trust Setting 0:
      Policy OID            : SSL
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : Code Signing
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Policy OID            : Unknown OID length 9, value { 2A 86 48 86 F7 63 64 01 14  }
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 4:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 5:
      Policy OID            : SMIME
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 6:
      Policy OID            : SMIME
      Allowed Error         : S/MIME Email address mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 7:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 8:
      Policy OID            : IPSec
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 9:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
gopherbot pushed a commit that referenced this issue Dec 5, 2018
The cgo path was not taking policies into account, using the last
security setting in the array whatever it was. Also, it was not aware of
the defaults for empty security settings, and for security settings
without a result type. Finally, certificates restricted to a hostname
were considered roots.

The API docs for this code are partial and not very clear, so this is a
best effort, really.

Updates #24652

Change-Id: I8fa2fe4706f44f3d963b32e0615d149e997b537d
Reviewed-on: https://go-review.googlesource.com/c/128056
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@golang.org>
gopherbot pushed a commit that referenced this issue Dec 5, 2018
Certificates without any trust settings might still be in the keychain
(for example if they used to have some, or if they are intermediates for
offline verification), but they are not to be trusted. The only ones we
can trust unconditionally are the ones in the system roots store.

Moreover, the verify-cert invocation was not specifying the ssl policy,
defaulting instead to the basic one. We have no way of communicating
different usages in a CertPool, so stick to the WebPKI use-case as the
primary one for crypto/x509.

Updates #24652

Change-Id: Ife8b3d2f4026daa1223aa81fac44aeeb4f96528a
Reviewed-on: https://go-review.googlesource.com/c/128116
Reviewed-by: Adam Langley <agl@google.com>
Reviewed-by: Adam Langley <agl@golang.org>
@gopherbot gopherbot closed this in 9536c5f Dec 5, 2018
@cvigo
Copy link

@cvigo cvigo commented Dec 14, 2018

Same error with go 1.11.3 😠

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 14, 2018

@cvigo, the 1.11 backport of this issue is #26039, which is still open.

@cvigo
Copy link

@cvigo cvigo commented Dec 16, 2018

@cvigo, the 1.11 backport of this issue is #26039, which is still open.

true... as it is an old issue I missed the 1.12 milestone link

I will keep surviving with Dockerized go mod

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Dec 17, 2018

This issue should now be fixed in master, but more testing would be appreciated before we backport it, in particular by anyone who had the self-contained test fail.

You can use our new golang.org/dl/gotip tool to easily test the development branch:

go get golang.org/dl/gotip
gotip download
GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509

You can also use gotip build or gotip get in place of go build/go get to test if the patches resolve any issues you were experiencing. Please report back! Thank you.

/cc @cvigo @dichque @wdec @adamrothman @calmh @dadrian @vdemario

@cvigo
Copy link

@cvigo cvigo commented Dec 17, 2018

Mmmmm it does not look good...

 ❯ gotip version                                                                                                                                                                               [11:06:34]
go version devel +bc175e5 Mon Dec 17 03:49:37 2018 +0000 darwin/amd64

CA is Global Root CA

image

GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509
 ❯ GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
--- SKIP: TestSystemRoots (0.00s)
    root_darwin_test.go:21: skipping on darwin/amd64 until golang.org/issue/24652 has been resolved.
PASS
ok  	crypto/x509	0.012s
GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
 ❯ GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: 16 certs have a trust policy
crypto/x509: verify-cert approved CN=mycompanyname CA Raiz,O=mycompanyname
crypto/x509: verify-cert approved CN=mycompanyname Autoridad de Certificacion Digital,O=My Company Name
crypto/x509: verify-cert approved CN=127.0.0.1
crypto/x509: verify-cert approved CN=AutoFirma ROOT
crypto/x509: verify-cert approved CN=mycompanyname CA Servidores,O=mycompanyname
crypto/x509: verify-cert approved CN=mycompanyname Servidores Autoridad de Certificacion Digital,OU=Para Uso Interno mycompanyname,O=My Company Name
crypto/x509: verify-cert approved CN=Global Root CA Work,OU=Security Architecture Cryptography,O=mycompanyname,C=ES
crypto/x509: verify-cert approved CN=Global Root CA,OU=Security Architecture Cryptography,O=mycompanyname,C=ES
crypto/x509: verify-cert approved CN=wifiaccess.mycompanyname.com,OU=Comunicaciones,O=mycompanyname,L=Bilbao,C=ES
crypto/x509: verify-cert approved CN=isepsncorpeditc2.imycompanyname,O=mycompanyname,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=link.live.es.platform.mycompanyname.com,OU=SECURITY,O=mycompanyname,L=MADRID,ST=MADRID,C=ES
crypto/x509: verify-cert rejected CN=Xcode Server Builder (05/11/2018\, 09:57:44): "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=vpnaas.es.datacenter.imycompanyname,OU=Security Architecture,O=mycompanyname,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert rejected CN=armadillo.smlb.secaas-live-es.ext.es.iaas.imycompanyname,OU=Dyd,O=mycompanyname,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert approved CN=Global Root CA Work,OU=Security Architecture Cryptography,O=mycompanyname,C=ES
crypto/x509: verify-cert approved CN=wifiaccess.mycompanyname.com,OU=Comunicaciones,O=mycompanyname,L=Bilbao,C=ES
crypto/x509: verify-cert approved CN=vpnaas.es.datacenter.imycompanyname,OU=Security Architecture,O=mycompanyname,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=isepsncorpeditc2.imycompanyname,O=mycompanyname,L=Madrid,ST=Madrid,C=ES
crypto/x509: verify-cert approved CN=link.live.es.platform.mycompanyname.com,OU=SECURITY,O=mycompanyname,L=MADRID,ST=MADRID,C=ES
crypto/x509: verify-cert rejected CN=armadillo.smlb.secaas-live-es.ext.es.iaas.imycompanyname,OU=Dyd,O=mycompanyname,L=Madrid,ST=Madrid,C=ES: "Cert Verify Result: CSSMERR_TP_NOT_TRUSTED"
crypto/x509: verify-cert rejected CN=Xcode Server Builder (05/11/2018\, 09:57:44): "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN1 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN2 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN1 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=Global Root CA Work,OU=Security Architecture Cryptography,O=mycompanyname,C=ES
crypto/x509: verify-cert rejected SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN2 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: ran security verify-cert 26 times
--- PASS: TestSystemRoots (0.94s)
    root_darwin_test.go:34:     cgo sys roots: 197.258052ms
    root_darwin_test.go:35: non-cgo sys roots: 698.610683ms
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=wifiaccess.mycompanyname.com,OU=Comunicaciones,O=mycompanyname,L=Bilbao,C=ES
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=isepsncorpeditc2.imycompanyname,O=mycompanyname,L=Madrid,ST=Madrid,C=ES
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=link.live.es.platform.mycompanyname.com,OU=SECURITY,O=mycompanyname,L=MADRID,ST=MADRID,C=ES
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
    root_darwin_test.go:96: off-EKU certificate only present in cgo pool (acceptable): SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN1 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES
    root_darwin_test.go:96: off-EKU certificate only present in cgo pool (acceptable): SERIALNUMBER=IDCES-XXXXXXXXX,CN=CN2 REMOVED FOR PRIVACY - XXXXXXXXX,C=ES
    root_darwin_test.go:96: off-EKU certificate only present in cgo pool (acceptable): CN=Xcode Server Builder (05/11/2018\, 09:57:44)
PASS
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: AutoFirma ROOT returned 1
crypto/x509: 127.0.0.1 returned 2
crypto/x509: mycompanyname CA Servidores returned 2
crypto/x509: mycompanyname Servidores Autoridad de Certificacion Digital returned 2
crypto/x509: Global Root CA returned 1
crypto/x509: wifiaccess.mycompanyname.com returned 4
crypto/x509: vpnaas.es.datacenter.imycompanyname returned 1
crypto/x509: isepsncorpeditc2.imycompanyname returned 4
crypto/x509: link.live.es.platform.mycompanyname.com returned 4
crypto/x509: armadillo.smlb.secaas-live-es.ext.es.iaas.imycompanyname returned 4
crypto/x509: Xcode Server Builder (05/11/2018, 09:57:44) returned 2
crypto/x509: CN2 REMOVED FOR PRIVACY - XXXXXXXXX returned 2
crypto/x509: CN1 REMOVED FOR PRIVACY - XXXXXXXXX returned 2
crypto/x509: Global Root CA Work returned 1
crypto/x509: mycompanyname Autoridad de Certificacion Digital returned 1
crypto/x509: mycompanyname CA Raiz returned 1
ok  	crypto/x509	(cached)
go get -v -u scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf (FAIL)
 ❯ GODEBUG=1 go get -v -u scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf
Fetching https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1
https fetch failed: Get https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1: x509: certificate signed by unknown authority
Fetching https://scm.es.datacenter.imycompanyname/connectors?go-get=1
https fetch failed: Get https://scm.es.datacenter.imycompanyname/connectors?go-get=1: x509: certificate signed by unknown authority
Fetching https://scm.es.datacenter.imycompanyname?go-get=1
https fetch failed: Get https://scm.es.datacenter.imycompanyname?go-get=1: x509: certificate signed by unknown authority
go get scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf: unrecognized import path "scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf" (https fetch: Get https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1: x509: certificate signed by unknown authority)
gotip get -v -u scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf (also FAIL)
 ❯ GODEBUG=x509roots=1 gotip get -v -u scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf
Fetching https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: AutoFirma ROOT returned 1
crypto/x509: 127.0.0.1 returned 2
crypto/x509: mycompanyname CA Servidores returned 2
crypto/x509: mycompanyname Servidores Autoridad de Certificacion Digital returned 2
crypto/x509: Global Root CA returned 1
crypto/x509: wifiaccess.mycompanyname.com returned 4
crypto/x509: vpnaas.es.datacenter.imycompanyname returned 1
crypto/x509: isepsncorpeditc2.imycompanyname returned 4
crypto/x509: link.live.es.platform.mycompanyname.com returned 4
crypto/x509: armadillo.smlb.secaas-live-es.ext.es.iaas.imycompanyname returned 4
crypto/x509: Xcode Server Builder (05/11/2018, 09:57:44) returned 2
crypto/x509: CN2 REMOVED FOR PRIVACY - XXXXXXXXX returned 2
crypto/x509: CN1 REMOVED FOR PRIVACY - XXXXXXXXX returned 2
crypto/x509: Global Root CA Work returned 1
crypto/x509: mycompanyname Autoridad de Certificacion Digital returned 1
crypto/x509: mycompanyname CA Raiz returned 1
https fetch failed: Get https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1: x509: certificate signed by unknown authority
Fetching https://scm.es.datacenter.imycompanyname/connectors?go-get=1
https fetch failed: Get https://scm.es.datacenter.imycompanyname/connectors?go-get=1: x509: certificate signed by unknown authority
Fetching https://scm.es.datacenter.imycompanyname?go-get=1
https fetch failed: Get https://scm.es.datacenter.imycompanyname?go-get=1: x509: certificate signed by unknown authority
go get scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf: unrecognized import path "scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf" (https fetch: Get https://scm.es.datacenter.imycompanyname/connectors/titan_sql_protobuf?go-get=1: x509: certificate signed by unknown authority)
@dadrian
Copy link

@dadrian dadrian commented Dec 17, 2018

@FiloSottile tests passed for me.

$ GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509

=== RUN   TestSystemRoots
crypto/x509: 3 certs have a trust policy
crypto/x509: verify-cert approved CN=X Proto CA,OU=0x21,O=University of Michigan,L=Ann Arbor,ST=Michigan,C=US
crypto/x509: verify-cert approved CN=radius.umnet.umich.edu,OU=Information Technology Services,O=University of Michigan,POSTALCODE=48105-3640,L=Ann Arbor,ST=MI,C=US
crypto/x509: verify-cert rejected CN=dlv-cert: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=radius.umnet.umich.edu,OU=Information Technology Services,O=University of Michigan,POSTALCODE=48105-3640,L=Ann Arbor,ST=MI,C=US
crypto/x509: ran security verify-cert 4 times
--- PASS: TestSystemRoots (0.63s)
    root_darwin_test.go:34:     cgo sys roots: 315.417942ms
    root_darwin_test.go:35: non-cgo sys roots: 245.18544ms
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=radius.umnet.umich.edu,OU=Information Technology Services,O=University of Michigan,POSTALCODE=48105-3640,L=Ann Arbor,ST=MI,C=US
    root_darwin_test.go:76: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
    root_darwin_test.go:96: off-EKU certificate only present in cgo pool (acceptable): CN=dlv-cert
PASS
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: dlv-cert returned 1
crypto/x509: X Proto CA returned 1
crypto/x509: radius.umnet.umich.edu returned 4
ok  	crypto/x509	0.648s
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Dec 17, 2018

@cvigo Hmm, that's weird because the tests pass, suggesting cgo and non-cgo agree, and the debug output suggests "Global Root CA" was added to the pool. Is the chain well-formed, with all the necessary intermediates?

@cvigo
Copy link

@cvigo cvigo commented Dec 17, 2018

OMG... Just found this:

When I reported the issue, the Cert chain was:
image
image
image

Global Root CA was set by me as "Always Trust", the intermediate Global Issuing CA Infrastructure and the final Cert were, therefore, implicitly trusted and reported as valid by Google Chrome and curl, but rejected by go get and go mod

Now I changed the intermediate Global Issuing CA Infrastructure from "Use Syetem Defaults" to explicit "Always Trust" and everything works, including go 1.11.3

image
image

I thought the trustworthiness would be inherited from the root certificate down the tree and, actually, that is the case for browsers including curl, but Go is behaving differently.

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Dec 18, 2018

It is indeed supposed to be inherited, so that's not intended behavior, if the server sends a full chain including the intermediate. There might also be something about the root CA that makes it disqualified from forming chains, if you can share it at filippo@golang.org I can look into that.

@cvigo
Copy link

@cvigo cvigo commented Dec 19, 2018

It is indeed supposed to be inherited, so that's not intended behavior, if the server sends a full chain including the intermediate. There might also be something about the root CA that makes it disqualified from forming chains, if you can share it at filippo@golang.org I can look into that.

Done, thanks!!

@diligiant
Copy link

@diligiant diligiant commented Jan 9, 2019

@FiloSottile failed

gotip version
go version devel +5efe9a8 Wed Jan 9 07:21:16 2019 +0000 darwin/amd64
 GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: 2 certs have a trust policy
crypto/x509: verify-cert rejected CN=com.apple.servermgrd,C=US: "Cert Verify Result: CSSMERR_TP_CERT_EXPIRED"
crypto/x509: verify-cert approved CN=sks-keyservers.net CA,O=sks-keyservers.net CA,ST=Oslo,C=NO
crypto/x509: verify-cert approved CN=sks-keyservers.net CA,O=sks-keyservers.net CA,ST=Oslo,C=NO
crypto/x509: ran security verify-cert 3 times
Number of trusted certs = 1
Cert 0: sks-keyservers.net CA
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Number of trusted certs = 1
Cert 0: com.apple.servermgrd
   Number of trust settings : 0
--- FAIL: TestSystemRoots (0.52s)
    root_darwin_test.go:35:     cgo sys roots: 221.194313ms
    root_darwin_test.go:36: non-cgo sys roots: 153.880594ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=sks-keyservers.net CA,O=sks-keyservers.net CA,ST=Oslo,C=NO (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
    root_darwin_test.go:106: expired certificate only present in cgo pool (acceptable): CN=com.apple.servermgrd,C=US
FAIL
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: com.apple.servermgrd returned 1
crypto/x509: sks-keyservers.net CA returned 4
FAIL	crypto/x509	0.543s
@dmitshur
Copy link
Member

@dmitshur dmitshur commented Jan 9, 2019

Latest tip (commit 99ea99e) passed on my personal Mac with macOS Mojave 10.14.2 (18C54).


$ ../bin/go version
go version devel +99ea99ec4c Wed Jan 9 14:49:46 2019 +0000 darwin/amd64
$ GODEBUG=x509roots=1 ../bin/go test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: exec ["/usr/bin/security" "trust-settings-export" "/var/folders/b8/66r1c5856mqds1mrf2tjtq8w0000gn/T/x509trustpolicy762449975/user"]: exit status 1, SecTrustSettingsCreateExternalRepresentation: No Trust Settings were found.

crypto/x509: exec ["/usr/bin/security" "trust-settings-export" "-d" "/var/folders/b8/66r1c5856mqds1mrf2tjtq8w0000gn/T/x509trustpolicy762449975/admin"]: exit status 1, SecTrustSettingsCreateExternalRepresentation: No Trust Settings were found.

crypto/x509: 0 certs have a trust policy
crypto/x509: ran security verify-cert 0 times
--- PASS: TestSystemRoots (0.30s)
    root_darwin_test.go:35:     cgo sys roots: 178.537621ms
    root_darwin_test.go:36: non-cgo sys roots: 96.198864ms
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
PASS
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
ok  	crypto/x509	0.315s
@zdjones
Copy link
Contributor

@zdjones zdjones commented Jan 20, 2019

Failed on macOS High Sierra 10.13.6 (17G4015)

go $ devgo version
go version devel +5538a9a34f Fri Jan 18 22:41:47 2019 +0000 darwin/amd64
go $ GODEBUG=x509roots=1 devgo test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: exec ["/usr/bin/security" "trust-settings-export" "-d" "/var/folders/7j/jp_42qg96kq95d514353znfc0000gn/T/x509trustpolicy901297289/admin"]: exit status 1, SecTrustSettingsCreateExternalRepresentation: No Trust Settings were found.
crypto/x509: 1 certs have a trust policy
crypto/x509: verify-cert approved CN=GD-ISE-1.msad.igl
crypto/x509: verify-cert approved CN=GD-ISE-1.msad.igl
crypto/x509: ran security verify-cert 2 times
Number of trusted certs = 1
Cert 0: GD-ISE-1.msad.igl
   Number of trust settings : 3
   Trust Setting 0:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
SecTrustSettingsCopyCertificates: No Trust Settings were found.
--- FAIL: TestSystemRoots (0.50s)
    root_darwin_test.go:35:     cgo sys roots: 222.450266ms
    root_darwin_test.go:36: non-cgo sys roots: 153.410374ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=GD-ISE-1.msad.igl (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
FAIL
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: GD-ISE-1.msad.igl returned 4
FAIL	crypto/x509	0.525s
@vdemario
Copy link
Contributor

@vdemario vdemario commented Jan 21, 2019

Tests are passing for me now! Yay 🎉
Sorry for the delay.

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 15, 2019

Change https://golang.org/cl/162860 mentions this issue: [release-branch.go1.11] crypto/x509: fix root CA extraction on macOS (cgo path)

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 15, 2019

Change https://golang.org/cl/162861 mentions this issue: [release-branch.go1.11] crypto/x509: fix root CA extraction on macOS (no-cgo path)

@Lax77
Copy link

@Lax77 Lax77 commented Feb 15, 2019

I am running go 1.11.5 version on Mac 10.13.6 version, I keep getting the x509: certificate signed by unknown authority when I try to get dependencies. eg: k8s.io/api. My gotip test result is a pass for me though. All I was doing is trying to fetch dependencies with operator-sdk. Any suggestions on how I can go about resolving it ?

$go get k8s.io/api

package k8s.io/api: unrecognized import path "k8s.io/api" (https fetch: Get https://k8s.io/api?go-get=1: x509: certificate signed by unknown authority)

$dep ensure

The following issues were found in Gopkg.toml:

  ✗ unable to deduce repository and source type for "k8s.io/cli-runtime": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/cli-runtime?go-get=1": Get https://k8s.io/cli-runtime?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/helm": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/helm?go-get=1": Get https://k8s.io/helm?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/kube-aggregator": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/kube-aggregator?go-get=1": Get https://k8s.io/kube-aggregator?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/apimachinery": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/apimachinery?go-get=1": Get https://k8s.io/apimachinery?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/kubernetes": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/kubernetes?go-get=1": Get https://k8s.io/kubernetes?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/apiextensions-apiserver": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/apiextensions-apiserver?go-get=1": Get https://k8s.io/apiextensions-apiserver?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/client-go": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/client-go?go-get=1": Get https://k8s.io/client-go?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/kube-openapi": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/kube-openapi?go-get=1": Get https://k8s.io/kube-openapi?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/api": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/api?go-get=1": Get https://k8s.io/api?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "k8s.io/apiserver": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://k8s.io/apiserver?go-get=1": Get https://k8s.io/apiserver?go-get=1: x509: certificate signed by unknown authority
  ✗ unable to deduce repository and source type for "sigs.k8s.io/controller-runtime": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://sigs.k8s.io/controller-runtime?go-get=1": Get https://sigs.k8s.io/controller-runtime?go-get=1: x509: certificate signed by unknown authority

ProjectRoot name validation failed```
</p>
</details>
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Feb 18, 2019

@Lax77 Please run the test from #24652 (comment). If it passes, just wait for 1.11.6 or 1.12. If not, please open a new issue with the output and tag me.

(Locking this issue because we shipped a fix, it's getting hard to follow and we shouldn't keep pinging everyone. If you have a similar issue or if you run the tests and they fail, please open a new issue referencing this one and tagging me.)

@golang golang locked as resolved and limited conversation to collaborators Feb 18, 2019
@gopherbot
Copy link

@gopherbot gopherbot commented Apr 14, 2020

Change https://golang.org/cl/227037 mentions this issue: crypto/x509: use Security.framework without cgo for roots on macOS

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.