Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/x509: Trust setting not inherited on darwin #30471

Closed
vdobler opened this issue Feb 28, 2019 · 16 comments

Comments

Projects
None yet
9 participants
@vdobler
Copy link
Contributor

commented Feb 28, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/m/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/m/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/m/go/src/git.intern.migros.net/devops-api-mobile/ch.migros.lottery/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f6/vr81k40j5y35bpj039w1hwfdrvg106/T/go-build545405925=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go build

What did you expect to see?

Nothing (a successful build).

What did you see instead?

go: git.intern.migros.net/package7path@v1.3.0: unrecognized import path "git.intern.migros.net/package/path" (https fetch: Get https://git.intern.migros.net/package/path?go-get=1: x509: certificate signed by unknown authority)
go: error loading module requirements

More details

This is basically just issue #24652 but with Go 1.12

The same fix (mark root ca as "Allways trust") as described in #24652 (comment)
makes the problem go away.

Curl and Browsers do accept the certificate. E.g

$ curl -v "https://git.intern.migros.net/package/path?go-get=1"
*   Trying 164.xxx.xxx.xxx...
* TCP_NODELAY set
* Connected to git.intern.migros.net (164.xxx.xxx.xxx) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: git.intern.migros.net
* Server certificate: Migros System CA 2
* Server certificate: Migros Root CA 2
> GET /package/path?go-get=1 HTTP/1.1
> Host: git.intern.migros.net
> User-Agent: curl/7.54.0
> Accept: */*

But running TestSystemRoots yields:

$ GODEBUG=x509roots=1 ../bin/go test -v -run TestSystemRoots crypto/x509
=== RUN   TestSystemRoots
crypto/x509: 6 certs have a trust policy
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=webproxy.dc.migros.ch,OU=Informatik,O=Migros,L=Zuerich,ST=Zuerich,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert rejected CN=dlv-cert: "Cert Verify Result: Invalid Extended Key Usage for policy"
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=Migros Root CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
crypto/x509: verify-cert approved CN=Migros User CA 2,O=Migros,C=CH
crypto/x509: verify-cert approved CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
crypto/x509: verify-cert approved CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
crypto/x509: ran security verify-cert 13 times
Number of trusted certs = 2
Cert 0: Migros Root CA 2
   Number of trust settings : 4
   Trust Setting 0:
      Policy OID            : SSL
      Policy String         : autodiscover.migros.net
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 1:
      Policy OID            : SSL
      Policy String         : autodiscover.migros.net
      Allowed Error         : Host name mismatch
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 2:
      Policy OID            : Apple X509 Basic
      Policy String         : autodiscover.migros.net
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
   Trust Setting 3:
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustRoot
Cert 1: macman.migros.net
   Number of trust settings : 2
   Trust Setting 0:
      Policy OID            : EAP
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
   Trust Setting 1:
      Policy OID            : Apple X509 Basic
      Allowed Error         : CSSMERR_TP_CERT_EXPIRED
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Number of trusted certs = 5
Cert 0: Migros User CA 2
   Number of trust settings : 1
   Trust Setting 0:
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 1: QuoVadis Swiss Advanced CA G2
   Number of trust settings : 1
   Trust Setting 0:
      Result Type           : kSecTrustSettingsResultTrustAsRoot
Cert 2: webproxy.dc.migros.ch
   Number of trust settings : 0
Cert 3: dlv-cert
   Number of trust settings : 0
Cert 4: Migros Root CA 2
   Number of trust settings : 0
--- FAIL: TestSystemRoots (0.31s)
    root_darwin_test.go:35:     cgo sys roots: 56.90889ms
    root_darwin_test.go:36: non-cgo sys roots: 150.793582ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros User CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=QuoVadis Swiss Advanced CA G2,O=QuoVadis Trustlink Switzerland Ltd.,C=CH
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=Migros Root CA 2,O=Migros,C=CH (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=macman.migros.net,OU=Migros Genossenschafts Bund,O=Migros Genossenschafts Bund,L=Zuerich,ST=Zueich,C=CH
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US
    root_darwin_test.go:99: off-EKU certificate only present in cgo pool (acceptable): CN=dlv-cert
FAIL
crypto/x509: kSecTrustSettingsResultInvalid = 0
crypto/x509: kSecTrustSettingsResultTrustRoot = 1
crypto/x509: kSecTrustSettingsResultTrustAsRoot = 2
crypto/x509: kSecTrustSettingsResultDeny = 3
crypto/x509: kSecTrustSettingsResultUnspecified = 4
crypto/x509: Migros User CA 2 returned 4
crypto/x509: QuoVadis Swiss Advanced CA G2 returned 4
crypto/x509: webproxy.dc.migros.ch returned 1
crypto/x509: dlv-cert returned 1
crypto/x509: Migros Root CA 2 returned 4
crypto/x509: Migros Root CA 2 returned 4
crypto/x509: macman.migros.net returned 4
FAIL	crypto/x509	0.317s
@vdobler

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2019

@FiloSottile In #24652 you requested to be tagged if the problem is not completely solved in Go 1.12.

@SteelPhase

This comment has been minimized.

Copy link

commented Feb 28, 2019

I've run into the same issue, and while I can't share the output of the test in question. I can confirm that marking the intermediate ca as Always Trusted resolves the issue for me. I would hope that this would be fixed in a Go 1.12.x release

@FiloSottile

This comment has been minimized.

Copy link
Member

commented Mar 1, 2019

Thank you for the report. Is this a regression or was it broken in Go 1.11 as well?

@SteelPhase

This comment has been minimized.

Copy link

commented Mar 1, 2019

This was working for me in Go 1.11. I installed my companies root CA, and two intermediate CAs in the system keychain. At the time when I did this, it was required because Go didn't check the login keychain. That specific details appears to have changed, but I'm not 100% sure that was introduced in Go 1.12. I only had the root CA marked as Always Trusted.

After running into this issue today, I cleaned up the certs in my system keychain, and moved them to the login keychain. I tested this in both locations. When only the root CA is marked Always Trusted, go doesn't trust either of the intermediate certs. When all 3 certs are marked Always Trusted, go has no trust issues with the certificates.

As an additional note, The ouput of GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509 didn't change based on Always Trusted settings, and I assume that is a red herring.

I can confirm that a script that relies on the certificate being trusted works correctly when I change the intermediate CA it depends on to Always Trusted

@SteelPhase

This comment has been minimized.

Copy link

commented Mar 1, 2019

I'm going to include this tidbit just because it's slightly related

I'm on macOS Mojave 10.14.3, and it appears security handles pointing login.keychain to login.keychain-db internally. I shasum'd the output of the commands with the different paths below it it's the same value.

keychains = append(keychains,
filepath.Join(home, "/Library/Keychains/login.keychain"),
// Fresh installs of Sierra use a slightly different path for the login keychain
filepath.Join(home, "/Library/Keychains/login.keychain-db"),
)

steel@computer:src$ls -la ~/Library/Keychains/
drwxr-xr-x  11 steel  000000000     352 Feb 28 19:36 .
drwx------@ 75 steel  000000000    2400 Feb 11 10:21 ..
-rw-r--r--@  1 steel  000000000    6148 Jan  7 15:33 .DS_Store
-rw-r--r--@  1 steel  000000000  736696 Feb 28 19:36 login.keychain-db
-rw-------   1 steel  000000000   30740 Feb 12 11:13 metadata.keychain-db
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9  -
steel@computer:src$security find-certificate -a ~/Library/Keychains/login.keychain-db | shasum
a2b9a124e87200b0663f8b45df45c39584663bd9  -
@kewilson

This comment has been minimized.

Copy link

commented Mar 26, 2019

@FiloSottile .. I sent you an email yesterday regarding this same issue as mentioned in 24652 still occurring. I'm trying to get past the issue now so if you want me to try some stuff to help out please let me know.

golang 1.12.1.
macos 10.13.6
intellij ultimate 2018.3.5

@kewilson

This comment has been minimized.

Copy link

commented Mar 26, 2019

I employed the All Trust work around on three certs using system defaults, all the others in play were already trusted and still receive "x509: certificate signed by unknown authority" when calling urls from golang project. As mentioned by others pasting the url in chrome and it returns data as expected.

@kewilson

This comment has been minimized.

Copy link

commented Mar 27, 2019

Anyone? Bueller? Bueller?

@adamdecaf

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

Could your issue be fixed by #30672 ? There's an open CL, could you try it?

https://go-review.googlesource.com/c/go/+/166219

@kewilson

This comment has been minimized.

Copy link

commented Apr 1, 2019

@adamdecaf .. how might I go about trying this since it hasn't been merged as far as I can tell?

$>brew upgrade golang
golang 1.12.1 already installed

@andlabs

This comment has been minimized.

Copy link
Contributor

commented Apr 17, 2019

I'll try it, because I'm having a similar problem with the same error message (in this case, it's an IRC server's certificate; I don't remember why it's misconfigured). Patching 33e5da4, running on 10.12.6, using the Cherry-Pick option in Gerrit.

Before:

=== RUN   TestSystemRoots
--- FAIL: TestSystemRoots (0.30s)
    root_darwin_test.go:35:     cgo sys roots: 81.121343ms
    root_darwin_test.go:36: non-cgo sys roots: 172.672874ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=irc.rustedlogic.net,OU=Server Admins,O=Rusted Logic IRC Network,L=Example City,ST=Example State,C=US (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=irc.gnome.org,OU=Domain Control Validated+OU=PositiveSSL Multi-Domain
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

After:

=== RUN   TestSystemRoots
--- PASS: TestSystemRoots (0.30s)
    root_darwin_test.go:35:     cgo sys roots: 88.204333ms
    root_darwin_test.go:36: non-cgo sys roots: 168.548068ms
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

@kewilson You will need to build Go from source, downloading the repository via git and keeping it at HEAD. In Gerrit, there will be a button at the bottom right of the description that says DOWNLOAD; click it to get instructions on how to apply the Gerrit patch locally.

@gopherbot

This comment has been minimized.

Copy link

commented May 22, 2019

Change https://golang.org/cl/178539 mentions this issue: crypto/x509: include roots with empty or multiple policies on macOS

@gopherbot gopherbot closed this in 42bb476 May 22, 2019

@FiloSottile

This comment has been minimized.

Copy link
Member

commented May 22, 2019

This should be now fixed at tip. Please test it with https://golang.org/dl/gotip and report back.

$ go get golang.org/dl/gotip
$ gotip download
$ GODEBUG=x509roots=1 gotip test -v -run TestSystemRoots crypto/x509
$ gotip run [YOUR_PROGRAM]
@andlabs

This comment has been minimized.

Copy link
Contributor

commented May 22, 2019

Can confirm the normal ./all.bash from HEAD works for me; thanks. (Not sure about gotip; I was waiting for this so I could set up my Go development environment to begin with =P )

@tommyknows

This comment has been minimized.

Copy link

commented Jul 1, 2019

cannot confirm this to be working.
I cloned the repository and ran ./all.bash. It works if running as root / sudo, but I guess the certificates are not present there.

Running it as my normal user, I get:

--- FAIL: TestSystemRoots (0.82s)
    root_darwin_test.go:35:     cgo sys roots: 464.908686ms
    root_darwin_test.go:36: non-cgo sys roots: 293.10636ms
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=[redacted] (verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=[redacted]
    root_darwin_test.go:77: certificate only present in non-cgo pool: CN=[redacted](verify error: x509: certificate signed by unknown authority)
    root_darwin_test.go:79: signed certificate only present in non-cgo pool (acceptable): CN=Developer ID Certification Authority,OU=Apple Certification Authority,O=Apple Inc.,C=US

Am I missing something?

@FiloSottile

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

@tommyknows Please post the full (redacted) output of #30471 (comment) in a new issue and tag me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.