Skip to content
This repository has been archived by the owner on Jul 4, 2023. It is now read-only.

openssl: apply yosemite certificate fix #38495

Closed
wants to merge 1 commit into from
Closed

openssl: apply yosemite certificate fix #38495

wants to merge 1 commit into from

Conversation

DomT4
Copy link
Member

@DomT4 DomT4 commented Apr 9, 2015

Emergency workaround for #38491.

@DomT4
Copy link
Member Author

DomT4 commented Apr 9, 2015

Failure was unrelated. Looks good otherwise.

==> brew test --verbose postgresql                   FAILED
Testing postgresql
==> /usr/local/Cellar/postgresql/9.4.1/bin/initdb /private/tmp/postgresql20150409-97200-fud3mg
The files belonging to this database system will be owned by user "brew".
This user must also own the server process.

@DomT4
Copy link
Member Author

DomT4 commented Apr 9, 2015

Removed the wget commit here since the wget test passed the bot successfully and we don't actually want to mass-revision wget.

@tdsmith
Copy link
Contributor

tdsmith commented Apr 9, 2015

👎; I think Misty's point that this is probably intentional is important. Even if it's a mistake I don't believe Homebrew should be making decisions about trust.

@DomT4
Copy link
Member Author

DomT4 commented Apr 9, 2015

I don't believe Homebrew should be making decisions about trust.

FWIW, We already are. Just less explicitly. We choose to trust that Apple's keychains are secure over choosing to trust something like Mozilla's public bundle, and so on. Which is risky in itself, because we presume nothing has compromised the keychain or the user or piece of software hasn't inserted a rogue root. The trust is usually just much less explicit.

As I said in the other thread, not attached to this PR. Asked whether a workaround was worthwhile, and this is the best workaround to there is to the situation. I don't love it either, but I don't love it because playing with roots is like sticking your hand in the devil's mouth and expecting that entity to not to set you on fire. We're trusting someone whatever decision we make in the end.

@tdsmith
Copy link
Contributor

tdsmith commented Apr 9, 2015

I intended to express my opinion about whether we should accept this PR, not an opinion about your judgement. :) Your point that the system keychain was not received inscribed in stone from a pure, faultless deity is well taken.

@DomT4
Copy link
Member Author

DomT4 commented Apr 10, 2015

Apologies. Didn't mean to imply offence or anything, or rub you the wrong way either. I just find it hard to find nice things about the whole CA system, heh.

Sent from my iPhone

On 10 Apr 2015, at 00:55, tim smith notifications@github.com wrote:

I intended to express my opinion about whether we should accept this PR, not an opinion about your judgement. :) Your point that the system keychain was not received inscribed in stone from a pure, faultless deity is well taken.


Reply to this email directly or view it on GitHub.

@chdiza
Copy link
Contributor

chdiza commented Apr 10, 2015

Gmail does not work anymore because of this, if you use a client such as mutt[1], or have a script that uses, e.g., the ruby imap library. Something has to be done, in Homebrew, even if it is merely putting temporary advice in a prominent place about how to workaround the bug.

As long as @DomT4's tweak is removed as soon as the situation is fixed, I don't see what's wrong with it. It merely puts 10.10 users in the same "trusty" position as Mavericks users (unless...Oh God, did the Security Update for Mavericks break it there too?).

[1] Actually, one can get around this in mutt by manually accepting the cert when mutt prompts you, but this is basically the same as @DomT4's solution: it puts the cert on your machine, where it will be accepted by mutt until you delete it.

@felixbuenemann
Copy link
Sponsor Contributor

Given that even the recent RapidSSL SHA256 intermediate CA goes back to the Equifax root (as signer of the GeoTrust Global CA), this is going to be a lot of fun to get rid off. There's a vast number of RapidSSL DV certs out there…

Not putting a fix for this in homebrew just puts the burden of brewing their own ca bundle on the users, which will most likely carry out the first fix they find on google instead of a minimally invasive change as proposed by this PR.

@MikeMcQuaid
Copy link
Member

Doing wget https://google.com fails without this which seems kinda crazy. I'm in favour of carrying a short-term fix which we then revision and remove when Google has sorted their 💩 out.

@MikeMcQuaid
Copy link
Member

Pushed. Happy to revisit this and remove it when at least https://google.com isn't broken.

@tdsmith
Copy link
Contributor

tdsmith commented Apr 10, 2015

Should clarify that a GeoTrust Global CA certificate with fingerprint DE 28 F4 A4 FF E5 B9 2F A3 C5 03 D1 A3 49 A7 F9 96 2A 82 12 which is listed on the Geotrust roots page is and has been an OS X trusted root. It is signed by the outgoing Equifax root but trusting the Equifax root is not necessary because the GeoTrust cert is in the trusted root store. It is self-signed.

When I connect to google.com with openssl s_client I see trust is mediated through a certificate with subject CN=GeoTrust Global CA with fingerprint 73:59:75:5C:6D:F9:A0:AB:C3:06:0B:CE:36:95:64:C8:EC:45:42:A3 which is signed by the outgoing Equifax Secure Certificate Authority root. This certificate presents the same public key as the certificate with the same CN but a different fingerprint in the OS X trust store.

I don't understand why a trust path can't be constructed which terminates at the trusted GeoTrust Global CA certificate since the public keys match.

@tdsmith
Copy link
Contributor

tdsmith commented Apr 10, 2015

This apparently outstanding OpenSSL issue may be relevant: https://rt.openssl.org/Ticket/Display.html?id=2732&user=guest&pass=guest

@tdsmith
Copy link
Contributor

tdsmith commented Apr 10, 2015

The validation path is presented by the server: https://tools.ietf.org/html/rfc5246#section-7.4.2
so as I understand it Google could present the self-signed DE 28 F4 cert instead of the cross-signed 73:59:75 cert if Google believed that enough people have accepted GeoTrust Global CA cert into their trust store, and then the problem would disappear. Alternatively, I think openssl ought to handle this gracefully but doesn't seem to.

@DomT4 DomT4 deleted the ohapple branch April 10, 2015 14:14
@DomT4
Copy link
Member Author

DomT4 commented Apr 10, 2015

Just to confirm, this impacts LibreSSL as well. On the upside, very little is built against LibreSSL right now.

@DomT4
Copy link
Member Author

DomT4 commented Apr 10, 2015

Adam Langley at Google was kind enough to reply on the Google cert issue:

This is because the chain building on OS X is pretty poor. The Google
chain should terminate at GeoTrust for any modern system that has the
GeoTrust root. We also send the cross-sign to Equifax for older
systems. OS X is following the cross-sign and then getting stuck
because it doesn't know how to backtrack :( This is one of the reasons
that we're currently implementing our own certificate verification
library for Chrome so that we don't have to use OS X's.

Cheers

AGL

@tdsmith
Copy link
Contributor

tdsmith commented Apr 10, 2015 via email

@DomT4
Copy link
Member Author

DomT4 commented Apr 10, 2015

Yeah, It's a shared OpenSSL/LibreSSL failure. Secure Transport is actually working in this case.

Using cURL as an example because it builds against nearly any crypto library on the planet, using the same system-generated Equifax-free .pem - OpenSSL and LibreSSL fail identically, GnuTLS connects as expected, as does Secure Transport. PolarSSL doesn't build at the moment because obviously they went and renamed their lib last update and cURL seems to have the old lib name built in.

Interestingly, both OpenSSL and LibreSSL will actually make secure connections against Google.com via openssl s_client -connect google.com:443 but if you do it without specifying the -CApath you get this:

CONNECTED(00000003)
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify error:num=20:unable to get local issuer certificate
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority G2
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---

If you do instead openssl s_client -CApath /usr/local/etc/openssl/cert.pem -connect google.com:443 you get this:

CONNECTED(00000003)
depth=3 C = US, O = Equifax, OU = Equifax Secure Certificate Authority
verify return:1
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify return:1
depth=1 C = US, O = Google Inc, CN = Google Internet Authority G2
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = *.google.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=*.google.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority G2
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---

Welp.

@DomT4
Copy link
Member Author

DomT4 commented Apr 10, 2015

There's an old 2009 Ubuntu bug on the s_client differences here. I don't think it's super relevant/related to our problem, but it's an interesting issue.

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

So, is this bug in openssl actually reported to them? As in, within the last couple of days? I tried looking on rt.openssl.org, but I find that site opaque.

I hate to think what we're in for, if the reports linked to in this thread have just languished unfixed for years.

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

Also: this bug doesn't seem to affect the system openssl on 10.10.3. Is that because it just doesn't affect old versions of openssl, or is it because Apple have patched their way around this?

@tdsmith
Copy link
Contributor

tdsmith commented Apr 11, 2015

If you want to check the openssl098 formula in -versions that could shed light.

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

You're right about the system OpenSSL. Just did a full wget build against it and brew test --verbose wget passed fine. Interesting. Trying ours now.

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

Our openssl098 causes brew test --verbose wget to fail. Apple must have danced with the OpenSSL source code somehow, or something in the last 2 versions of the 098 branch caused the issue.

System is OpenSSL 0.9.8zd 8 Jan 2015.
We are OpenSSL 0.9.8zf 19 Mar 2015

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

Diff of headers between Apple OpenSSL 0.9.8zd and Homebrew OpenSSL 0.9.8zd. If you throw it an editor that colorises diffs automatically, it's a lot easier to read.

Aside Apple's love of deprecating stuff, there's a few completely new files and some semi-interesting #ifdefing. Not much particularly jumps out as an "Ah!" moment.

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

I've just discovered more instances of what must be dropped certs. When I try to use mutt on a hotmail account, I am now (10.10.3) getting asked to manually accept the following two certs, which used to be handled automatically by the system certs wedged into openssl:

This certificate belongs to:
   GlobalSign Organization Validation CA - G2
   Unknown
   GlobalSign nv-sa
   Unknown
   Unknown

This certificate was issued by:
   GlobalSign Root CA
   Unknown
   GlobalSign nv-sa
   Root CA
   Unknown

This certificate is valid
   from Apr 13 10:00:00 2011 GMT
     to Apr 13 10:00:00 2022 GMT

Fingerprint: 650B EA04 1977 80E1 D00D 24E8 DC91 8A15

And,

This certificate belongs to:
   *.hotmail.com
   Unknown
   Microsoft Corporation
   Unknown
   Redmond

This certificate was issued by:
   GlobalSign Organization Validation CA - G2
   Unknown
   GlobalSign nv-sa
   Unknown
   Unknown

This certificate is valid
   from Apr 24 20:35:09 2013 GMT
     to Apr 24 20:35:09 2016 GMT

Fingerprint: 8660 F638 1C84 A6AC 9492 512F 679A 7D76

If I temporarily replace osx_cert.pem with a copy of the one that gets brewed in 10.10.2, mutt doesn't ask me to verify those.

BTW, I apologize if this is noise and not adding anything of interest. (I don't know much about certs, other than "they exist, and they have to do with security".) Perhaps this is some evidence that we're dealing with an Apple bug after all.

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

@chdiza If you have a copy of the untouched 10.10.2 keychain-generated .pem could you possibly gist that? Would allow us to generate a diff and see what got the axe.

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

Thanks! Did you add the Equifax cert back? The top cert in green is the Equifax cert that's supposed to be missing on 10.10.3.

Unless in this case, green = removed and red = added. My brain is barely functioning today, I've apparently forgotten how to read a diff 😖

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

Thanks! Did you add the Equifax cert back?

Yeah, I forgot, I did add it back. I added it to the System keychain, which means brewing openssl put it into osx_cert.pem.

@chdiza
Copy link
Contributor

chdiza commented Apr 11, 2015

Are these things base64 encoded in the .pem files?

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

Yeah, I forgot, I did add it back.

Cool, thanks. That was confusing me 😸.

Are these things base64 encoded in the .pem files?

Yup. Looks like it.

@DomT4
Copy link
Member Author

DomT4 commented Apr 11, 2015

Everything added and removed in 10.10.3: Clicky. Base64 is a nightmare to work with.

@DomT4
Copy link
Member Author

DomT4 commented Apr 13, 2015

The latest news here is... not great. Essentially, we're going to be waiting on OpenSSL to fix their cert chain building, so... yeah. Next step is filing a bug report with OpenSSL and OpenBSD.

@DomT4
Copy link
Member Author

DomT4 commented Apr 18, 2015

@chdiza
Copy link
Contributor

chdiza commented Apr 18, 2015

@DomT4 Thanks for filing those! The four-year waiting period for a fix has thus begun :)

MikeMcQuaid pushed a commit that referenced this pull request Apr 27, 2015
This *will* land in 1.0.2b, but it's a better solution than us
applying an old, outdated, weak Equifax cert till that point.

I've pinged OpenSSL to check I'm not being stupid to cherry-pick these
patches, but they should be fine - I pulled both related patches,
so it's not like we're being overly selective. I also asked whether
there was a release schedule for the 1.0.2b release with these fixes,
but I don't particularly expect to be given an answer given OpenSSL's
often (understandably) sensitive release schedule.

Fixes #38495
Fixes #38491

Upstream discussion:
https://www.mail-archive.com/openssl-dev@openssl.org/msg38674.html

Closes #38897.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
Noctem pushed a commit to Noctem/homebrew that referenced this pull request May 2, 2015
This *will* land in 1.0.2b, but it's a better solution than us
applying an old, outdated, weak Equifax cert till that point.

I've pinged OpenSSL to check I'm not being stupid to cherry-pick these
patches, but they should be fine - I pulled both related patches,
so it's not like we're being overly selective. I also asked whether
there was a release schedule for the 1.0.2b release with these fixes,
but I don't particularly expect to be given an answer given OpenSSL's
often (understandably) sensitive release schedule.

Fixes Homebrew#38495
Fixes Homebrew#38491

Upstream discussion:
https://www.mail-archive.com/openssl-dev@openssl.org/msg38674.html

Closes Homebrew#38897.

Signed-off-by: Mike McQuaid <mike@mikemcquaid.com>
@Homebrew Homebrew locked and limited conversation to collaborators Jul 10, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants