CERTIFICATE_VERIFY_FAILED with 2.5.2, works with 2.5.1 #2455
Comments
|
FWIW, this is with python 3.4.2 and OS X 10.10. |
|
@t-8ch seems related to the urllib3 patch to disable built-in hostname verification possibly. |
|
Would it be helpful for me to run a git-bisect or something? |
|
Not really. The change is upstream of us. Thanks for the bug though. |
|
@sigmavirus24 I would rather blame the new ciphersuite or cert bundle (not tested) |
|
Could easily be the new cert bundle. |
|
I'm assuming that you hand-typed that code @rmcgibbo? The reason I ask is because you're accessing HTTP urls there. |
|
I cannot reproduce this on my copy of 2.5.2. |
|
That suggests it's not the cert bundle. |
|
@Lukasa I can now reproduce it, the URLs are right, there seems to be an redirect |
|
So there is. I still can't reproduce though. |
|
@Lukasa Are you running from a git checkout? It works for me there to, but not from a wheel, will try a sdist |
|
Nope, from a wheel. Python 2.7.9 (default, Dec 19 2014, 06:00:59)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.__version__
'2.5.2'
>>> requests.__file__
'/usr/local/lib/python2.7/site-packages/requests/__init__.pyc'
>>> r = requests.get("http://conda.binstar.org/omnia/linux-64/fftw3f-3.3.3-1.tar.bz2")
>>> r.status_code
200
>>> r.url
u'https://binstar-cio-packages-prod.s3.amazonaws.com/531fd36fe1dad154c8ea89c5/54b0d1045e76836c22735094?Signature=JRp8nPqRc6wXijqGg5JjDnIytz8%3D&Expires=1424763499&AWSAccessKeyId=ASIAIIYL6XQN5BMSPEFA&response-content-disposition=attachment%3B%20filename%3Dfftw3f-3.3.3-1.tar.bz2&response-content-type=application/x-tar&x-amz-security-token=AQoDYXdzEM///////////wEa4APi0P1wChyd8cFWFddkWZVgQYQQPAli9WBPWotWMA77n%2BqFSO92c2mb607CMadC7Jdo21sJVmtuKu4gM2vQhzX8Uy8NXWHtrBpkAgulTshstH1oFEcrkfV2wVtcxiHPrWTrEw7UTHAqDBQppLNHIHTQjs%2Bks0d/yQ//i6iCJTulVjNUWsXv%2BGre%2BMX2s%2BetTldscc2vbHsrzLG%2BvlOGrKbgUaDzLHWa7RgA6z4DxJdu0inmSPGIoV02fFonVRrwjI3qUrw50dmIFIW5SFxN9XYnDnXf1DiJ2b3vKnrj9I%2BYEY%2ByP5pHpSWvF%2BIRrmBDX6KHf4QKF5%2Bo01Y2Oeqq%2BhQncFFdEt6QBU6Eqa%2B3kb6iwdJdPAXr7HOgd5TugixLDNnKaWuF%2B1grizrD8Drz6nYhb0sgZjazDmjNSFpNXwCyYDWQNNIL7o13Sh6q2fB8sWD5oCEJu22HB4CWBxq3AC/APi86cPSZ/5RiRnv2pz%2BBB7HGNwTZveJaSfSX9ZR2pq6oZOT5kpP2aJrj4wwvE7BC0kf%2B2au4B%2BGH5uHNZvm42%2BwWophhx9NcIjdX/F9g%2B8cnkrx9nooNcB%2BVzxgXZG%2BCzPuhdFbcRFzWaqz0W/T5eJWVpmCIcKpGlIibqcwAjGAg16%2BwpwU%3D' |
|
Potentially also relevant: >>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2 22 Jan 2015' |
|
It seems to be the bundle. |
|
@t-8ch What makes you say that? |
|
It works with the old one and does not with the new one, also it works with my system bundle. |
|
Aha, yep, can now reproduce. Don't know what I was doing differently before. =( |
|
So the cert chain is: |
|
That Class 3 Public Primary Certification Authority - G5 certificate is in the bundle. The one Mozilla uses as its trust root when I access the page in Firefox is definitely 100% in that bundle. Serial numbers match and everything. |
|
Weirdly though, if I pass that certificate to curl to use to validate the connection, it doesn't like it. |
|
A warning that so far I've not been able to reproduce this in requests on either Windows or OS X, presumably because their built in cert-stores are saving the day. |
|
Also, running |
|
But cert verification definitely fails with requests. So that's perplexing. |
|
At least we get the same leaf cert with s_client and requests. |
|
@t-8ch To verify: does your bundle succeed with |
|
Now I am really confused: |
|
Installed from wheel: py2 + 2.5.1: works py2 + 2.5.2: breaks Git checkout: py2 + 2.5.1: works py2 + 2.5.2: breaks |
|
All on linux with 3.4.2 and 2.7.9 |
|
@asmeurer I wasn't clear enough: I think we should write such a tool. |
|
Hi, sorry to add my experience, even if I can't take the time to read all of this. Yes. It sounds amazing, but for me, it worked, and I really don't have a clue why. For information, here is the last certificate in the cacert.pem: |
|
@Glandos I'm not certain the issue you're having is the same as is being discussed here. |
|
On a related note, I think we should change the issue title. |
|
OK I've opened a new issue then, in #2541. |
|
Ok, this is utterly craptastic. I'd hoped to be able to programmatically find the cross-signed roots by examining the bundles, but in practice that doesn't work. I think I'll need to do this by hand: emit the certs that were removed and those that were added, and look for those that are plausibly cross-signing related. This sucks pretty hard. |
|
/cc @kennethreitz: this is the issue that tracks the work we need to do for certifi, btw. |
|
Alright, fuck that. Here's my new plan. The following are the Mozilla issues that track the removal of 1024 bit certs that I could find: I plan to identify every certificate that was removed by these purges and check whether any of them are in the current requests cert bundle. If they are, I plan to restore them to the new certifi bundle. Does anyone think this plan is bad? @reaperhulk @dstufft @alex @sigmavirus24 |
|
What's the long term plan for getting rid of them? Certainly that plan is better than just not updating the bundle, but those certificates were removed for reason. With them no longer in the Mozilla bundle, Mozilla is no longer going to be monitoring those certificates for bad-ness so it feels like, in the long term, keeping them around is a bad scene. |
|
My plan is as follows:
Thoughts? |
|
(Oh, btw, requests will probably stay on step 1 of that plan until step 3 lands in certifi). |
|
Ok, in practice 881553 appears to contain the full list. Applying that list to the old cacerts.pem from requests found 14 certificates currently in the bundle that are not in what I believe re-adding these certificates will fix our problem. |
|
(Oh, /cc @t-8ch as well). |
|
That plan sounds reasonable to me then, as long as we have a plan for it and we're not just adding the certificates back and YOLO. |
|
I plan to solidify those fuzzy 'month or so' measurements into definite dates, before we release. |
|
Alright, the 'fixed' (s/fixed/ruined/) bundle is at certifi/python-certifi#20. I don't want to merge it yet, but it would be good if people would play about with it. |
|
Can everyone affected in this thread please download the latest certifi bundle and confirm that it works/does not work with your particular problem website? Absent any feedback from you, that bundle will go into the next minor release of requests after 2.7.0, so it's in your best interests to confirm that the bundle functions appropriately for you. |
|
What's the best way to do that? If I install certifi will requests use that instead of its built-in bundle? |
|
Yes it will.
|
|
OK, I cloned the master of |
|
This actual bug is fixed, so we don't need to keep this issue open. The longer term plan can be dealt with elsewhere. |
|
For what it's worth, I was getting this intermittently on one of two hosts. Difference was OpenSSL version. OpenSSL v1.0.1m failed intermittently. Upgrading to v1.0.2e (and recompiling Python v3.4.3 (this is Gentoo, we compile EVERYTHING!)) made the problem go away. |
I'm having a somewhat odd issue. A get() request seems to work fine with requests-2.5.1, but after upgrading to requests 2.5.2, the same URL leads to
CERTIFICATE_VERIFY_FAILED. cc @mpharriganThe text was updated successfully, but these errors were encountered: