-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Twisted relies on OpenSSL to configure trust roots, resulting in an empty trust store by default on Windows and non-functional HTTPS in treq #94
Comments
Decided to test with Twisted 14.0.2, seeing as there are some build failures on Travis with the latest Twisted, and..
|
Same error here using this code:
Windows 7 x64 It works on my Debian box, but that's on twisted 14 and treq 0.2.1. Will update this post with newer version once I've tested it, unless @gdude2002 can. |
Can confirm, also works for me on my Ubuntu box using your code. Python 2.7.9 (x64) |
Did some further testing.. I attempted to compile OpenSSL with MingW but after several failed attempts, more than one wasted hour, and tons of bad makefile output, I gave up and downloaded a pre-build OpenSSL for x64 Windows. As PyOpenSSL uses the location of Removing the old versions from my PATH and reinstalling PyOpenSSL in the hopes that this would fix the issue were futile however, and the problem still continues. |
Hi @rakiru - I tried running your script with your documented versions of treq, pyOpenSSL, and Twisted on OS X, and it just prints |
@gdude2002 @rakiru - I have recently heard about some issues with Twisted's HTTPS client failing with a very specific patch-level of OpenSSL. Can you report exact OpenSSL versions (including not just the version reported by |
I am trying to reproduce in a windows environment with the openssl bundled with cryptography but my windows development VM is not cooperating ;) |
@glyph We don't have any other problems with OpenSSL in Python - Even generating and verifying keys with it works just fine. It does appear to be a Windows-only issue, but I note that stuff like |
|
Yeah, I figured that that's what it was doing - Although now that I think of it, I'm not sure what requests actually uses for SSL verification. Might be something to look into I guess. Well, what might be different about how Twisted does this stuff? We are using some of the SSL stuff in other parts of our app, but nothing else explicitly does (or needs) verification right now, I guess. |
Are those other parts of your app using |
No, it's mostly SSL sockets via |
The "standard" context factory? Do you mean |
After saying this, I realise I should have bumped up the priority of a cert verification option... Maybe I'll look into how it works here once we figure out the issue. It was never something that seemed prominently mentioned in any docs, probably because there's no high-level context factory implementation to do all this for you in twisted? Neither of our usages are HTTP related either. Edit: Sorry for going a bit off topic. I decided it was a good idea to check my email right before bed... |
It's OK to go off-topic; the more places we can get the message of " The place in the documentation where I hope this is "prominently mentioned" enough (go go gadget PageRank) is the Twisted TLS HOWTO. |
OK. I can reproduce the issue now, and I am surprised we didn't get a bug report until now. It's a bug in Twisted, sort of; more like, the default configuration of Cryptography on Windows. There is a workaround. The bug is that Twisted does not include a set of trust roots of its own, or automatically interface with any other certificate bundles, such as The workaround is to (A) install If you're using bash (say, from git for windows) then this would be $ pip install certifi
$ SSL_CERT_FILE="$(python -m certifi)" python my_application.py or, in cmd / batch script, it would be this somewhat obscurer snippet: C:\...> pip install certifi
C:\...> for /f %i in ('python -m certifi') do set SSL_CERT_FILE=%i
C:\...> python my_application.py |
@glyph Is this something we could ship with installs of our app for all platforms, or should it only be part of the Windows version? |
@glyph Thanks for the link. Google wasn't in the mood to give me that last night (although I think I might have got an older version where ctrl-f "verify" had no results). |
@gdude2002 It's really up to you. I would personally ship it just with the Windows version, because it is better to trust the platform-configured (and, critically, platform-updated) trust roots than to have to ship security updates every time the bundle in |
So, this isn't really a treq bug, but I am not going to close it until we have some kind of resolution from Twisted because it is still a terrible experience from treq. |
Appreciated; I'm kind of surprised that Twisted has had this problem for all this time, to be honest. |
@gdude2002 Me too. I think the reason is that not a lot of people use Windows and not a lot of people configure SSL properly :-. |
@glyph It seems that twisted's networking stuff is kind of lacking in other areas in relation to Windows - but you've seen this question before ;) Came across this one while we were also working with async DNS, and.. yeah. |
@gdude2002 did you ever file that ticket against Twisted? |
@glyph Twisted issue tracker is scary D: I could I guess, I only really use GitHub though |
@gdude2002 Not that scary - you don't have to submit the patch, just log in and type the description of the bug :) |
Maybe I just like GitHub too much. This feels archaic and like the admins are leaning over your shoulder a bit. :P Dunno if I filled everything out correctly, but here it is I guess. |
I meant for the SO question you cited above. |
@glyph Ah, whoops. No, it's a relatively old SO question that you appear to have commented on yourself, and it's not one of mine - I'd assumed there was already a ticket. |
Yeah, I don't see a ticket - I'll report this one as well then, unless you're already typing one up. |
Nope - thanks! |
Add codecov badge to README.rst
I've tried to summarize the current state of things here in the Twisted tracker, since many of the original issues are closed with only part of their scope being resolved. |
Not sure if this is my setup, but I've just done a fresh install of Python 2.7.10 x64 and installed Twisted and Treq (and the required libs) through pip, using MingW to compile where appropriate.
Let's say I'm attempting to access
https://google.com
:My errback gets called with an error of type
ResponseNeverReceived
, and iterating overerror.value.reasons
gives me the following Failure:I've tried a bunch of stuff, including trusting the cacerts.pem provided by cURL in my OS's trust store, installing
idna
andservice_identity
and reinstalling OpenSSL. Any ideas on this?System info:
The text was updated successfully, but these errors were encountered: