HTTPS requests fail through a proxy #50

Closed
bhendo opened this Issue Feb 23, 2012 · 9 comments

Comments

Projects
None yet
5 participants

bhendo commented Feb 23, 2012

Requests to access a secure website (SSL/TLS) fail through a proxy.
Urllib3 does not properly implement the HTTP CONNECT method.

For example the following code should print 200.
Instead, with a burp proxy, it prints 502.

import urllib3

proxy = urllib3.proxy_from_url('http://localhost:8080/'
response = proxy.urlopen('GET', 'https://www.google.com/index.html')

print response.status

bhendo commented Feb 23, 2012

import urllib3
import unittest

class TestProxyConnect(unittest.TestCase):
    def setUp(self):
        self.proxy_url='http://localhost:8080/'
        self.https_url='https://www.google.com/index.html'
    def test_connect(self):
        proxy=urllib3.proxy_from_url(self.proxy_url)
        res=proxy.urlopen('GET', self.https_url)
        self.assertEqual(res.status, 200)

if __name__ == '__main__':
    unittest.main()
Owner

shazow commented Feb 26, 2012

Thank you for porting the report and adding the test, @khaosx. We'll have to port the test to one that doesn't depend on external services at some point.

I've been thinking about this bug and sadly I suspect it won't be trivial given the simplicity of the ProxyManager implementation right now. I think the best thing to do is to block this one on #44 which will require some added complexity for ProxyManager on its own and we can sneak this fix in there somehow.

If you'd like to take a crack at fixing this one yourself, you're more than welcome. :-)

bhendo commented Mar 2, 2012

I came up with a fix that works for me. If you want to test it and add it please do.
I forked your code to post the fixes

https://github.com/khaosx/urllib3

bhendo commented Mar 3, 2012

Following redirects through a connect tunnel is not thread safe at the moment.

After additional testing it probably makes sense to create an subclass of httpsconnectionpool and verifiedhttpsconnection and override a few methods instead of adding the fix right into connectionpool.py.

For now the new subclass of httpsconnectionpool should not follow redirects.

Owner

shazow commented Mar 3, 2012

@khaosx Thanks for diving into this.

I took a quick look. Looks like you're trying to fix Issue #8 as well. That's a tricky one because there are cases where you do want to pass in the host. I'll post some feedback in comments.

I presume you're going to continue iterating on this, let me know when you think this is ready to be merged and I'll take a look and give more feedback.

foxx commented Mar 15, 2012

@khaosx If you're interested, please see discussions on:
kennethreitz/requests#478

We've implemented HTTP/HTTPS/SOCKS proxy support into Python Requests (which contains urllib3 modifications).

Collaborator

sigmavirus24 commented Oct 8, 2012

So for reference's sake, seems like #68 is work towards this?

Contributor

schlamar commented Jan 27, 2015

FYI, this was done in #170 :)

Owner

shazow commented Jan 27, 2015

@schlamar Thanks! :)

@shazow shazow closed this Jan 27, 2015

@kellerb kellerb referenced this issue in irods/irods May 5, 2015

Closed

Schema validation failing on CentOS 6 #2673

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment