urllib2 cannnot handle https and BasicAuth via Proxy.

[masatokawamura]

BPO	17518
Nosy	@orsenthil

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-03-22.11:18:20.010>
labels = ['type-bug', 'library']
title = 'urllib2 cannnot handle https and BasicAuth via Proxy.'
updated_at = <Date 2018-07-11.10:16:06.786>
user = 'https://bugs.python.org/masatokawamura'

bugs.python.org fields:

activity = <Date 2018-07-11.10:16:06.786>
actor = 'BreamoreBoy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-03-22.11:18:20.010>
creator = 'masato kawamura'
dependencies = []
files = []
hgrepos = []
issue_num = 17518
keywords = []
message_count = 2.0
messages = ['184958', '228408']
nosy_count = 2.0
nosy_names = ['orsenthil', 'masato kawamura']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17518'
versions = ['Python 2.7']

[masatokawamura]

When urllib2 module is used, https connections to servers that BasicAuth is required, via proxies, can not be established properly.

Sample code:
import urllib2

def main():
    url = "https://example.com/aplication/rpc"
    
    proxy_support = urllib2.ProxyHandler({'http': 'http://proxy.server.com:8080/','https': 'http://proxy.server.com:8080/'})

    pass_mngr = urllib2.HTTPPasswordMgrWithDefaultRealm()
    pass_mngr.add_password(None, url, 'user', 'password')
    auth_handler = urllib2.HTTPBasicAuthHandler(pass_mngr)
    
    opener = urllib2.build_opener(proxy_support,auth_handler)
    req = urllib2.Request(url)
    f = opener.open(req)
    print f.read()

"opener.open" method throws an exception.

I found a similar case at http://bugs.python.org/issue7291. However, this issue indicates a case of authentication at a proxy server. In my case, the origin server uses basic authentication.

The origin server address is not set in Request URI in the https request in case of using proxy servers, because https request gets encripted with SSL when it passes through proxies.

urllib2 works well for the first on-going request and in-comming return. However, after the first in-comming return (401 code), urllib2 module sends a worng request that includes the origin server URL.

I suggest modification for set_proxy method in Request class.

Original code:
def set_proxy(self, host, type):
if self.type == 'https' and not self._tunnel_host:
self._tunnel_host = self.host
else:
self.type = type
self.__r_host = self.__original

        self.host = host

As a value is already set to tunnel_host when the second out-going request is sent, "self.type == 'https' and not self._tunnel_host" is false.

Code suggested:
def set_proxy(self, host, type):
if self.type == 'https' and not self._tunnel_host:
self._tunnel_host = self.host
else:
self.type = type
if self.type != 'https':
self.__r_host = self.__original

        self.host = host

regards.

[BreamoreBoy]

Slipped under the radar? Note the patch suggested in msg184958.