https under proxy doesn't work

[Eric-Cui]

The current Requests implementation seems not to work for https request under proxy.

I have posted a question on stackoverflow:

http://stackoverflow.com/questions/13005301/https-proxy-support-in-python-requests-library

[sigmavirus24]

This is a repeat of #881. The problem actually exists in urllib3 for which there is a pull request pending and is related to urllib3/urllib3#50 and urllib3/urllib3#68

[Eric-Cui]

Thanks for the reply. So it seems to be resolved in the latest release. But why it still exist in my installation, which is already latest release. My environment is different in that it has 2 levels proxy, one is the ISA server in my corporation, which needs the Windows domain account and password. The other is set by me on my workstation using the free proxy software ntlmaps, which intends to automatically answer the challenges requested the ISA server. Will this cause any difference?

[sigmavirus24]

It isn't resolved. #881 was more specifically about non-HTTPS proxies but I noted in there that it applied to HTTPS Proxies indirectly (I'll go edit that comment now). The fix is still being worked on in the two urllib3 issues I mentioned. Once those are merged/resolved (and the next version of urllib3 is vendored into requests) then it will be resolved in requests as well. As for having two proxy layers, someone else will have to answer that.

[exhuma]

We are experiencing the same problem. We are writing an application making requests to the RIPE database, which run on SSL. I tried a solution with urllib2 which is working for now. So we'll be using this until this issue is resolved.

[aleks-mariusz]

I have read through a ton of links on github about this urllib3 issue, and all i see instead are a lot of references to newly added (or soon to be) socks support (which i don't need).

But when is THIS ISSUE going to actually be fixed (in urllib3 at least, so i can swap it into requests's 3rd party packages manually if need be). i spent all day coding up some bad ass patch syncer that relies on https (but has to be via a http proxy due to my corp network), only to be thrwarted when it came time to test and all i get are http-400 codes because the proxy isn't reached using CONNECT. quite frustrated right now.. :-\

[dahpgjgamgan]

@cynikalX - you can use foxx branch of requests - last time I checked, https through proxy worked there (the only trick is that after cloning his repository you have to switch to the branch on which he did the changes).

Another option is using an old requests branch that works on top of urllib2. Last option is to use some mitm-capable proxy like fiddler, which is aware that is has to communicate with other proxies and does the HTTP CONNECT.

I also agree that HTTPS and SOCKS proxy support could be introduced separately.

[exhuma]

@dahpgjgamgan: I looked at the repo, and it seems that foxx's repository is actually far behind of kennethreitz's. If I get the time, I'll test if it works. For now, we have re-written our code to use urllib2. So it's not very pressing for us.

Additionally, one can clone a specific branch with the '-b' flag:

git clone -b <banch_name> <url>

I suppose the one in question is develop?

[aleks-mariusz]

@dahpgjgamgan Ahh yes i ended up manually applying the patches one by one by hand before i realized i could just clone his branch (i'm a bit of a git newb). It ended up working swimingly and i'm using a local copy of requests bundled with my project. I would really prefer that the support/fixes be merged into requests's master branch and a new release was cut so i can get my org to sign off on that instead of having to incl an extra library to support (like requests is doing with urllib3). I didn't want to go the urllib2 route as i need concurrency/keep alives to use grequests

[machinae]

Can anyone submit a patch to fix this? This is a urllib3 issue. I switched to Requests 0.7.6(the last version to use urllib2), and https requests through proxy work fine.

[sigmavirus24]

@machinae two things:

First, @kennethreitz is working on a rewrite using transport adapters which should alleviate or fix this issue.

Second, if you want the patch at shadow/urllib3 to get through so it can be used here, go and work on that issue. It's already known to be am issue that affects plenty of people but complaining won't help anyone. I'm sure @shazow would love help on that pull request but until he gets some, it isn't going anywhere fast.

[Lukasa]

@sigmavirus24: +1.

The problem is in urllib3, not Requests. I'm only leaving the issue open to attempt to prevent too many people opening duplicates. Problems in urllib3 should be raised against and resolved in urllib3.

[machinae]

@Lukasa I was under the impression this was already fixed in urllib3 as per urllib3/urllib3#68 (comment) and Requests just needs to be updated with the latest version of urllib3.

[sigmavirus24]

That's quite wrong but understandable

Ilya Lichtenstein notifications@github.com wrote:

@Lukasa I was under the impression this was already fixed in urllib3 as
per urllib3/urllib3#68 (comment) and
Requests just needs to be updated with the latest version of urllib3.

---

Reply to this email directly or view it on GitHub:
https://github.com/kennethreitz/requests/issues/905#issuecomment-10781919

[tcorbettclark]

I've just added some details on http://stackoverflow.com/questions/14390605/python-requests-ssl-issue which may be useful. I'm available for testing any fixes against our proxy...