Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary SOCKS proxy support #690

Closed
shazow opened this issue Aug 7, 2015 · 26 comments

Comments

Projects
None yet
@shazow
Copy link
Member

commented Aug 7, 2015

We've had some attempts to get this going over the years, this is the umbrella issue tracking these attempts and the goal is to get somekind of baseline support for SOCKS proxies into the urllib3 before we aspire towards more advanced features.

Related attempts (should be a great reference if anyone wants to tackle this):

  • #68 (original attempt using socksipy; outdated code)
  • #284 (another attempt using pysocks—a more up-to-date fork of socksipy; outdated code)
  • #486 (most recent attempt, refactored version of pysocks attempt)
  • #692 (another recent attempt, but mixes pysocks logic into the connectionpools)
  • #630 (recipe for wrapping pysocks)
  • #762 (@Lukasa)

Suggestions:

  • SOCKS proxy support should ideally be built into ProxyManager (or a similar manager if combining is not possible).
  • Should use pysocks (unless there's anything newer?). Avoid vendoring if possible, we can enable support when pysocks is installed (also include a urllib3[socks] install tag which will pull in the dependency).

To complete this issue, the feature needs to be documented and have full test coverage so that it can be merged into urllib3 core (or at least contrib).

@JeremyRand

This comment has been minimized.

Copy link

commented Aug 7, 2015

As mentioned in #486 (comment) , I can put $100 toward this via BountySource. If I do so, BountySource would release the funds when the GitHub issue in question is closed -- given that, is this issue the correct one to put the bounty on?

@shazow

This comment has been minimized.

Copy link
Member Author

commented Aug 7, 2015

@JeremyRand Yes, I think this would be best. Keep in mind, this issue is just for baseline SOCKS support—it doesn't necessarily include your specific requirements, though I suspect all those things should Just Work by default.

@shazow

This comment has been minimized.

Copy link
Member Author

commented Aug 7, 2015

Ping to everyone who expressed interest in SOCKS support for urllib3, we're going to try raise a bounty for this issue here, please contribute: https://www.bountysource.com/issues/25474088-preliminary-socks-proxy-support

@Ofekmeister @zhao-ji @bpowers @laike9m @eordano @YS- @9b @ctoth @aleks-mariusz @josegonzalez @fmoo @itdaniher @cattleyavns @a3nm @jbrockmendel — I hope some of you will to the bounty, and maybe ask your employer if they'd be willing to help with this too?

And of course, contributors with prior attempts — @Anorov @schlamar @foxx

Ideally one of our original contributors will claim this. If we can get enough funding for at least a week of full-time work, I'll be willing to take this issue on myself.

@shazow shazow added the Bounty label Aug 7, 2015

@JeremyRand

This comment has been minimized.

Copy link

commented Aug 7, 2015

I just sent $50 via the Namecoin BountySource account; I'm in the process of buying some more BTC to fund my BountySource account, once I've done that I'll send the other $50.

@shazow

This comment has been minimized.

Copy link
Member Author

commented Aug 7, 2015

@JeremyRand Thank you, much appreciated! 🍮

@Lukasa

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2015

Depending on how my time pans out, I may consider taking this one on. It's annoying me now. ;)

@zhao-ji

This comment has been minimized.

Copy link

commented Aug 9, 2015

I've writen a socks5 proxy, i'll try it!

@Anorov

This comment has been minimized.

Copy link

commented Aug 9, 2015

I have made a PR at #692. No tests have been added yet, but functionality is in place.

@JeremyRand

This comment has been minimized.

Copy link

commented Aug 16, 2015

Remaining $50 posted as promised.

@jbrockmendel

This comment has been minimized.

Copy link

commented Aug 16, 2015

I'm not likely to be helpful on the coding front, but have a handful of socks5 proxies set up that I'm happy to run tests on.

@shazow

This comment has been minimized.

Copy link
Member Author

commented Aug 16, 2015

Thanks. :) If anyone is up for testing out #692 and giving feedback, that'd be great.

@AlJohri

This comment has been minimized.

Copy link

commented Dec 3, 2015

Just added a little bit to the bounty. Thanks for working on this!

@shazow

This comment has been minimized.

Copy link
Member Author

commented Dec 3, 2015

@AlJohri Thanks! If you'd like to get involved, pulling the code mention above and providing feedback would be most helpful. :)

@AlJohri

This comment has been minimized.

Copy link

commented Dec 3, 2015

I'll try to give it a shot in the near future, this is definitely a feature I'd want to invest in. I see that @lorien tried the branch and got some errors? #692

@lorien

This comment has been minimized.

Copy link

commented Dec 4, 2015

Hey guys, I've added +$50 for this bounty. Please, anybody, make socks support :)

@shazow

This comment has been minimized.

Copy link
Member Author

commented Dec 14, 2015

Thanks for everyone providing bounties. We're currently at $415. I set a goal for $1200, if it breaks that then I'll give it a shot myself. :)

@Lukasa

This comment has been minimized.

Copy link
Contributor

commented Dec 15, 2015

Ok, so I'm planning to move this forward myself now. I've got some initial work in #762, and I'm going to try to press this forwards over the next few days to try to get this ready.

@kislyuk

This comment has been minimized.

Copy link

commented Dec 15, 2015

👍 :shipit:

@anatasiajp

This comment has been minimized.

Copy link

commented Dec 15, 2015

Love you so much, Lukasa! ~~~♥

@shazow

This comment has been minimized.

Copy link
Member Author

commented Dec 16, 2015

Hey everyone, @Lukasa is making great progress in #762. Please take a look and provide last-minute feedback before we merge. :)

@shazow

This comment has been minimized.

Copy link
Member Author

commented Dec 19, 2015

Alright, it's merged into master, will be included in the next release (v1.14) some weeks later. Big thanks to @Lukasa who will be collecting the bounty, wee.

@shazow shazow closed this Dec 19, 2015

@Lukasa

This comment has been minimized.

Copy link
Contributor

commented Dec 19, 2015

\o/ 🍮 🍰 🍪

@JeremyRand

This comment has been minimized.

Copy link

commented Dec 19, 2015

Awesome work @Lukasa , thank you for getting this done! I will approve the collection of the bounty as soon as BountySource tech support gets back to me on why I don't see a button for doing so. (I just sent them an email; my guess is that they'll get back to me on Monday.) Cheers!

@JeremyRand

This comment has been minimized.

Copy link

commented Dec 29, 2015

BountySource hasn't yet gotten back to me; I assume this is because they've been away for holidays. Hopefully they'll figure out why I don't see the approve button soon; if not, it looks like the bounty will be automatically awarded on Jan 2.

@ofek

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2015

So, is SOCKS fully supported now? HTTPS, IPv6, etc.? If so, does anyone know how soon it will be incorporated in the Requests library?

@Lukasa

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2015

@Ofekmeister HTTPS is fully supported. IPv6 is not, due to limitations in PySocks, but relatively little work is required to add that support and no changes need to be made in urllib3 to take advantage of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.