Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add proxy #78

Open
zhajingwen opened this issue Jan 7, 2021 · 4 comments
Open

add proxy #78

zhajingwen opened this issue Jan 7, 2021 · 4 comments
Assignees

Comments

@zhajingwen
Copy link

should support us to use proxy

@gabrielfiorelli
Copy link

I can be wrong, but i'm just trying to help.

@JoMingyu codes uses urlopen from urllib to open the web page. In the following question of stackoverflow there's an example on how to allow proxies on those calls. But, maybe there's other problems that I don't know. Maybe no one is having ip ban for scraping in the first place.

https://stackoverflow.com/questions/3168171/how-can-i-open-a-website-with-urllib-via-proxy-in-python

@kluhan
Copy link
Collaborator

kluhan commented Sep 14, 2022

@gabrielfiorelli thank you for your research, I will look into this in the near future. But I can't guarantee that it will be included in the next update.

@uppy1985
Copy link

Looking at the link that @gabrielfiorelli posted I found the answer to my proxy issue in this response. Simply added this to my code and now I have no issues using this package at work.

`import urllib.request

proxy_support = urllib.request.ProxyHandler({'http' : 'http://user:pass@server:port',
'https': 'https://...'})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)`

@JoMingyu
Copy link
Owner

JoMingyu commented Jan 3, 2023

It's a little late, but I'll update it by this January.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants