Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change proxy settings after the Webdriver has been created #1425

Open
sudiptomajumdar opened this issue May 25, 2019 · 0 comments
Open

Change proxy settings after the Webdriver has been created #1425

sudiptomajumdar opened this issue May 25, 2019 · 0 comments

Comments

@sudiptomajumdar
Copy link

Feature Proposal
Currently the proxy settings can be supplied in the Options object while creating a Webdriver. Once created, the proxy settings of that Webdriver cannot be altered. The only way to obtain a Webdriver with a different proxy settings is to discard the existing Webdriver and create a new one.

The proposal is to expose proxy settings of a Webdriver in the method - Webdriver.manage()

When the proxy settings have been changed from the Webdriver.manage() method, any subsequent Webdriver.get() call will use the modified proxy settings.

Motivation
Certain sites have very low rate limit threshold for requests from a single IP address because they are aggressive in their effort to prevent attack as well as spider bots. Writing an automation to scan for even a few hundred pages on such sites become a challenge especially when they ban IPs. The solution is to use an HTTP proxy to distribute the requests across multiple IP addresses.
To mitigate the rate limit problem, every webdriver.get() request needs to be made from a different proxy. Currently the only was to do that is to discard the webdriver after each webdriver.get() request and create a new driver.
This is inefficient and slow, but the bigger problem is that Webdriver instances do not always close gracefully. If one runs the automation for some time, one notices a lot of zombie Webdriver instances hanging around if one uses the Unix command "ps -ef". This causes memory leaks.

Example
Proxy proxy = webdriver.manage().getProxy();
proxy.setHttpProxy("New proxy server to use");
webdriver.get("New URL will use the new proxy server");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant