-
Notifications
You must be signed in to change notification settings - Fork 308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proxies #368
Comments
you can pass in browser configs via puppeteerOptions.
|
+1, I also would like to use different proxy to each browser instance. @Yannicko have you found a solution? |
I found the solution literally a few searches later after I posted the comment. Anyway anyone else looking for a solution use this: proxy-per-page |
Somehow it doesn't work for me :/ |
I tried |
I experience the same behavior. Have you found a solution to this yet? Edit: First of all I created a new Concurrency by copying the class BrowserProxy extends ConcurrencyImplementation_1.default {
...
let page;
let context; // puppeteer typings are old...
const proxyServer = options.args.find(arg => arg.includes('--proxy-server=')).split('--proxy-server=')[1] || null;
const contextOptions = {proxyServer: proxyServer ? proxyServer : null};
return {
jobInstance: () => __awaiter(this, void 0, void 0, function* () {
... If so the proxy-server value will be saved and provided to the ...
jobInstance: () => __awaiter(this, void 0, void 0, function* () {
yield util_1.timeoutExecute(BROWSER_TIMEOUT, (() => __awaiter(this, void 0, void 0, function* () {
context = yield chrome.createIncognitoBrowserContext(contextOptions);
page = yield context.newPage();
}))());
return {
... After that make changes to all the Concurrency files so your Concurrency can be used by puppeteer-cluster like this: const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_BROWSERPROXY,
maxConcurrency: 1,
timeout: properties.taskTimeout,
puppeteerOptions: {
headless: false,
ignoreHTTPSErrors: true,
args: [
`--proxy-server=${proxy_server}`,
'--no-sandbox',
]
},
puppeteer: puppeteer,
monitor: false,
retryLimit: 3,
retryDelay: 3500
}); There is probably a better way to handle that, but this was my first approach in fixing this issue. |
@cedricdsc I'm sorry for the delayed response, just got a chance to reply to your question. I did something similar to you but a little different, here's my solution: I created a proxyServer variable with the proxy server for this instance: and i changed context to: |
That's another way to do it. Good you found it too 👍 |
Hi all, I have found a solution for those who may have been struggling with a lack of proxy support per request or per browser in I hope this solution helps others in a similar situation. Please see the example code below for implementation details. proxies.json [
"xxx.xxx.xxx.xxx:xxxxx",
"xxx.xxx.xxx.xxx:xxxxx",
"xxx.xxx.xxx.xxx:xxxxx",
"xxx.xxx.xxx.xxx:xxxxx",
"xxx.xxx.xxx.xxx:xxxxx",
"xxx.xxx.xxx.xxx:xxxxx"
] index.ts import { Cluster } from 'puppeteer-cluster';
import ProxyList from '../proxies.json';
import useProxy from 'puppeteer-page-proxy';
(async () => {
const cluster = await Cluster.launch({
concurrency: Cluster.CONCURRENCY_CONTEXT,
maxConcurrency: 2,
monitor: true,
puppeteerOptions: {
headless: false
}
});
await cluster.task(async ({page, data: url}) => {
await useProxy(page, `direct://${getProxy()}`);
await page.goto(url);
});
cluster.queue('https://ipinfo.io');
cluster.queue('https://ipinfo.io');
await cluster.idle();
await cluster.close();
})();
function getProxy() {
return ProxyList[Math.floor(Math.random() * ProxyList.length)];
} |
I have implemented proxy support in my forked version, available at: https://github.com/joone/headless-cluster. |
Is it possible to use different proxies per browser instance?
I want to create something so that every instance has a different proxy through which the browser will retrieve information.
The text was updated successfully, but these errors were encountered: