puppeteer 1.11.0 ignore --proxy-server for localhost URLs

[panthony]

Steps to reproduce

Tell us about your environment:

• Puppeteer version: v1.11.0
• Platform / OS version: macOS High Sierra
• URLs (if applicable): None
• Node.js version: v8.6.0

What steps will reproduce the problem?

1. start a proxy on localhost on 8081
2. start browserless with --proxy-server=localhost:8081
3. use goto with an URL on localhost

The request will bypass the configured proxy although it worked up until v1.10.0.

If you use an url on some domain (ex: google.com) then the proxy will be used.

const puppeteer = require('puppeteer');
const { setupServer } = require('./test/helper');

(async () => {
  const proxy = await setupServer(8081);
  const server = await setupServer(80);

  const browser = await puppeteer.launch({
    args: [
      '--proxy-server=localhost:8081',
    ]
  });

  const page = await browser.newPage();

  try {
    const response = await page.goto('http://localhost/page.html');
    console.log('res', response.status());
  } catch (e) {
    console.log('err', e.message);
  }

  await browser.close();
  await proxy.stop();
  await server.stop();
})();

The setupServer was copied from puppeteer tests, I only added this:

  requests() {
    return this._requests;
  }

    reset() {
    this._requests = [];
    // rest of code
  }

  _onRequest(request, response) {
    this._requests.push(request.url);
    // rest of code
  }

For my own tests to verify that the request goes through the proxy I started which looks like:

  describe('HTTP proxy', () => {
    it('uses configured HTTP proxy', async () => {
      const proxy = await setupServer(8081);
      try {
        await fetch({ url: server.EMPTY_PAGE, http_proxy: 'localhost:8081' });
        expect(proxy.requests()).to.include(server.EMPTY_PAGE);
      } finally {
        await proxy.stop();
      }
    });
  });

What is the expected result?

• The request to go trough the configured proxy

What happens instead?

• The proxy is ignored

[HassyMasa]

Additional this option?
--proxy-bypass-list

  const browser = await puppeteer.launch({
    args: [
      '--proxy-server=localhost:8081',
      '--proxy-bypass-list=localhost',
    ]
  });

https://peter.sh/experiments/chromium-command-line-switches/#proxy-bypass-list

[panthony]

@HassyMasa The issue is that localhost is ignored with v1.11.0 whereas it was not until then.

I'm not trying to bypass the proxy for localhost.

Didn't know this option, tried --proxy-bypass-list="" in an attempt to override a possible new default I missed but with no effect

[HassyMasa]

@panthony I'm sorry I can't be of any help.

[sethfowler]

@panthony If you're on Chrome 72, you may be hitting the problem discussed in this Cypress bug report.

Apparently --proxy-bypass-list=<-loopback> will fix the issue.

[panthony]

@sethfowler It does! Awesome, thanks for your help 🙇

@aslushnikov Hi, do you think this option could be added by default in puppeteer as, I think, working with localhost is a common use case?

Or maybe add something somewhere, maybe in troubleshooting (although I'm just discovering this page)?

[aslushnikov]

@panthony @sethfowler awesome, thanks for the investigation!

Or maybe add something somewhere, maybe in troubleshooting (although I'm just discovering this page)?

I've added this option to the //examples/proxy.js - should be good enough for now.