page.setExtraHTTPHeaders cause 'ERR_INVALID_ARGUMENT' error

[andrew0213]

Steps to reproduce

Tell us about your environment:

• Puppeteer version: 1.20.0
• Platform / OS version: mac OS
• URLs (if applicable):
• Node.js version: v11.11.0

What steps will reproduce the problem?

const puppeteer = require('puppeteer');
(async () => {
    const browser = await puppeteer.launch({
        headless: false,
        ignoreHTTPSErrors: true,
        ignoreDefaultArgs: ["--enable-automation"],
        args: [
            '–no-sandbox',
        ]
    });
    const page = await browser.newPage();
    await page.setExtraHTTPHeaders(
        {'proxy-authorization': 'test'}
    );
    await page.goto('http://example.com/');
    await page.waitFor(10000);
    await browser.close();
})();

What is the expected result?
go to the url success

What happens instead?

(node:86672) UnhandledPromiseRejectionWarning: Error: net::ERR_INVALID_ARGUMENT at http://example.com/
    at navigate (/Users/andrew/Documents/otherProject/vs/demo/node_modules/puppeteer/lib/FrameManager.js:120:37)
    at processTicksAndRejections (internal/process/next_tick.js:81:5)
  -- ASYNC --
    at Frame.<anonymous> (/Users/andrew/Documents/otherProject/vs/demo/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.goto (/Users/andrew/Documents/otherProject/vs/demo/node_modules/puppeteer/lib/Page.js:674:49)
    at Page.<anonymous> (/Users/andrew/Documents/otherProject/vs/demo/node_modules/puppeteer/lib/helper.js:112:23)
    at /Users/andrew/Documents/otherProject/vs/demo/demo.js:16:16
    at processTicksAndRejections (internal/process/next_tick.js:81:5)

But, if I replace header proxy-authorization to proxyauthorization, it's fine.
So,it seems that it's not allowed containing '-' in header, is this corrent? Or is it an issue?

[danielabel]

I wonder if setting (resetting?) some headers is bad?

https://github.com/chromium/chromium/blob/b2254b38369f4f8ce8685b7a82dee1251f6cea22/services/network/url_loader.cc#L679-L683

https://github.com/chromium/chromium/blob/3611052c055897e5ebbc5b73ea295092e0c20141/services/network/public/cpp/header_util_unittest.cc#L50

[j-bruce]

fwiw, we started seeing this error when we upgraded to puppeteer 2.0.0. And like @danielabel said, it turned out to be because of a header we'd always been sending (Cookie2) making chrome barf. The fix was just to stop sending that header.

[agarzola]

We’re getting the same ERR_INVALID_ARGUMENT error when setting a Host header via page.setExtraHTTPHeaders() and loading http://localhost/path/to/tested/page. When I curl it, I get the response I expect:

curl -H "Host: [REDACTED]" http://localhost/path/to/tested/page

Using v2.0.0.

[sorrell]

Agreed with @agarzola, the Host header causes problems when I set it explicitly.

[sdeprez]

FYI we've also encountered this error with Puppeteer 2.0 (not before) and when overriding header Keep-Alive. The unit test link provided by @danielabel https://github.com/chromium/chromium/blob/3611052c055897e5ebbc5b73ea295092e0c20141/services/network/public/cpp/header_util_unittest.cc#L50 lists some unsafe header name / value (from what I understand) but it'd be nice to have an exhaustive list! It looks like it can depend of the header value too.

[squatica]

It seems that some headers cannot be set using setExtraHTTPHeaders. "Proxy-" cannot be set probably because that's a browser-wide setting and should be done by

puppeteer.launch({
    args:[ '--proxy-server=http://example.com' ]
});

others are Hop-by-hop headers that only make sense per-request https://www.freesoft.org/CIE/RFC/2068/143.htm (setExtraHTTPHeaders works per-tab and influences for example all XHR requests from that tab too).

But it would be nice if Puppeteer returned a more helpful error message than just net::ERR_INVALID_ARGUMENT

[mastershaig]

same issue

[JialieY]

same issue here, when I call
page.setExtraHTTPHeaders({ 'host': 'www.amazon.com' });
I saw net::ERR_INVALID_ARGUMENT issue

[JialieY]

We’re getting the same ERR_INVALID_ARGUMENT error when setting a Host header via page.setExtraHTTPHeaders() and loading http://localhost/path/to/tested/page. When I curl it, I get the response I expect:

curl -H "Host: [REDACTED]" http://localhost/path/to/tested/page

Using v2.0.0.

Is there any workaround?

[mastershaig]

If you want to pass proxy-authorization then use below code instead of page.setExtraHTTPHeaders():

await page.authenticate({username:'', password:''})

[JialieY]

If you want to pass proxy-authorization then use below code instead of page.setExtraHTTPHeaders():

await page.authenticate({username:'', password:''})

Thanks for replying me. Actually, I want to get workaround for page.setExtraHTTPHeaders({ 'host': 'www.amazon.com' });

[hahanov]

Fix?

[JialieY]

I removed it, and then everything works

[hahanov]

I meant an error in principle, I think it is wrong that some arguments cannot be passed.

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[stale]

We are closing this issue. If the issue still persists in the latest version of Puppeteer, please reopen the issue and update the description. We will try our best to accomodate it!