extraHeaders doesn't work when using chrome started with --headless

[zackiles]

I'm using chrome-launcher in node to start a chrome instance with the chrome flag --headless set. I'm then using the npm module version of lighthouse and passing it this chrome instance's port to use. When setting extraHeaders it will only work if I don't pass the --headless option to chrome launcher. If I remove it, the headers are sent fine to the server, if I enable it my headers are missing even though the headers show fine in the report.json under runtimeConfig.extraHeaders.

What further confuses me, is if I don't use the lighthouse npm module, and instead use the cli (bypassing the need for chrome launcher) and I pass the --extra-headers argument it also works fine and sends the headers correctly.

I'm using the current version of lighthouse.

[patrickhulce]

To clarify, everything works if you use extra headers and headless from the CLI? It's possible you've discovered a headless parity issue if this isn't the case.

[zackiles]

@patrickhulce when I use the lighthouse CLI I'm running lighthouse www.domain.com --extra-headers "..some headers...". Which works fine and sets the headers for requests.

When I use lighthouse as an npm.module I'm using the npm package chrome-launcher to start chrome with the --headless flag and passing the chrome port that's returned to lighthouse along with extraHeaders. In this latter case the headers are not being sent even though they'll show in the lighthouse Json report under runtime config.

[patrickhulce]

I got all that, what I was trying to ask was if everything works as expected if you do lighthouse www.domain.com --extra-headers='{cookie: "foo"}' --chrome-flags="--headless" or if it's still broken.

It sounds like the command we use to set extra headers simply isn't supported by headless, in which case crbug.com is where you'd want to file an issue :)

[zackiles]

@patrickhulce sorry for the confusion.

I've confirmed it is not working passing --headless through the CLI like you've described. You can close this issue and I'll follow up with the Chrome team. Thanks for your help!

[alekseykulikov]

Hi @zackiles
Which version of HeadlessChrome do you use?
I experienced the same issue running HeadlessChrome v64 with a "cookie" header.
But today I tried with HeadlessChrome v68, and it seems to work fine. Possibly the issue was fixed.

Simple steps to confirm. Create a header logging server:

const express = require('express')
const app = express()
app.use((req, res, next) => {
  console.log(req.headers)
  next()
})
app.listen(3000)

Start it with node index.js and run lighthouse audit on it to see headers:

google-chrome --version
# Google Chrome 68.0.3418.0 canary

lighthouse --version
# 2.9.4

lighthouse http://localhost:3000 --chrome-flags="--headless" --extra-headers "{\"Cookie\":\"monster=blue\", \"x-men\":\"wolverine\"}"
# all headers are logged, including:
# cookie: 'monster=blue',
#  'x-men': 'wolverine',

[santoshjoseph99]

I'm trying to use the --extra-headers from the node api and it doesn't seem to work:

const PWMetrics = require('pwmetrics');

const options = {
  flags: {
    runs: 1,
    chromeFlags:'--extra-headers "{\"Cookie\":\"monster=blue\", \"x-men\":\"wolverine\"}',
    disableDeviceEmulation: true,
    disableNetworkThrottling: true,
    disableCpuThrottling: true
  }
};

const pwMetrics = new PWMetrics('http://localhost:3000/', options);
pwMetrics.start()
    .then(result => {
        console.log('RESULT:', result);
    })

output from node program:

{ host: 'localhost:3000',
  connection: 'keep-alive',
  'upgrade-insecure-requests': '1',
  'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3482.0 Safari/537.36',
  accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
  'accept-encoding': 'gzip, deflate, br',
  'accept-language': 'en-US,en;q=0.9' }

Is this a known problem or I'm just not using the options correctly?

[patrickhulce]

@santoshjoseph99 --extraHeaders is a LH flag, not a chrome flag :)

It seems like pwmetrics passes the flags straight through to LH, so you should be able to do

const options = {
  flags: {
    extraHeaders: {Cookie: 'monster=blue', 'x-men': 'wolverine'}
  }
}

seems like the original topic of this issue has been resolved though, so I'll go ahead and close this.

[santoshjoseph99]

@patrickhulce thank you! extraHeaders works.