New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ECONNREFUSED When launching multiple Chrome instances #2462

Closed
LucianoGanga opened this Issue Jun 7, 2017 · 20 comments

Comments

Projects
None yet
7 participants
@LucianoGanga

LucianoGanga commented Jun 7, 2017

Hello!
I don't know why, but when I try to execute multiple instances of Chrome in a Linux env, I get this error:

[ERROR] connect ECONNREFUSED 127.0.0.1:39537
at Object.exports._errnoException (util.js:1050:11)
at exports._exceptionWithHostPort (util.js:1073:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1093:14)

I've been trying to find more information about this, but I don't know where to start from.

Any idea about what may be the problem?

@patrickhulce

This comment has been minimized.

Show comment
Hide comment
@patrickhulce

patrickhulce Jun 7, 2017

Collaborator

How are you launching and consuming chrome? Lighthouse as a node module? chrome-launcher and your own connection to Chrome?

If you're using the latest chrome-launcher module, it should be waiting until the port is available but you'll have to make sure you're awaiting for the promise resolution before connecting.

Collaborator

patrickhulce commented Jun 7, 2017

How are you launching and consuming chrome? Lighthouse as a node module? chrome-launcher and your own connection to Chrome?

If you're using the latest chrome-launcher module, it should be waiting until the port is available but you'll have to make sure you're awaiting for the promise resolution before connecting.

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 7, 2017

Thanks for your answer Patrick!

I'm using the NPM version of lighthouse 2.0.0.

Then, I'm launching chrome here:
https://github.com/LucianoGanga/simple-headless-chrome/blob/beta/lib/browser.js#L149

Then I connect to that instance here:
https://github.com/LucianoGanga/simple-headless-chrome/blob/beta/lib/browser.js#L183

(I use "Chrome Remote Interface" for that: https://github.com/cyrus-and/chrome-remote-interface)

LucianoGanga commented Jun 7, 2017

Thanks for your answer Patrick!

I'm using the NPM version of lighthouse 2.0.0.

Then, I'm launching chrome here:
https://github.com/LucianoGanga/simple-headless-chrome/blob/beta/lib/browser.js#L149

Then I connect to that instance here:
https://github.com/LucianoGanga/simple-headless-chrome/blob/beta/lib/browser.js#L183

(I use "Chrome Remote Interface" for that: https://github.com/cyrus-and/chrome-remote-interface)

@ezesculli

This comment has been minimized.

Show comment
Hide comment
@ezesculli

ezesculli Jun 7, 2017

So far, It appears to be an specific Linux problem (because it works fine in our macs, but not on our servers):

We're basically reproducing this on Linux when:

const chromeInstance = require('lighthouse/chrome-launcher')
chromeOptions = {
    port: 0,
    autoSelectChrome: true, // False to manually select which Chrome install.
    additionalFlags: ['--disable-gpu', '--headless', '--enable-logging']
  }

chromeInstance.launch(chromeOptions)
// Instance 1 -> Chrome Launched succesfully

chromeInstance.launch(chromeOptions)
// Instance 2 -> Chrome Launched succesfully

chromeInstance.launch(chromeOptions)
// Instance 3 -> Error: connect ECONNREFUSED 127.0.0.1:60520

chromeInstance.launch(chromeOptions)
// Instance 4 -> Error: connect ECONNREFUSED 127.0.0.1:40814

Can you help us to reproduce if you're getting the same error on your Linux env?

ezesculli commented Jun 7, 2017

So far, It appears to be an specific Linux problem (because it works fine in our macs, but not on our servers):

We're basically reproducing this on Linux when:

const chromeInstance = require('lighthouse/chrome-launcher')
chromeOptions = {
    port: 0,
    autoSelectChrome: true, // False to manually select which Chrome install.
    additionalFlags: ['--disable-gpu', '--headless', '--enable-logging']
  }

chromeInstance.launch(chromeOptions)
// Instance 1 -> Chrome Launched succesfully

chromeInstance.launch(chromeOptions)
// Instance 2 -> Chrome Launched succesfully

chromeInstance.launch(chromeOptions)
// Instance 3 -> Error: connect ECONNREFUSED 127.0.0.1:60520

chromeInstance.launch(chromeOptions)
// Instance 4 -> Error: connect ECONNREFUSED 127.0.0.1:40814

Can you help us to reproduce if you're getting the same error on your Linux env?

@ezesculli

This comment has been minimized.

Show comment
Hide comment
@ezesculli

ezesculli Jun 8, 2017

Any comments will be really appreciated! We're currently blocked with this!

Anyone? @patrickhulce?

ezesculli commented Jun 8, 2017

Any comments will be really appreciated! We're currently blocked with this!

Anyone? @patrickhulce?

@ebidel

This comment has been minimized.

Show comment
Hide comment
@ebidel

ebidel Jun 8, 2017

Member

@ezesculli it looks like you're using some old flags: autoSelectChrome and additionalFlags.

autoSelectChrome is gone in 2.0 and additionalFlags -> chromeFlags.

Can you try this:

const CDP = require('chrome-remote-interface');
const chromeLauncher = require('lighthouse/chrome-launcher/chrome-launcher');

// Turn on logging for chromeLauncher.
// const log = require('lighthouse/lighthouse-core/lib/log');
// log.setLevel('info');

async function launchChrome(headless = true) {
  const chrome = await chromeLauncher.launch({
    // port: 9222, // uncomment to force a port of your choice.
    chromeFlags: [
      '--disable-gpu',
      '--enable-logging',
      headless ? '--headless' : '',
    ]
  });
  return chrome;
}

launchChrome(true).then(async chrome => {
  // Make sure to use the same debugging port that chromeLauncher used/selected.
  CDP({port: chrome.port}, DTP => {
     ...
  });
});
Member

ebidel commented Jun 8, 2017

@ezesculli it looks like you're using some old flags: autoSelectChrome and additionalFlags.

autoSelectChrome is gone in 2.0 and additionalFlags -> chromeFlags.

Can you try this:

const CDP = require('chrome-remote-interface');
const chromeLauncher = require('lighthouse/chrome-launcher/chrome-launcher');

// Turn on logging for chromeLauncher.
// const log = require('lighthouse/lighthouse-core/lib/log');
// log.setLevel('info');

async function launchChrome(headless = true) {
  const chrome = await chromeLauncher.launch({
    // port: 9222, // uncomment to force a port of your choice.
    chromeFlags: [
      '--disable-gpu',
      '--enable-logging',
      headless ? '--headless' : '',
    ]
  });
  return chrome;
}

launchChrome(true).then(async chrome => {
  // Make sure to use the same debugging port that chromeLauncher used/selected.
  CDP({port: chrome.port}, DTP => {
     ...
  });
});
@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 8, 2017

Thanks @ebidel !

We updated those parameters, but still getting the ECONNREFUSED error:

{ 
    Error: connect ECONNREFUSED 127.0.0.1:58333
        at Object.exports._errnoException (util.js:1050:11)
        at exports._exceptionWithHostPort (util.js:1073:20)
        at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1093:14)
    code: 'ECONNREFUSED',
    errno: 'ECONNREFUSED',
    syscall: 'connect',
    address: '127.0.0.1',
    port: 58333 
}

Besides that, I forced the opening of several instances, and I also get this:

2017-06-08T17:00:06.328813+00:00 app[web.1]: [ERROR] spawnSync /bin/sh EAGAIN
2017-06-08T17:00:06.328817+00:00 app[web.1]:
2017-06-08T17:00:06.328818+00:00 app[web.1]:       at exports._errnoException (util.js:1050:11)
2017-06-08T17:00:06.328818+00:00 app[web.1]:       at spawnSync (child_process.js:459:20)
2017-06-08T17:00:06.328819+00:00 app[web.1]:       at Object.execSync (child_process.js:515:13)
2017-06-08T17:00:06.328820+00:00 app[web.1]:       at makeUnixTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/utils.js:42:28)
2017-06-08T17:00:06.328820+00:00 app[web.1]:       at Object.makeTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/utils.js:33:20)
2017-06-08T17:00:06.328821+00:00 app[web.1]:       at Launcher.makeTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:75:24)
2017-06-08T17:00:06.328822+00:00 app[web.1]:       at Launcher.prepare (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:82:58)
2017-06-08T17:00:06.328823+00:00 app[web.1]:       at Launcher.<anonymous> (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:104:22)
2017-06-08T17:00:06.328823+00:00 app[web.1]:       at Generator.next (<anonymous>)
2017-06-08T17:00:06.328824+00:00 app[web.1]:       at /app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:12:71
2017-06-08T17:00:06.328830+00:00 app[web.1]:       at new wrappedPromise (/app/node_modules/newrelic/lib/instrumentation/promise.js:156:20)
2017-06-08T17:00:06.328831+00:00 app[web.1]:       at __awaiter (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:8:12)
2017-06-08T17:00:06.328832+00:00 app[web.1]:       at Launcher.launch (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:92:16)
2017-06-08T17:00:06.328832+00:00 app[web.1]:       at Object.<anonymous> (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:42:24)
2017-06-08T17:00:06.328833+00:00 app[web.1]:       at Generator.next (<anonymous>)
2017-06-08T17:00:06.328834+00:00 app[web.1]:       at /app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:12:71
2017-06-08T17:00:06.328834+00:00 app[web.1]:       at new wrappedPromise (/app/node_modules/newrelic/lib/instrumentation/promise.js:156:20)
2017-06-08T17:00:06.328835+00:00 app[web.1]:       at __awaiter (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:8:12)
2017-06-08T17:00:06.328835+00:00 app[web.1]:       at Object.launch (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:32:12)
2017-06-08T17:00:06.328836+00:00 app[web.1]:       at _launchChrome (/app/node_modules/simple-headless-chrome/lib/browser.js:152:39)
2017-06-08T17:00:06.328837+00:00 app[web.1]:   code: EAGAIN
2017-06-08T17:00:06.328838+00:00 app[web.1]:   errno: EAGAIN
2017-06-08T17:00:06.328838+00:00 app[web.1]:   syscall: spawnSync /bin/sh
2017-06-08T17:00:06.328839+00:00 app[web.1]:   path: /bin/sh
2017-06-08T17:00:06.328839+00:00 app[web.1]:   spawnargs:
2017-06-08T17:00:06.328840+00:00 app[web.1]:     - -c
2017-06-08T17:00:06.328841+00:00 app[web.1]:     - mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328841+00:00 app[web.1]:   cmd: mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328842+00:00 app[web.1]:   file: /bin/sh
2017-06-08T17:00:06.328842+00:00 app[web.1]:   args:
2017-06-08T17:00:06.328843+00:00 app[web.1]:     - /bin/sh
2017-06-08T17:00:06.328843+00:00 app[web.1]:     - -c
2017-06-08T17:00:06.328844+00:00 app[web.1]:     - mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328845+00:00 app[web.1]:   options:
2017-06-08T17:00:06.328845+00:00 app[web.1]:     shell: true
2017-06-08T17:00:06.328846+00:00 app[web.1]:     file: /bin/sh
2017-06-08T17:00:06.328850+00:00 app[web.1]:     args:
2017-06-08T17:00:06.328851+00:00 app[web.1]:       $ref: $["args"]
2017-06-08T17:00:06.328851+00:00 app[web.1]:     envPairs:
2017-06-08T17:00:06.328852+00:00 app[web.1]:       - LIBRARY_PATH=/app/.apt/usr/lib/x86_64-linux-gnu:/app/.apt/usr/lib/i386-linux-gnu:/app/.apt/usr/lib:
2017-06-08T17:00:06.328853+00:00 app[web.1]:       - WEB_MEMORY=512
2017-06-08T17:00:06.328854+00:00 app[web.1]:       - MEMORY_AVAILABLE=512
2017-06-08T17:00:06.328854+00:00 app[web.1]:       - INCLUDE_PATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328855+00:00 app[web.1]:       - LD_LIBRARY_PATH=/app/.apt/usr/lib/x86_64-linux-gnu:/app/.apt/usr/lib/i386-linux-gnu:/app/.apt/usr/lib:
2017-06-08T17:00:06.328856+00:00 app[web.1]:       - GOOGLE_CHROME_BIN=/app/.apt/opt/google/chrome/chrome
2017-06-08T17:00:06.328856+00:00 app[web.1]:       - CPATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328857+00:00 app[web.1]:       - DYNO=web.1
2017-06-08T17:00:06.328858+00:00 app[web.1]:       - PATH=/app/.heroku/node/bin:/app/.heroku/yarn/bin:/app/.apt/usr/bin:/usr/local/bin:/usr/bin:/bin:/app/bin:/app/node_modules/.bin
2017-06-08T17:00:06.328858+00:00 app[web.1]:       - WEB_CONCURRENCY=1
2017-06-08T17:00:06.328859+00:00 app[web.1]:       - PWD=/app
2017-06-08T17:00:06.328859+00:00 app[web.1]:       - PS1=\[\033[01;34m\]\w\[\033[00m\] \[\033[01;32m\]$ \[\033[00m\]
2017-06-08T17:00:06.328860+00:00 app[web.1]:       - NODE_ENV=production
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - GOOGLE_CHROME_SHIM=/tmp/build_c3b0f4a50c2733e6c09b27904d330d22/.apt/usr/bin/google-chrome-stable
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - SHLVL=1
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - HOME=/app
2017-06-08T17:00:06.328862+00:00 app[web.1]:       - PKG_CONFIG_PATH=/app/.apt/usr/lib/x86_64-linux-gnu/pkgconfig:/app/.apt/usr/lib/i386-linux-gnu/pkgconfig:/app/.apt/usr/lib/pkgconfig:
2017-06-08T17:00:06.328863+00:00 app[web.1]:       - PORT=37295
2017-06-08T17:00:06.328863+00:00 app[web.1]:       - DISPLAY=:42
2017-06-08T17:00:06.328864+00:00 app[web.1]:       - GOOGLE_CHROME_CHANNEL=stable
2017-06-08T17:00:06.328865+00:00 app[web.1]:       - DEBUG=HeadlessChrome:browser
2017-06-08T17:00:06.328865+00:00 app[web.1]:       - NODE_HOME=/app/.heroku/node
2017-06-08T17:00:06.328872+00:00 app[web.1]:       - CPPPATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328872+00:00 app[web.1]:       - _=/app/.heroku/node/bin/node
2017-06-08T17:00:06.328873+00:00 app[web.1]:     stdio:
2017-06-08T17:00:06.328873+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328874+00:00 app[web.1]:         readable: true
2017-06-08T17:00:06.328875+00:00 app[web.1]:         writable: false
2017-06-08T17:00:06.328875+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328876+00:00 app[web.1]:         readable: false
2017-06-08T17:00:06.328876+00:00 app[web.1]:         writable: true
2017-06-08T17:00:06.328877+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328877+00:00 app[web.1]:         readable: false
2017-06-08T17:00:06.328878+00:00 app[web.1]:         writable: true
2017-06-08T17:00:06.328879+00:00 app[web.1]:   envPairs:
2017-06-08T17:00:06.328879+00:00 app[web.1]:     $ref: $["options"]["envPairs"]
2017-06-08T17:00:06.328880+00:00 app[web.1]:   stderr: null
2017-06-08T17:00:06.328880+00:00 app[web.1]:   stdout: null
2017-06-08T17:00:06.328881+00:00 app[web.1]:   pid: 0
2017-06-08T17:00:06.328883+00:00 app[web.1]:   output: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   signal: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   status: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   error: null
2017-06-08T17:00:06.328885+00:00 app[web.1]:   httpStatusMsg: Internal Server Error
2017-06-08T17:00:06.328886+00:00 app[web.1]:   httpStatusCode: 500

LucianoGanga commented Jun 8, 2017

Thanks @ebidel !

We updated those parameters, but still getting the ECONNREFUSED error:

{ 
    Error: connect ECONNREFUSED 127.0.0.1:58333
        at Object.exports._errnoException (util.js:1050:11)
        at exports._exceptionWithHostPort (util.js:1073:20)
        at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1093:14)
    code: 'ECONNREFUSED',
    errno: 'ECONNREFUSED',
    syscall: 'connect',
    address: '127.0.0.1',
    port: 58333 
}

Besides that, I forced the opening of several instances, and I also get this:

2017-06-08T17:00:06.328813+00:00 app[web.1]: [ERROR] spawnSync /bin/sh EAGAIN
2017-06-08T17:00:06.328817+00:00 app[web.1]:
2017-06-08T17:00:06.328818+00:00 app[web.1]:       at exports._errnoException (util.js:1050:11)
2017-06-08T17:00:06.328818+00:00 app[web.1]:       at spawnSync (child_process.js:459:20)
2017-06-08T17:00:06.328819+00:00 app[web.1]:       at Object.execSync (child_process.js:515:13)
2017-06-08T17:00:06.328820+00:00 app[web.1]:       at makeUnixTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/utils.js:42:28)
2017-06-08T17:00:06.328820+00:00 app[web.1]:       at Object.makeTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/utils.js:33:20)
2017-06-08T17:00:06.328821+00:00 app[web.1]:       at Launcher.makeTmpDir (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:75:24)
2017-06-08T17:00:06.328822+00:00 app[web.1]:       at Launcher.prepare (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:82:58)
2017-06-08T17:00:06.328823+00:00 app[web.1]:       at Launcher.<anonymous> (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:104:22)
2017-06-08T17:00:06.328823+00:00 app[web.1]:       at Generator.next (<anonymous>)
2017-06-08T17:00:06.328824+00:00 app[web.1]:       at /app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:12:71
2017-06-08T17:00:06.328830+00:00 app[web.1]:       at new wrappedPromise (/app/node_modules/newrelic/lib/instrumentation/promise.js:156:20)
2017-06-08T17:00:06.328831+00:00 app[web.1]:       at __awaiter (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:8:12)
2017-06-08T17:00:06.328832+00:00 app[web.1]:       at Launcher.launch (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:92:16)
2017-06-08T17:00:06.328832+00:00 app[web.1]:       at Object.<anonymous> (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:42:24)
2017-06-08T17:00:06.328833+00:00 app[web.1]:       at Generator.next (<anonymous>)
2017-06-08T17:00:06.328834+00:00 app[web.1]:       at /app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:12:71
2017-06-08T17:00:06.328834+00:00 app[web.1]:       at new wrappedPromise (/app/node_modules/newrelic/lib/instrumentation/promise.js:156:20)
2017-06-08T17:00:06.328835+00:00 app[web.1]:       at __awaiter (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:8:12)
2017-06-08T17:00:06.328835+00:00 app[web.1]:       at Object.launch (/app/node_modules/simple-headless-chrome/node_modules/lighthouse/chrome-launcher/chrome-launcher.js:32:12)
2017-06-08T17:00:06.328836+00:00 app[web.1]:       at _launchChrome (/app/node_modules/simple-headless-chrome/lib/browser.js:152:39)
2017-06-08T17:00:06.328837+00:00 app[web.1]:   code: EAGAIN
2017-06-08T17:00:06.328838+00:00 app[web.1]:   errno: EAGAIN
2017-06-08T17:00:06.328838+00:00 app[web.1]:   syscall: spawnSync /bin/sh
2017-06-08T17:00:06.328839+00:00 app[web.1]:   path: /bin/sh
2017-06-08T17:00:06.328839+00:00 app[web.1]:   spawnargs:
2017-06-08T17:00:06.328840+00:00 app[web.1]:     - -c
2017-06-08T17:00:06.328841+00:00 app[web.1]:     - mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328841+00:00 app[web.1]:   cmd: mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328842+00:00 app[web.1]:   file: /bin/sh
2017-06-08T17:00:06.328842+00:00 app[web.1]:   args:
2017-06-08T17:00:06.328843+00:00 app[web.1]:     - /bin/sh
2017-06-08T17:00:06.328843+00:00 app[web.1]:     - -c
2017-06-08T17:00:06.328844+00:00 app[web.1]:     - mktemp -d -t lighthouse.XXXXXXX
2017-06-08T17:00:06.328845+00:00 app[web.1]:   options:
2017-06-08T17:00:06.328845+00:00 app[web.1]:     shell: true
2017-06-08T17:00:06.328846+00:00 app[web.1]:     file: /bin/sh
2017-06-08T17:00:06.328850+00:00 app[web.1]:     args:
2017-06-08T17:00:06.328851+00:00 app[web.1]:       $ref: $["args"]
2017-06-08T17:00:06.328851+00:00 app[web.1]:     envPairs:
2017-06-08T17:00:06.328852+00:00 app[web.1]:       - LIBRARY_PATH=/app/.apt/usr/lib/x86_64-linux-gnu:/app/.apt/usr/lib/i386-linux-gnu:/app/.apt/usr/lib:
2017-06-08T17:00:06.328853+00:00 app[web.1]:       - WEB_MEMORY=512
2017-06-08T17:00:06.328854+00:00 app[web.1]:       - MEMORY_AVAILABLE=512
2017-06-08T17:00:06.328854+00:00 app[web.1]:       - INCLUDE_PATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328855+00:00 app[web.1]:       - LD_LIBRARY_PATH=/app/.apt/usr/lib/x86_64-linux-gnu:/app/.apt/usr/lib/i386-linux-gnu:/app/.apt/usr/lib:
2017-06-08T17:00:06.328856+00:00 app[web.1]:       - GOOGLE_CHROME_BIN=/app/.apt/opt/google/chrome/chrome
2017-06-08T17:00:06.328856+00:00 app[web.1]:       - CPATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328857+00:00 app[web.1]:       - DYNO=web.1
2017-06-08T17:00:06.328858+00:00 app[web.1]:       - PATH=/app/.heroku/node/bin:/app/.heroku/yarn/bin:/app/.apt/usr/bin:/usr/local/bin:/usr/bin:/bin:/app/bin:/app/node_modules/.bin
2017-06-08T17:00:06.328858+00:00 app[web.1]:       - WEB_CONCURRENCY=1
2017-06-08T17:00:06.328859+00:00 app[web.1]:       - PWD=/app
2017-06-08T17:00:06.328859+00:00 app[web.1]:       - PS1=\[\033[01;34m\]\w\[\033[00m\] \[\033[01;32m\]$ \[\033[00m\]
2017-06-08T17:00:06.328860+00:00 app[web.1]:       - NODE_ENV=production
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - GOOGLE_CHROME_SHIM=/tmp/build_c3b0f4a50c2733e6c09b27904d330d22/.apt/usr/bin/google-chrome-stable
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - SHLVL=1
2017-06-08T17:00:06.328861+00:00 app[web.1]:       - HOME=/app
2017-06-08T17:00:06.328862+00:00 app[web.1]:       - PKG_CONFIG_PATH=/app/.apt/usr/lib/x86_64-linux-gnu/pkgconfig:/app/.apt/usr/lib/i386-linux-gnu/pkgconfig:/app/.apt/usr/lib/pkgconfig:
2017-06-08T17:00:06.328863+00:00 app[web.1]:       - PORT=37295
2017-06-08T17:00:06.328863+00:00 app[web.1]:       - DISPLAY=:42
2017-06-08T17:00:06.328864+00:00 app[web.1]:       - GOOGLE_CHROME_CHANNEL=stable
2017-06-08T17:00:06.328865+00:00 app[web.1]:       - DEBUG=HeadlessChrome:browser
2017-06-08T17:00:06.328865+00:00 app[web.1]:       - NODE_HOME=/app/.heroku/node
2017-06-08T17:00:06.328872+00:00 app[web.1]:       - CPPPATH=/app/.apt/usr/include:/app/.apt/usr/include/x86_64-linux-gnu:
2017-06-08T17:00:06.328872+00:00 app[web.1]:       - _=/app/.heroku/node/bin/node
2017-06-08T17:00:06.328873+00:00 app[web.1]:     stdio:
2017-06-08T17:00:06.328873+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328874+00:00 app[web.1]:         readable: true
2017-06-08T17:00:06.328875+00:00 app[web.1]:         writable: false
2017-06-08T17:00:06.328875+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328876+00:00 app[web.1]:         readable: false
2017-06-08T17:00:06.328876+00:00 app[web.1]:         writable: true
2017-06-08T17:00:06.328877+00:00 app[web.1]:       - type: pipe
2017-06-08T17:00:06.328877+00:00 app[web.1]:         readable: false
2017-06-08T17:00:06.328878+00:00 app[web.1]:         writable: true
2017-06-08T17:00:06.328879+00:00 app[web.1]:   envPairs:
2017-06-08T17:00:06.328879+00:00 app[web.1]:     $ref: $["options"]["envPairs"]
2017-06-08T17:00:06.328880+00:00 app[web.1]:   stderr: null
2017-06-08T17:00:06.328880+00:00 app[web.1]:   stdout: null
2017-06-08T17:00:06.328881+00:00 app[web.1]:   pid: 0
2017-06-08T17:00:06.328883+00:00 app[web.1]:   output: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   signal: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   status: null
2017-06-08T17:00:06.328884+00:00 app[web.1]:   error: null
2017-06-08T17:00:06.328885+00:00 app[web.1]:   httpStatusMsg: Internal Server Error
2017-06-08T17:00:06.328886+00:00 app[web.1]:   httpStatusCode: 500
@ebidel

This comment has been minimized.

Show comment
Hide comment
@ebidel

ebidel Jun 8, 2017

Member

Hmm, ok. I'll have to test on Linux. Do you think you could post a snippet of code that repros the problem?

Member

ebidel commented Jun 8, 2017

Hmm, ok. I'll have to test on Linux. Do you think you could post a snippet of code that repros the problem?

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 8, 2017

@ebidel I think that with the code that @ezesculli posted before it's enough, but I've launched a EC2 instance in AWS to test this (which gives me the ECONNREFUSED error even in the first attempt of launching a chrome instance).

If you want, I can privately send you the access credentials so you can test it there.

LucianoGanga commented Jun 8, 2017

@ebidel I think that with the code that @ezesculli posted before it's enough, but I've launched a EC2 instance in AWS to test this (which gives me the ECONNREFUSED error even in the first attempt of launching a chrome instance).

If you want, I can privately send you the access credentials so you can test it there.

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 8, 2017

I don't know if this helps, but on Linux I ran the code you gave us:

const CDP = require('chrome-remote-interface');
const chromeLauncher = require('lighthouse/chrome-launcher/chrome-launcher');

// Turn on logging for chromeLauncher.
const log = require('lighthouse/lighthouse-core/lib/log');
log.setLevel('info');

async function launchChrome(headless = true) {
  const chrome = await chromeLauncher.launch({
    // port: 9222, // uncomment to force a port of your choice.
    chromeFlags: [
      '--disable-gpu',
      '--enable-logging',
      headless ? '--headless' : '',
    ]
  });
  return chrome;
}

and then ran this several times:

launchChrome(true).then(async chrome => {
  // Make sure to use the same debugging port that chromeLauncher used/selected.
  CDP({port: chrome.port}, DTP => {
     console.log('DTP', DTP.port)
  });
});

and after a few runs, I've got this:

> (node:1963) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit

LucianoGanga commented Jun 8, 2017

I don't know if this helps, but on Linux I ran the code you gave us:

const CDP = require('chrome-remote-interface');
const chromeLauncher = require('lighthouse/chrome-launcher/chrome-launcher');

// Turn on logging for chromeLauncher.
const log = require('lighthouse/lighthouse-core/lib/log');
log.setLevel('info');

async function launchChrome(headless = true) {
  const chrome = await chromeLauncher.launch({
    // port: 9222, // uncomment to force a port of your choice.
    chromeFlags: [
      '--disable-gpu',
      '--enable-logging',
      headless ? '--headless' : '',
    ]
  });
  return chrome;
}

and then ran this several times:

launchChrome(true).then(async chrome => {
  // Make sure to use the same debugging port that chromeLauncher used/selected.
  CDP({port: chrome.port}, DTP => {
     console.log('DTP', DTP.port)
  });
});

and after a few runs, I've got this:

> (node:1963) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit
@patrickhulce

This comment has been minimized.

Show comment
Hide comment
@patrickhulce

patrickhulce Jun 8, 2017

Collaborator

@LucianoGanga that EventEmitter leak is a known leak with the SIGINT handling (you can flip the handleSIGINT option to false to disable since you likely don't need it anyway), but shouldn't impact the CONNREFUSED you're seeing

I'm assuming with these cases that you're observing a significant delay while it waits for the port to be available and it finally fails sometime later? Or is it immediately returning CONNREFUSED (meaning that Chrome Launcher thinks it was able to connect but CRI failed to connect)?

Collaborator

patrickhulce commented Jun 8, 2017

@LucianoGanga that EventEmitter leak is a known leak with the SIGINT handling (you can flip the handleSIGINT option to false to disable since you likely don't need it anyway), but shouldn't impact the CONNREFUSED you're seeing

I'm assuming with these cases that you're observing a significant delay while it waits for the port to be available and it finally fails sometime later? Or is it immediately returning CONNREFUSED (meaning that Chrome Launcher thinks it was able to connect but CRI failed to connect)?

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 8, 2017

@patrickhulce I just tested it again, It takes between 10 and 20 seconds since the chromeLauncher is excecuted until I get the error.

I made some new tests in a new ubuntu server (instead of using the Heroku instance) and it fails after 10/15 new instances. In my mac, I can run it more than 30 times without any problem.

LucianoGanga commented Jun 8, 2017

@patrickhulce I just tested it again, It takes between 10 and 20 seconds since the chromeLauncher is excecuted until I get the error.

I made some new tests in a new ubuntu server (instead of using the Heroku instance) and it fails after 10/15 new instances. In my mac, I can run it more than 30 times without any problem.

@ebidel

This comment has been minimized.

Show comment
Hide comment
@ebidel

ebidel Jun 8, 2017

Member

@patrickhulce I lied. Wasn't able to take a look today and I'm gone tomorrow for a wedding. Feel free to test if you get a chance. Otherwise, I can get to it next week.

Member

ebidel commented Jun 8, 2017

@patrickhulce I lied. Wasn't able to take a look today and I'm gone tomorrow for a wedding. Feel free to test if you get a chance. Otherwise, I can get to it next week.

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 12, 2017

We added more and better hardware and the problem seems to be solved... I think there's a OS limitation that prevents the instances from opening ports on time, or something around that (I'm not an expert in OS).

If you want, we can close this issue.

LucianoGanga commented Jun 12, 2017

We added more and better hardware and the problem seems to be solved... I think there's a OS limitation that prevents the instances from opening ports on time, or something around that (I'm not an expert in OS).

If you want, we can close this issue.

@patrickhulce

This comment has been minimized.

Show comment
Hide comment
@patrickhulce

patrickhulce Jun 12, 2017

Collaborator

Thanks for circling back around to let us know! That certainly would explain my difficulty reproing on a 64-core linux box :)

Feel free to reach out again if you run into new issues!

Collaborator

patrickhulce commented Jun 12, 2017

Thanks for circling back around to let us know! That certainly would explain my difficulty reproing on a 64-core linux box :)

Feel free to reach out again if you run into new issues!

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 26, 2017

Hi @patrickhulce !

We're having the issue again, but this time I can't even run one instance of chrome.

This is the code:

const chromeLauncher = require('chrome-launcher');

chromeOptions = {
        port: 0,
        autoSelectChrome: true, // False to manually select which Chrome install.
        additionalFlags: ["--disable-gpu", "--headless", "--enable-logging"]
}

// Launch chrome
chromeLauncher.launch(chromeOptions)

Which gives me this error:

(node:1223) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: connect ECONNREFUSED 127.0.0.1:36640
(node:1223) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: connect ECONNREFUSED 127.0.0.1:36763

Any ideas?

The error seems to be happening here:

Error code: ECONNREFUSED { Error: connect ECONNREFUSED 127.0.0.1:36640
    at Object.exports._errnoException (util.js:1016:11)
    at exports._exceptionWithHostPort (util.js:1039:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1138:14)
From previous event:
    at Launcher.isDebuggerReady (/home/ubuntu/sirena/themisto/node_modules/chrome-launcher/chrome-launcher.js:153:16)
    at poll (/home/ubuntu/sirena/themisto/node_modules/chrome-launcher/chrome-launcher.js:178:26)
    at runCallback (timers.js:800:20)
    at tryOnImmediate (timers.js:762:5)
    at processImmediate [as _immediateCallback] (timers.js:733:5)
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 36640 }

Any ideas?

Thanks!
Luciano

LucianoGanga commented Jun 26, 2017

Hi @patrickhulce !

We're having the issue again, but this time I can't even run one instance of chrome.

This is the code:

const chromeLauncher = require('chrome-launcher');

chromeOptions = {
        port: 0,
        autoSelectChrome: true, // False to manually select which Chrome install.
        additionalFlags: ["--disable-gpu", "--headless", "--enable-logging"]
}

// Launch chrome
chromeLauncher.launch(chromeOptions)

Which gives me this error:

(node:1223) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: connect ECONNREFUSED 127.0.0.1:36640
(node:1223) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: connect ECONNREFUSED 127.0.0.1:36763

Any ideas?

The error seems to be happening here:

Error code: ECONNREFUSED { Error: connect ECONNREFUSED 127.0.0.1:36640
    at Object.exports._errnoException (util.js:1016:11)
    at exports._exceptionWithHostPort (util.js:1039:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1138:14)
From previous event:
    at Launcher.isDebuggerReady (/home/ubuntu/sirena/themisto/node_modules/chrome-launcher/chrome-launcher.js:153:16)
    at poll (/home/ubuntu/sirena/themisto/node_modules/chrome-launcher/chrome-launcher.js:178:26)
    at runCallback (timers.js:800:20)
    at tryOnImmediate (timers.js:762:5)
    at processImmediate [as _immediateCallback] (timers.js:733:5)
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 36640 }

Any ideas?

Thanks!
Luciano

@paulirish

This comment has been minimized.

Show comment
Hide comment
@paulirish

paulirish Jun 26, 2017

Member

I am able to reproduce this. here's the steps

install travis node-js image

docker run --name travis-debug -dit travisci/ci-garnet:packer-1496954857 /sbin/init
docker exec -it travis-debug bash -l

setup things within the travis shell:

su - travis
mkdir tempproj && cd tempproj
yarn init # enter, enter, enter, enter...
yarn add chrome-launcher

then create an index.js with the code from @LucianoGanga's comment above. and run:

node index.js
Member

paulirish commented Jun 26, 2017

I am able to reproduce this. here's the steps

install travis node-js image

docker run --name travis-debug -dit travisci/ci-garnet:packer-1496954857 /sbin/init
docker exec -it travis-debug bash -l

setup things within the travis shell:

su - travis
mkdir tempproj && cd tempproj
yarn init # enter, enter, enter, enter...
yarn add chrome-launcher

then create an index.js with the code from @LucianoGanga's comment above. and run:

node index.js
@paulirish

This comment has been minimized.

Show comment
Hide comment
@paulirish

paulirish Jun 26, 2017

Member

@LucianoGanga

you have an error in your code when you upgraded to latest chrome-launcher.

heres the launch options you want:

chromeOptions = {
        chromeFlags: ["--disable-gpu", "--headless", "--enable-logging"] 
}

you may also want --no-sandbox in there as well.. I just ran into requiring that.

Member

paulirish commented Jun 26, 2017

@LucianoGanga

you have an error in your code when you upgraded to latest chrome-launcher.

heres the launch options you want:

chromeOptions = {
        chromeFlags: ["--disable-gpu", "--headless", "--enable-logging"] 
}

you may also want --no-sandbox in there as well.. I just ran into requiring that.

@LucianoGanga

This comment has been minimized.

Show comment
Hide comment
@LucianoGanga

LucianoGanga Jun 27, 2017

@paulirish I feel really lame now... Sorry for the trouble!

The issue wasn't the name of the properties (I had them right in my production code), but the thing was that the "--headless" flag depended on a variable that was set to false.

It didn't cause any kind of problem in my pc, but it did in Linux through command line (still don't know why)

I'm really sorry for this, really! You can close this issue.

Cheers,
Luciano

PS: at least now you know the trick to get an extra piece of stack in an error ! :P

LucianoGanga commented Jun 27, 2017

@paulirish I feel really lame now... Sorry for the trouble!

The issue wasn't the name of the properties (I had them right in my production code), but the thing was that the "--headless" flag depended on a variable that was set to false.

It didn't cause any kind of problem in my pc, but it did in Linux through command line (still don't know why)

I'm really sorry for this, really! You can close this issue.

Cheers,
Luciano

PS: at least now you know the trick to get an extra piece of stack in an error ! :P

@joelgriffith

This comment has been minimized.

Show comment
Hide comment
@joelgriffith

joelgriffith Jul 13, 2017

I'm getting some reports in Navalia with similar error types. I haven't been able to repro myself, but I'll post in this issue if I uncover something 🐟

joelgriffith commented Jul 13, 2017

I'm getting some reports in Navalia with similar error types. I haven't been able to repro myself, but I'll post in this issue if I uncover something 🐟

@svewag

This comment has been minimized.

Show comment
Hide comment
@svewag

svewag May 31, 2018

@LucianoGanga, you said after upgrading your hardware the problem was gone.
Can you remember what exactly made the point?

We are running into the exact same problems when doing more than four lighthouse runs in parallel.

(node:28) UnhandledPromiseRejectionWarning: Error: not opened
    at WebSocket.send (/builds/xxx/xxx-4.0/node_modules/lighthouse/node_modules/ws/lib/WebSocket.js:344:18)
    at CriConnection.sendRawMessage (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:157:14)
    at CriConnection.sendCommand (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/connections/connection.js:55:10)
    at Driver.sendCommand (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:205:29)
    at Promise (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:870:19)
    at new Promise (<anonymous>)
    at Driver.endTrace (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:862:12)
    at pass.then._ (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/gather-runner.js:274:23)
    at <anonymous>
(node:28) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 11)
(node:28) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
events.js:165
      throw er; // Unhandled 'error' event
      ^

Error: connect ECONNREFUSED 127.0.0.1:43219
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)
Emitted 'error' event at:
    at Socket.socketErrorListener (_http_client.js:395:9)
    at Socket.emit (events.js:180:13)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)

Thank you for your feedback.

svewag commented May 31, 2018

@LucianoGanga, you said after upgrading your hardware the problem was gone.
Can you remember what exactly made the point?

We are running into the exact same problems when doing more than four lighthouse runs in parallel.

(node:28) UnhandledPromiseRejectionWarning: Error: not opened
    at WebSocket.send (/builds/xxx/xxx-4.0/node_modules/lighthouse/node_modules/ws/lib/WebSocket.js:344:18)
    at CriConnection.sendRawMessage (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:157:14)
    at CriConnection.sendCommand (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/connections/connection.js:55:10)
    at Driver.sendCommand (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:205:29)
    at Promise (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:870:19)
    at new Promise (<anonymous>)
    at Driver.endTrace (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/driver.js:862:12)
    at pass.then._ (/builds/xxx/xxx-4.0/node_modules/lighthouse/lighthouse-core/gather/gather-runner.js:274:23)
    at <anonymous>
(node:28) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 11)
(node:28) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
events.js:165
      throw er; // Unhandled 'error' event
      ^

Error: connect ECONNREFUSED 127.0.0.1:43219
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)
Emitted 'error' event at:
    at Socket.socketErrorListener (_http_client.js:395:9)
    at Socket.emit (events.js:180:13)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at process._tickCallback (internal/process/next_tick.js:178:19)

Thank you for your feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment