EventEmitter memory leak detected. 11 exit listeners added #594

selfrefactor · 2017-08-29T09:15:47Z

This is the full error message:
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit

I run Node v8.1.2 with tip-of-tree Puppeteer

The code is written with Promise.all, but even if I run work() sequentially the same error appears.

const puppeteer = require('puppeteer')
const R = require('rambda')

const resolution = {
  x : 1920,
  y : 1080,
}

const args = [
  '--disable-gpu',
  `--window-size=${ resolution.x },${ resolution.y }`,
  '--no-sandbox',
]

const work = async () => {
  const browser = await puppeteer.launch({
    headless     : true,
    handleSIGINT : false,
    args         : args,
  })
  const page = await browser.newPage()

  const url = 'https://ilearnsmarter.com/learning-meme'

  await page.setViewport({
    width  : resolution.x,
    height : resolution.y,
  })

  await page.goto(url, { waitUntil : 'networkidle' })
  await browser.close()
}

const fn = async () => {
  const promised = R.range(0, 12).map(() => work())
  await Promise.all(promised)

  console.log('DONE')
}

fn()

The text was updated successfully, but these errors were encountered:

selfrefactor · 2017-08-29T09:39:02Z

One possible solution is using child process:

// workChild.js
const puppeteer = require('puppeteer')

const resolution = {
  x : 1920,
  y : 1080,
}

const args = [
  '--disable-gpu',
  `--window-size=${ resolution.x },${ resolution.y }`,
  '--no-sandbox',
]

const work = async () => {
  const browser = await puppeteer.launch({
    headless     : true,
    handleSIGINT : false,
    args         : args,
  })
  const page = await browser.newPage()

  const url = 'https://ilearnsmarter.com/learning-meme'

  await page.setViewport({
    width  : resolution.x,
    height : resolution.y,
  })

  await page.goto(url, { waitUntil : 'networkidle' })

  const result = await page.content()
  await browser.close()

  return result.length
}

work()
  .then(result => {
    process.send(result)
    process.exit()
  })

const { fork } = require("child_process")
const R = require('rambda')

const work = () => new Promise(resolve=>{
  const child = fork("workChild.js", [], {
    cwd : __dirname,
    env : process.env,
  })

  child.on("message", result => {
    return resolve(result)
  })
})

const fn = async () => {
  const promised = R.range(0, 12).map(() => work())
  const result = await Promise.all(promised)

  console.log(result, 'DONE')
}

fn()

I still wonder if this is the only current solution.

domarmstrong · 2017-08-29T16:26:47Z

I've also hit this in node 7.10 but I'm getting 'finish' listener. My service renders a pdf one at a time, it runs launch and creates a new page for each pdf. But this seems to have occurred after the service was running for a while.

(node:7) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit

ebidel · 2017-08-29T17:03:28Z

Lighthouse and chrome-launcher handled this in GoogleChrome/lighthouse#2959.

aslushnikov · 2017-08-30T05:53:15Z

@selfrefactor your script should be working fine now.
@domarmstrong there's no 'finish' event in the puppeteer - this looks like something from other modules.

selfrefactor · 2017-08-30T07:39:41Z

@aslushnikov Thank you for the fast fix, but it still doesn't work for me.

I tested it on local machine and Cloud9 instance and I make sure that the code reads your changes, but the same error appears again.

aslushnikov · 2017-08-30T08:25:02Z

@selfrefactor ah, indeed, you launch chrome processes in parallel. Every chrome instance adds a listener to the process's "exit" event to cleanup properly. 12 chrome instances add 12 listeners, which yields the warning.

You can fix this by adding process.setMaxListeners(Infinity) in the very beginning of your script:

const puppeteer = require('puppeteer')
const R = require('rambda')

process.setMaxListeners(Infinity); // <== Important line

const resolution = {
  x : 1920,
  y : 1080,
}
// ...

Note: you don't need to launch browser just to create a page. Instead, you can open multiple pages in the same browser:

const puppeteer = require('puppeteer')
const R = require('rambda')

const resolution = {
  x : 1920,
  y : 1080,
}

const args = [
  '--disable-gpu',
  `--window-size=${ resolution.x },${ resolution.y }`,
  '--no-sandbox',
]

let browser;

const work = async () => {
  const page = await browser.newPage()

  const url = 'https://ilearnsmarter.com/learning-meme'

  await page.setViewport({
    width  : resolution.x,
    height : resolution.y,
  })

  await page.goto(url, { waitUntil : 'networkidle' })
  await page.close();
}

const fn = async () => {
  browser = await puppeteer.launch({
    headless     : true,
    handleSIGINT : false,
    args         : args,
  })
  const promised = R.range(0, 12).map(() => work())
  await Promise.all(promised)
  browser.close()

  console.log('DONE')
}

fn()

domarmstrong · 2017-08-30T09:25:14Z

const puppeteer = require('puppeteer');

// bad url with no protocol fails fast
const jobs = [...new Array(12)].map(() => 'google.com');

const work = async (url) => {
  try {
    const browser = await puppeteer.launch({ dumpio: true });
    const page = await browser.newPage();
    await page.goto(url);
    browser.close();
  } catch (err) {
    console.log(err.message);
  } finally {
    if (jobs.length) {
      work(jobs.pop());
    } else {
      process.exit();
    }
  }
};

work(jobs.pop());

The listeners don't seem to get cleaned up, and if I dont have the process.exit it doesn't ever exit the process. Also you can see there are a bunch of event emitter errors show like unpipe, drain, error. These are only shown when using dumpio. Setting max event listeners is not an option for me as I'm using as part of a long running process so it will leak memory.

output:


DevTools listening on ws://127.0.0.1:34165/devtools/browser/abc53811-c8f6-4ed7-93e7-575cf12fd06d
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:38347/devtools/browser/12bd2c36-5418-47b5-827f-e4a233dd810c
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:39131/devtools/browser/4e1e31ca-bad1-44e9-bdc7-57e3e643266c
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:41805/devtools/browser/136993b1-537e-4780-a7ae-a9b23d001e23
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:44565/devtools/browser/a15f20f5-0127-499d-aec2-afde6ab437bb
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:44539/devtools/browser/b2a1da7d-0d00-481e-8661-0bd663fdaa6c
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:33217/devtools/browser/c8dc09cb-6ed2-4a60-8b95-ad68a4d94432
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:33041/devtools/browser/b8c723e9-621b-49a7-b1fe-d169e92699e0
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:33847/devtools/browser/bab79465-acfa-48ff-af3c-951594c525bb
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit

DevTools listening on ws://127.0.0.1:45271/devtools/browser/374bffab-c78f-4f6a-a617-4e7d3c9e2b00
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 drain listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit
(node:5618) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit

DevTools listening on ws://127.0.0.1:44721/devtools/browser/fbfb2737-aaee-4145-889b-3b0796a63103
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

DevTools listening on ws://127.0.0.1:45515/devtools/browser/5b561d62-cb15-4633-b248-c16bbae0d938
Protocol error (Page.navigate): Cannot navigate to invalid URL undefined

aslushnikov · 2017-08-30T09:44:05Z

@domarmstrong the reason your script doesn't work with tip-of-tree puppeteer is because you don't close browsers if exception is thrown. This is a memory leak.

The following script works just fine:

const puppeteer = require('puppeteer');

// bad url with no protocol fails fast
const jobs = [...new Array(12)].map(() => 'google.com');

const work = async (url) => {
  const browser = await puppeteer.launch({dumpio: true});
  try {
    const page = await browser.newPage();
    await page.goto(url);
  } catch (err) {
    console.log(err.message);
  } finally {
    browser.close();
    if (jobs.length) {
      work(jobs.pop());
    } else {
      process.exit();
    }
  }
};

work(jobs.pop());

if you pass https://google.com instead of google.com, no exception will be thrown and your original script will work just fine.

domarmstrong · 2017-08-30T10:01:32Z

OK, that's a very good point lol. But still you get the exit and SIGINT errors.

const puppeteer = require('puppeteer');

// bad url with no protocol fails fast
const jobs = [...new Array(12)].map(() => 'https://google.com');

const work = async (url) => {
  let browser;
  try {
    browser = await puppeteer.launch({ dumpio: true });
    const page = await browser.newPage();
    await page.goto(url);
  } catch (err) {
    console.log(err.message);
  } finally {
    if (browser) {
      browser.close();
    }

    if (jobs.length) {
      work(jobs.pop());
    } else {
      process.exit();
    }
  }
};

work(jobs.pop());

DevTools listening on ws://127.0.0.1:44485/devtools/browser/61c2fd13-086d-4546-8738-eb91ebe3eb6b

DevTools listening on ws://127.0.0.1:36193/devtools/browser/0f73b4b5-9ac1-48be-a137-e7bec712d949

DevTools listening on ws://127.0.0.1:43067/devtools/browser/7b19c8f0-f173-417d-a77d-b27c22d75805

DevTools listening on ws://127.0.0.1:45905/devtools/browser/39c2c0fb-ef8a-4f58-8d31-90b7347c0138

DevTools listening on ws://127.0.0.1:35571/devtools/browser/b6125682-90ed-4f6b-b456-f3c79078f9ee

DevTools listening on ws://127.0.0.1:40393/devtools/browser/dd7a2064-0053-40f6-b957-9e7e5fceba9b

DevTools listening on ws://127.0.0.1:40311/devtools/browser/3f944449-ac8d-4165-b239-48597520ee1c

DevTools listening on ws://127.0.0.1:46481/devtools/browser/ea98a4aa-11d6-4e3e-b614-ae9968ba4187

DevTools listening on ws://127.0.0.1:40055/devtools/browser/66e1626e-c038-470b-b585-3c9b0b051169

DevTools listening on ws://127.0.0.1:46827/devtools/browser/717005b0-7033-4b4a-aeeb-fdd756dbb5d6
(node:30356) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit
(node:30356) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit

DevTools listening on ws://127.0.0.1:35135/devtools/browser/e29d2f79-67e9-4128-956e-bfac92064682

DevTools listening on ws://127.0.0.1:44217/devtools/browser/c4cb9006-155f-49f5-8012-ff38be09ed3f

aslushnikov · 2017-08-30T10:05:28Z

@domarmstrong What's your puppeteer version? Can you run the npm list --depth=0 from the root of your project? It looks like you're not using the tip-of-tree puppeteer, which is labeled as 0.10.2-alpha.

domarmstrong · 2017-08-30T10:17:32Z

Yep your right. That's fixed then. 👍 thanks.

selfrefactor · 2017-08-30T11:16:13Z

@aslushnikov Actually I don't need it to run in parallel and the code really works in sequential context. Thank you for the efforts.

stereobooster · 2018-03-12T14:28:48Z

Had the same issue with gulp-critical. Solution is to use gulp-limit

intellix · 2018-03-14T16:30:46Z

Got a queue of 12 items to process at once. I was creating 12 browsers with a page for each and getting the above error. Then I tried 1 browser for all tabs and noticed the time to process all 12 of those items doubled and had issues when closing pages breaking the others.

Then tried 4x browser with 3 pages each and still got lots of issues with previously closed pages causing the others to break.

Not sure what happened but for me I'm going to just increase the limit to remove the error.

ankur-21 · 2020-01-18T16:29:00Z

(node:25120) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit

ndotie · 2020-08-14T21:46:10Z

right before on your app.listen(port,() => {})
add process.setMaxListeners(0) for infinity event listener chaining

Mutale85 · 2021-05-09T17:31:20Z

For me, I would like to ask, whats the best way to capture more than 20 screenshots or different Urls?
I have tried the following code.

async function sCapture(url, site_name) {
 	const browser = await puppeteer.launch();
 	const page = await browser.newPage();
 	await page.setViewport({ width: 1280, height: 720 })
 	await page.goto(url);
 	await page.screenshot({
 	path:`statusImage/${site_name}.jpg`
 	});
 	await browser.close();
 }

 Am getting the Urls from my DB like this. 
 
 db_connection.promise()
.execute("SELECT * FROM `urls`")
.then(([rows]) => {
  
    rows.forEach(user => {
        const url 	= user.link;
    	const name 	= user.link_name;
    	console.log(name);
    	sCapture(url, name)
    	
    });
	db_connection.end();
}).catch(err => {
    console.log(err);
});

Because my DB Table contains more than 50 urls
Before, I was getting this error:
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit

After I added the line below. Its just killing my server and I have to do a manual reboot for my site to work again.
require('events').EventEmitter.prototype._maxListeners = 100;

I will appreciate any help rendered.

karlsnyder0 · 2021-06-12T14:12:06Z

Be aware that there are well known event listener leaks in several base Node modules. For example, net socket has this issue.

aslushnikov assigned JoelEinbinder Aug 29, 2017

aslushnikov added the bug label Aug 29, 2017

aslushnikov mentioned this issue Aug 30, 2017

Fix launcher to not leak event listeners #605

Merged

aslushnikov closed this as completed in #605 Aug 30, 2017

This was referenced Nov 28, 2017

Exit event limit and double ^C fix #1481

Closed

Unlimited browser instances and ^c fix. #1528

Closed

lukecoy mentioned this issue Apr 6, 2018

[CLEANUP beta] #16391 Cleaning up the test output emberjs/ember.js#16470

Merged

ernsheong mentioned this issue Jun 16, 2018

EventEmitter memory leak detected. 11 exit listeners added GoogleChromeLabs/pptraas.com#36

Closed

dennismuench mentioned this issue Nov 19, 2018

puppeteer: MaxListenersExceededWarning ycmjason/svg-to-img#2

Closed

moog16 mentioned this issue Nov 29, 2018

fix(infrastructure): memoize puppeteer browser instance during screenshot test suite material-components/material-components-web-react#480

Closed

hashtafak mentioned this issue Jan 14, 2019

MaxListenersExceededWarning: Possible EventEmitter memory leak detected thomasdondorf/puppeteer-cluster#61

Open

upugo-dev mentioned this issue Jan 29, 2019

browser.close from puppeteer.connect() not closing Chromium #561

Closed

thescientist13 mentioned this issue Aug 6, 2019

MaxListenersExceededWarning: Possible EventEmitter memory leak detected ProjectEvergreen/greenwood#141

Closed

5 tasks

jasonbyrne mentioned this issue Jun 12, 2021

MaxListenersExceededWarning even with limit set at 10 or below flagpolejs/flagpole#95

Closed

mgifford mentioned this issue Oct 6, 2021

Error: Protocol error (Page.navigate): Cannot navigate to invalid URL #6002

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EventEmitter memory leak detected. 11 exit listeners added #594

EventEmitter memory leak detected. 11 exit listeners added #594

selfrefactor commented Aug 29, 2017 •

edited

Loading

selfrefactor commented Aug 29, 2017

domarmstrong commented Aug 29, 2017

ebidel commented Aug 29, 2017

aslushnikov commented Aug 30, 2017

selfrefactor commented Aug 30, 2017 •

edited

Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017 •

edited

Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017 •

edited

Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017

selfrefactor commented Aug 30, 2017

stereobooster commented Mar 12, 2018

intellix commented Mar 14, 2018

ankur-21 commented Jan 18, 2020

ndotie commented Aug 14, 2020 •

edited

Loading

Mutale85 commented May 9, 2021

karlsnyder0 commented Jun 12, 2021

EventEmitter memory leak detected. 11 exit listeners added #594

EventEmitter memory leak detected. 11 exit listeners added #594

Comments

selfrefactor commented Aug 29, 2017 • edited Loading

selfrefactor commented Aug 29, 2017

domarmstrong commented Aug 29, 2017

ebidel commented Aug 29, 2017

aslushnikov commented Aug 30, 2017

selfrefactor commented Aug 30, 2017 • edited Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017 • edited Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017 • edited Loading

aslushnikov commented Aug 30, 2017

domarmstrong commented Aug 30, 2017

selfrefactor commented Aug 30, 2017

stereobooster commented Mar 12, 2018

intellix commented Mar 14, 2018

ankur-21 commented Jan 18, 2020

ndotie commented Aug 14, 2020 • edited Loading

Mutale85 commented May 9, 2021

karlsnyder0 commented Jun 12, 2021

selfrefactor commented Aug 29, 2017 •

edited

Loading

selfrefactor commented Aug 30, 2017 •

edited

Loading

domarmstrong commented Aug 30, 2017 •

edited

Loading

domarmstrong commented Aug 30, 2017 •

edited

Loading

ndotie commented Aug 14, 2020 •

edited

Loading