MaxListenersExceededWarning with @pm2/io and express

[blackshadev]

Hi I am running an small app http server with packages like express , pg and @pm/io. After I moved from pmx to @pm/io I start getting an MaxListenersExceededWarning . I installed a plugin which will throw on the MaxListenersExceededWarning , you can find the error stack below. After I removed the @pm/io init call everything worked fine without any MaxListenersExceededWarnings.

If you need more information or if I did something wrong myself, let me know.

Versions

@pm2/io: ^4.2.5

Initialization code

import * as pmx from "@pm2/io";
try {
    pmx.init({
        metrics: {
            http: true,
            v8: true,
            eventLoop: true,
            network: true,
        },
    });
} catch(err) {
   console.error("PMX init failed", err);
}

Error stack

Error: Too many listeners of type "data" added to EventEmitter. Max is 10 and we've added 12.
    at Socket.addListener (C:\Users\Vincent\Git\divecomputer\divingserver\node_modules\max-listeners-exceeded-warning\index.js:10:19)
    at Socket.Readable.on (_stream_readable.js:826:35)
    at Socket.<anonymous> (C:\Users\Vincent\Git\divecomputer\divingserver\node_modules\@pm2\io\build\main\metrics\network.js:78:26)
    at flow (_stream_readable.js:957:34)
    at emitReadable_ (_stream_readable.js:552:3)
    at onEofChunk (_stream_readable.js:517:7)
    at readableAddChunk (_stream_readable.js:237:5)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:125:12)
Error: Too many listeners of type "data" added to EventEmitter. Max is 10 and we've added 12.
    at Socket.addListener (C:\Users\Vincent\Git\divecomputer\divingserver\node_modules\max-listeners-exceeded-warning\index.js:10:19)
    at Socket.Readable.on (_stream_readable.js:826:35)
    at Socket.<anonymous> (C:\Users\Vincent\Git\divecomputer\divingserver\node_modules\@pm2\io\build\main\metrics\network.js:78:26)
    at flow (_stream_readable.js:957:34)
    at emitReadable_ (_stream_readable.js:552:3)
    at onEofChunk (_stream_readable.js:517:7)
    at readableAddChunk (_stream_readable.js:237:5)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:125:12)

[flipace]

We just switched from pmx to @pm2/io and started seeing these as well:

2019-08-21T12:22:25.262Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.264Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.266Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.266Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.266Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.267Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.267Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:25.267Z error #1-epic_neumann: (node:100) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit
2019-08-21T12:22:27.116Z error #2-fervent_poincare: (node:107) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 data listeners added. Use emitter.setMaxListeners() to increase limit

[Pusikas]

If I use this:

const io = require("@pm2/io").init({
  metrics: {
    network: true
  },
});

Then I get this kind of warning. But if I dnot not use init:

const io = require("@pm2/io");

Then not. I have not done a lot of testing yet, but it's either the collection of notwork metrics, or using init.

[flipace]

If I use this:

const io = require("@pm2/io").init({
  metrics: {
    network: true
  },
});

Then I get this kind of warning. But if I dnot not use init:

const io = require("@pm2/io");

Then not. I have not done a lot of testing yet, but it's either the collection of notwork metrics, or using init.

for us, setting network: false also prevents this issue.

[soonderella9]

if i want to collect network related metrics, what should i do? is there any way to use network: true option with no warning?

[Unitech]

Basically we only wrap the 'read' and 'write' function of the 'net' module
https://github.com/keymetrics/pm2-io-apm/blob/master/src/metrics/network.ts#L124

Maybe just increasing the number of listeners allowed via emitter.setMaxListeners() would suffice. If anyone has a clue about that let me know

[lkho]

Hi everyone,

@pm2/io: 4.3.5

I found that the warning is from the 'read' instead of 'write', as writing does not add listeners.

pm2-io-apm/src/metrics/network.ts

Lines 89 to 98 in 7b09b73

	shimmer.wrap(netModule.Socket.prototype, 'read', function (original) {
	return function () {
	this.on('data', (data) => {
	if (typeof data.length === 'number') {
	downloadMeter.mark(data.length)
	}
	})
	return original.apply(this, arguments)
	}
	})

this.on('data') is called on EVERY call of this.read(), and is not removed ever after until the socket is GCed. I think this caused leak and also wrong metric calculation.

If you have some sockets that are long lasting, keep alived for a long time, in which it becomes idle and readable and idle for many times, the read() method will be called many times. In just a simple keep alived http connection, it may accumulate hundreds to thousands of data listeners like this.