Memory leak and process.exit issue

[chrisblossom]

There is a fairly large memory leak that is happening. I am unsure of the root cause, but you can see the memory leak in action here:
https://github.com/chrisblossom/memory-leak-example

There is also an issue when I have this in my development environment. It stops my node process from restarting and end up with this error:

27 Jul 10:56:48 - uncaughtException: c8841060f9114c6895e59c61d194b312
{ message: 'uncaughtException',
  error:
   { [Error: listen EADDRINUSE :::3002]
     code: 'EADDRINUSE',
     errno: 'EADDRINUSE',
     syscall: 'listen',
     address: '::',
     port: 3002 } }

Ideas?

Thank you!

[megastef]

Thx. Will have look into it. I think your app does not call process.exit() thats why it not terminates. I will look into exit handlers or timers keeping the process alive.

[chrisblossom]

I wanted to see if the issue was being caused by logsene-js so I added it to chrisblossom/memory-leak-example

I could only get it to complete the test once or twice, which it didn't look like the memory leak was caused there, but it does error out the majority of the time with the following message:

Error: Uncaught, unspecified "error" event. ([object Object])
    at Logsene.emit (events.js:144:17)
    at Request._callback (/Users/chris/github/memory-leak-example/node_modules/logsene-js/index.js:235:12)
    at Request.self.callback (/Users/chris/github/memory-leak-example/node_modules/request/request.js:187:22)
    at emitTwo (events.js:87:13)
    at Request.emit (events.js:172:7)
    at Request.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:1044:10)
    at emitOne (events.js:77:13)
    at Request.emit (events.js:169:7)
    at IncomingMessage.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:965:12)
    at emitNone (events.js:72:20)

Here is how I start and restart my development environment:
https://gist.github.com/chrisblossom/c67268ba865d7dd00b0a8b38dc557a43

This would be similar to using something like nodemon.

Let me know if you need any other information.

[chrisblossom]

I believe I found the issue. The arrays created on lines 19 and 20 have the result of each log pushed on here. When the log is processed they are never removed so this.errorOutput and this.writeOutput grow indefinitely.

[megastef]

@chrisblossom great find! It looks Winston Memory transport was the "blueprint" https://github.com/winstonjs/winston/search?utf8=%E2%9C%93&q=errorOutput for the initial winston-logsene implementation and it is a left over and both arrays are not used by the winston framework (searched in winston repository ...).

[chrisblossom]

176cf50 fixed the memory leak but the exit issue is still present for me. I will try and do more research what is causing it soon.

Any thoughts on the logsene-js error above? Were you able to reproduce?

[megastef]

Please note exit can be delayed until logs are flushed to Logsene, but max. 10 seconds (after that a process manager like Linux upstart would kill the process with kill -9). The error was that process.exit() was not called when no logs got flushed (e.g. empty buffer). Fixed in e4be2e8 and released as winston-logsene@1.1.11.

[megastef]

@chrisblossom RE. Error message

Error: Uncaught, unspecified "error" event. ([object Object])
    at Logsene.emit (events.js:144:17)
    at Request._callback (/Users/chris/github/memory-leak-example/node_modules/logsene-js/index.js:235:12)
    at Request.self.callback (/Users/chris/github/memory-leak-example/node_modules/request/request.js:187:22)
    at emitTwo (events.js:87:13)
    at Request.emit (events.js:172:7)
    at Request.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:1044:10)
    at emitOne (events.js:77:13)
    at Request.emit (events.js:169:7)
    at IncomingMessage.<anonymous> (/Users/chris/github/memory-leak-example/node_modules/request/request.js:965:12)
    at emitNone (events.js:72:20)

Logsene-js emits an error event when http requests to Logsene fail, or when you use wrong token... But winston-logsene registers for this error event and emits the error to winston-logsene users https://github.com/sematext/winston-logsene/blob/master/lib/index.js#L44
Do you have an error handler for winston-logsene in your code? If not its clear that it leads to uncaught errors.

So please add something like this to you code:

  winstonLogseneLogger.on('error', function () { 
   console.error('error in winston-logsene', arguments) 
  })

[megastef]

@chrisblossom I assume we can close this issue, right?

[chrisblossom]

@megastef yep. Memory leak issue is fixed. Thanks for the help!