Pm2 reload doesn't send SIGINT signal on cluster mode

[mmenoxx]

If reporting a bug, please use the following template.

Expected behaviour

When using the command:
pm2 reload idOfPm2Process

on a cluster of pm2 processes, pm2 should send a SIGINT signal to the process that we can catch in a listener as:
process.on('SIGINT', function() {
console.log('SIGINT signal catched')
});
...

Actual behaviour

It seems no signal is sent to the process because no log message is sent to the output/console.
If i do the same thing on a process in fork mode it works perfecrly
...

Steps to reproduce

...

Software versions used

OS         :
node.js    :5.12.0
PM2        :1.1.3

PM2 Log output

Use the command: tail --lines 50 ~/.pm2/pm2.log

...

cf CONTRIBUTING.md for more informations

[Unitech]

Just tried here, looks like the SIGINT is catched by the application in cluster mode, but the log message does not show via pm2 logs. Tests have been added to verify this behavior, now inspecting pm2 logs

[Unitech]

The way the Reload feature is implemented today does not permit to output log on exit (when inside the SIGINT callback).
It's due to the nature of how the Reload mechanism is implemented, when a Reload happens, we will disconnect the process (to stop accepting new connections) that in the same time stops the IPC channel between PM2 and the application. So we cannot catch that messages once process is disconnected.

[mmenoxx]

I think this is an important issue because when you use the reload feature, with a SIGINT callback, it is because you want to do some gracefull shutdown ,allowing the application to finish any work was initiated by requests that came into the application just before the graceful shutdown. So usually we put a timer (for example 30 seconds), so that the application can finish calling whatever callback is still in the event queue. During that time we have all logs that we putted everywhere in application code. This means that when we reload with pm2 we lost all logs for all application code that is executed during the 30 seconds jobs. And this means also that in production we will lose a lot of logs

[Unitech]

Nope, all your logs are still stored in the files. This is a minor bug, it's only about the PM2 log command that does not display the log output. These logs are stored anyway in the log files and the SIGINT callback is called for gracefull shutdown.

[mmenoxx]

Ah ok if it is like you say it is perfect, i will check this behaviour my side

[Unitech]

Sorry my explanation was not so crystal clear :) Anyway, we plan to create a new TCP LB in the following months

[mmenoxx]

But how can Pm2 continue to write logs (to files) when we disconnect the worker from the cluster master process?

[vmarchaud]

@markmenox Actually the problem come from a weird behavior on cluster mode, we added a documentation here to describe the problem. We can't really make deeper inspection so feel free to try find an explanation.

[Lalem001]

Regarding cluster mode and SIGINT:

Each worker in the cluster would receive a beforeExit and exit events AFTER it has been disconnected (no SIGINT). If you log using FS, instead of console, you will see that intercepting exit can give you a graceful shutdown. Unfortunately, as mentioned, disconnect would have disconnected all peers before even beforeExit rendering a graceful shutdown moot.

Short of getting a beforeDisconnect event implemented in node's source, or the ability to override the _disconnect method, I have found no way for a worker to gracefully shutdown before being cut off from the world.

[vmarchaud]

@Lalem001 Worker can receive SIGINT, see my writing in the documentation, the weird behavior is that depending of the code running, it will or will not receive the SIGINT.

[Unitech]

SIGINT handle now works as expected (it will be called every time the process get exited).

To try it:

$ npm install pm2@next -g
$ pm2 update

[Unitech]

Patch available on pm2@2.1.4 (main):

$ npm install pm2 -g
$ pm2 update