Processes are sometimes restarted after pm stop

[morkai]

Sometimes after pm2 stop all processes may be restarted. See this gist (10 running processes, after pm2 stop all the test-1 process is online and its restart count is 1).

You can see that when it works, the process enters God.handleExit() with status stopping and when it doesn't work the status is stopped, because sometimes God.stopProcessId() finishes before the process emits the close event.

Windows 8.1 64-bit, Node.js v0.10.35, pm2 0.12.6 & development.
Couldn't reproduce it on Ubuntu 14.

[jshkurti]

Does this solve the problem here ? :)

[morkai]

Can't make any process restart after pm2 stop all, so I think it's fixed!

But while spamming pm2 stop all & pm2 restart all I've accidentally executed pm2 restart all twice and got an error: [PM2][ERROR] Process 1 not found
When all processes are offline restart all works, when all are online, restart all fails with the above error (different process ID) like 70% of the time.

[jshkurti]

Strange, works like a charm on my pc :/
Can you try the very last version on development ?
And show some pm2 logs if it still doesn't work ? :)

[morkai]

https://gist.github.com/morkai/aad717dd1f5737d2894f
I did npm install -g Unitech/PM2#development before testing it (stop all was fixed after all ;)

[Unitech]

Could someone try the PM2 development version, there are some Windows fixes

$ npm install Unitech/pm2#development -g
$ pm2 update

[morkai]

Now on pm2 restart all processes are not killed ([Error: kill ESRCH]) but new ones are still spawned. So if there were 10 processes running, after restart all, there will be 20.
https://gist.github.com/morkai/43c6c1c91b5a67338436
(Windows 10)

EDIT:
The pid can not be killed error originates at line 245, which is process.kill(-(parseInt(pid)), 'SIGINT');.
If I run process.kill(-8112) from the Node's REPL then I get this error. Works without the minus sign.

[chiihuang]

+1 with @morkai
I found the similar issue on Ubuntu with the following in pm2.log after I executed $ pm2 restart app

2015-12-30 14:51:45: Stopping app:app id:28
e/deployer_punwave/source/app/worker.js: ../src/smalloc.cc:281: void node::smalloc::SliceOnto(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `start <= end' failed.
2015-12-30 14:51:45: App name:app id:28 disconnected
2015-12-30 14:51:45: 16121 pid can not be killed { [Error: kill ESRCH] code: 'ESRCH', errno: 'ESRCH', syscall: 'kill' }
2015-12-30 14:51:45: Starting execution sequence in -cluster mode- for app name:app id:28
2015-12-30 14:51:45: App name:app id:28 exited with code SIGABRT

Ubuntu 14.04
Node.js v0.12.7
NPM 2.11.3
PM2 v0.15.8

[Im-Victor]

+1
me too~

2016-01-07 16:02:16: Stopping app:ourselves-test-serv-app id:0
2016-01-07 16:02:16: App name:ourselves-test-serv-app id:0 disconnected
2016-01-07 16:02:16: 18621 pid can not be killed { [Error: kill ESRCH] code: 'ESRCH', errno: 'ESRCH', syscall: 'kill' }
2016-01-07 16:02:16: Starting execution sequence in -cluster mode- for app name:ourselves-test-serv-app id:0
2016-01-07 16:02:16: App name:ourselves-test-serv-app id:0 exited with code 100
2016-01-07 16:02:16: Starting execution sequence in -cluster mode- for app name:ourselves-test-serv-app id:0

[webuser@test71 ourselves]$ ps aux|grep www
webuser 18620 102 0.3 775460 111032 ? Rl 16:01 0:59 node /var/www/site/ourselves/bin/www
webuser 18655 73.0 0.2 748388 82632 ? Rl 16:02 0:01 node /var/www/site/ourselves/bin/www
webuser 18656 69.0 0.2 748352 82624 ? Rl 16:02 0:01 node /var/www/site/ourselves/bin/www
linux 2.6.32
Node.js v0.12.7
NPM 2.11.3
PM2 0.14.3

[morkai]

I've just restarted 10 processes over 1000 times without any issues (using this script).

pm2 ls at the beginning:

┌──────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────────────┬──────────┐
│ App name │ id │ mode │ pid   │ status │ restart │ uptime │ memory      │ watching │
├──────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────────────┼──────────┤
│ test-0   │ 0  │ fork │ 7360  │ online │ 0       │ 12s    │ 20.980 MB   │ disabled │
│ test-1   │ 1  │ fork │ 1092  │ online │ 0       │ 11s    │ 20.980 MB   │ disabled │
│ test-2   │ 2  │ fork │ 9460  │ online │ 0       │ 10s    │ 21.004 MB   │ disabled │
│ test-3   │ 3  │ fork │ 9664  │ online │ 0       │ 9s     │ 20.980 MB   │ disabled │
│ test-4   │ 4  │ fork │ 8820  │ online │ 0       │ 9s     │ 20.910 MB   │ disabled │
│ test-5   │ 5  │ fork │ 10212 │ online │ 0       │ 8s     │ 20.914 MB   │ disabled │
│ test-6   │ 6  │ fork │ 7132  │ online │ 0       │ 7s     │ 20.957 MB   │ disabled │
│ test-7   │ 7  │ fork │ 4648  │ online │ 0       │ 6s     │ 20.957 MB   │ disabled │
│ test-8   │ 8  │ fork │ 5276  │ online │ 0       │ 5s     │ 20.910 MB   │ disabled │
│ test-9   │ 9  │ fork │ 10764 │ online │ 0       │ 4s     │ 20.934 MB   │ disabled │
└──────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────────────┴──────────┘

pm2 ls after 1000 restarts:

┌──────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────────────┬──────────┐
│ App name │ id │ mode │ pid   │ status │ restart │ uptime │ memory      │ watching │
├──────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────────────┼──────────┤
│ test-0   │ 0  │ fork │ 11304 │ online │ 1000    │ 1s     │ 20.645 MB   │ disabled │
│ test-1   │ 1  │ fork │ 976   │ online │ 1000    │ 1s     │ 20.660 MB   │ disabled │
│ test-2   │ 2  │ fork │ 6396  │ online │ 1000    │ 1s     │ 20.715 MB   │ disabled │
│ test-3   │ 3  │ fork │ 508   │ online │ 1000    │ 1s     │ 20.656 MB   │ disabled │
│ test-4   │ 4  │ fork │ 6204  │ online │ 1000    │ 1s     │ 20.613 MB   │ disabled │
│ test-5   │ 5  │ fork │ 1520  │ online │ 1000    │ 1s     │ 20.309 MB   │ disabled │
│ test-6   │ 6  │ fork │ 2020  │ online │ 1000    │ 0s     │ 20.340 MB   │ disabled │
│ test-7   │ 7  │ fork │ 8668  │ online │ 1000    │ 0s     │ 20.371 MB   │ disabled │
│ test-8   │ 8  │ fork │ 12132 │ online │ 1000    │ 0s     │ 20.270 MB   │ disabled │
│ test-9   │ 9  │ fork │ 5660  │ online │ 1000    │ 0s     │ 20.387 MB   │ disabled │
└──────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────────────┴──────────┘

Windows 10
Node.js 5.1.1
PM2 1.0.0

[Unitech]

closing in favor of #2061