dead process - not restarted

[fredericosilva]

Hello,

Lately I'm having this problem with PM2

pm2 logs

┌───────────┬────┬──────┬───────┬────────┬─────────┬────────┬──────────────┬──────────┐
│ App name  │ id │ mode │ pid   │ status │ restart │ uptime │ memory       │ watching │
├───────────┼────┼──────┼───────┼────────┼─────────┼────────┼──────────────┼──────────┤
│ app_7     │ 7  │ fork │ 31378 │ online │ 0       │ 39h    │ 159.066 MB   │ disabled │
│ app_8     │ 9  │ fork │ 31404 │ online │ 0       │ 39h    │ 0 B          │ disabled │
│ app_9     │ 10 │ fork │ 31412 │ online │ 0       │ 39h    │ 34.688 MB    │ disabled │
│ app_12    │ 13 │ fork │ 31448 │ online │ 0       │ 39h    │ 0 B          │ disabled │
│ app_13    │ 14 │ fork │ 31464 │ online │ 0       │ 39h    │ 60.113 MB    │ disabled │
└───────────┴────┴──────┴───────┴────────┴─────────┴────────┴──────────────┴──────────┘

The app crash

cat ~/.pm2/logs/app_8-error-9.log

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: read ECONNRESET
    at errnoException (net.js:905:11)
    at TCP.onread (net.js:559:19)

ps -s 31404

  PID TTY          TIME CMD

The app_8 process died but was not restarted by pm2, if i try to manually restart it,

pm2 restart app_8

[PM2][ERROR] Process 9 not found

tail ~/.pm2/pm2.log

...
2015-03-16 10:25:33: Process with pid 31404 killed
2015-03-16 10:25:33: Process with pid 31404 already exists

pm2 logs

┌───────────┬────┬──────┬───────┬─────────┬─────────┬────────┬──────────────┬──────────┐
│ App name  │ id │ mode │ pid   │ status  │ restart │ uptime │ memory       │ watching │
├───────────┼────┼──────┼───────┼─────────┼─────────┼────────┼──────────────┼──────────┤
│ app_7     │ 7  │ fork │ 31378 │ online  │ 0       │ 39h    │ 159.066 MB   │ disabled │
│ app_8     │ 9  │ fork │ 31404 │ stopped │ 1       │ 39h    │ 0 B          │ disabled │
│ app_9     │ 10 │ fork │ 31412 │ online  │ 0       │ 39h    │ 34.688 MB    │ disabled │
│ app_12    │ 13 │ fork │ 31448 │ online  │ 0       │ 39h    │ 0 B          │ disabled │
│ app_13    │ 14 │ fork │ 31464 │ online  │ 0       │ 39h    │ 60.113 MB    │ disabled │
└───────────┴────┴──────┴───────┴─────────┴─────────┴────────┴──────────────┴──────────┘

Any idea how to debug this?

Thank you

[beatjoerg]

I see the same problem also.

[neonstalwart]

i think this is a duplicate of #1068

[jshkurti]

It looks like the process exited in an unusual way and PM2 didn't receive the 'exit' signal.
I just released PM2 0.12.8 in which I have refactored a bit the stop/restart procedure.
Can you guys give it a try ?

[jshkurti]

Waiting confirmation by @fredericosilva before closing the issue.

[fredericosilva]

@jshkurti It seems to be fixed, thank you.

[fredericosilva]

@jshkurti it happened again

Now the manual restart works fine. How can I help you to debug this?

[jshkurti]

Show me the ~/.pm2/pm2.log file at the time when the bug happenned please :)

[fredericosilva]

Here it goes,

2015-03-17 17:58:03: App name:app id:9 exited with code 8
2015-03-17 17:58:03: Starting execution sequence in -fork mode- for app name:app id:9
2015-03-17 17:58:03: App name:app id:9 online
2015-03-18 00:00:01: Reloading logs...
2015-03-18 00:00:01: Reloading logs for process id 0
2015-03-18 00:00:01: Reloading logs for process id 1
2015-03-18 00:00:01: Reloading logs for process id 2
2015-03-18 00:00:01: Reloading logs for process id 3
2015-03-18 00:00:01: Reloading logs for process id 4
2015-03-18 00:00:01: Reloading logs for process id 5
2015-03-18 00:00:01: Reloading logs for process id 6
2015-03-18 00:00:01: Reloading logs for process id 7
2015-03-18 00:00:01: Reloading logs for process id 8
2015-03-18 00:00:01: Reloading logs for process id 9
2015-03-18 00:00:01: Reloading logs for process id 10
2015-03-18 00:00:01: Reloading logs for process id 11
2015-03-18 00:00:01: Reloading logs for process id 12
2015-03-18 00:00:01: Reloading logs for process id 13
2015-03-18 00:00:01: Reloading logs for process id 14
2015-03-18 18:34:56: Stopping app:app id:9
2015-03-18 18:34:56: Process with pid 27319 killed
2015-03-18 18:34:56: Starting execution sequence in -fork mode- for app name:app id:9
2015-03-18 18:34:56: App name:app id:9 online

My manual restart was at 2015-03-18 18:34:56

Thanks

[jshkurti]

Ok i got it, I guess you're using 0.12.8.
Do npm i -g git://github.com/Unitech/PM2#development and let me know tomorrow if it still happens ;)
But I think it won't :)

dc010d7

[fredericosilva]

It's running, I'll keep an eye on that. tks :)

[fredericosilva]

@jshkurti it seems to be fixed now, tks :)