Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running reloadLogs while run as a non-root user causes process to not release and hang #4781

Closed
mbrooks opened this issue Jul 15, 2020 · 4 comments
Closed

Running reloadLogs while run as a non-root user causes process to not release and hang #4781

mbrooks opened this issue Jul 15, 2020 · 4 comments
Labels
stale

Comments

@mbrooks
Copy link
Contributor

mbrooks commented Jul 15, 2020 •
edited
Loading

What's going wrong?

Running reloadLogs while run as a non-root user causing process to not release and hang.

How could we reproduce this issue?

Reproducible in the FreePBX 15 Distro running the pm2 module version 15.0.3.7. Upgraded module manually to PM2 4.4.0 and the issue still persists.

Supporting information

Logs:

2020-07-14T20:57:49: PM2 log: App [ucp:1] starting in -fork mode-
2020-07-14T20:57:50: PM2 log: App [core-fastagi:0] online
2020-07-14T20:57:50: PM2 log: App [ucp:1] online
2020-07-14T20:57:50: PM2 log: App [xmpp:2] starting in -fork mode-
2020-07-14T20:57:50: PM2 log: App [xmpp:2] online
2020-07-14T20:58:29: PM2 log: Reloading logs...
2020-07-14T20:58:29: PM2 log: Reloading logs for process id 0
2020-07-14T20:58:29: PM2 log: Reloading logs for process id 1
2020-07-14T20:58:29: PM2 log: Reloading logs for process id 2
2020-07-14T20:58:32: PM2 log: Reloading logs...
2020-07-14T20:58:32: PM2 log: Reloading logs for process id 0
2020-07-14T20:58:32: PM2 log: ===============================================================================
2020-07-14T20:58:32: PM2 log: --- PM2 global error caught ---------------------------------------------------
2020-07-14T20:58:32: PM2 log: Time                 : Tue Jul 14 2020 20:58:32 GMT+0000 (UTC)
2020-07-14T20:58:32: PM2 error: Cannot read property 'destroy' of undefined
2020-07-14T20:58:32: PM2 error: TypeError: Cannot read property 'destroy' of undefined
    at ChildProcess.cspr._reloadLogs (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:288:23)
    at /var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ActionMethods.js:651:19
    at Array.forEach (<anonymous>)
    at God.reloadLogs (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ActionMethods.js:633:16)
    at Server.onmessage (/var/www/html/admin/modules/pm2/node/node_modules/pm2-axon-rpc/lib/server.js:104:6)
    at emitTwo (events.js:126:13)
    at RepSocket.emit (events.js:214:7)
    at Parser.<anonymous> (/var/www/html/admin/modules/pm2/node/node_modules/pm2-axon/lib/sockets/rep.js:51:15)
    at emitOne (events.js:116:13)
    at Parser.emit (events.js:211:7)
2020-07-14T20:58:32: PM2 log: ===============================================================================
2020-07-14T20:58:32: PM2 error: [PM2] Resurrecting PM2
2020-07-14T20:58:32: PM2 log: Stopping app:core-fastagi id:0
2020-07-14T20:58:32: PM2 log: Stopping app:ucp id:1
2020-07-14T20:58:32: PM2 log: Stopping app:xmpp id:2
2020-07-14T20:58:32: PM2 error: Trace: TypeError: Cannot read property 'destroy' of undefined
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:278:25)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
    at Object.God.logAndGenerateError (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/Methods.js:34:15)
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:283:26)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
2020-07-14T20:58:32: PM2 log: App [core-fastagi:0] exited with code [0] via signal [SIGINT]
2020-07-14T20:58:32: PM2 error: Trace: TypeError: Cannot read property 'destroy' of undefined
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:278:25)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
    at Object.God.logAndGenerateError (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/Methods.js:34:15)
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:283:26)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
2020-07-14T20:58:32: PM2 log: App [ucp:1] exited with code [0] via signal [SIGINT]
2020-07-14T20:58:32: PM2 error: Trace: TypeError: Cannot read property 'destroy' of undefined
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:278:25)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
    at Object.God.logAndGenerateError (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/Methods.js:34:15)
    at ChildProcess.forkClose (/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/God/ForkMode.js:283:26)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:131:20)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)

--- PM2 report ----------------------------------------------------------------
Date                 : Tue Jul 14 2020 22:27:34 GMT+0000 (UTC)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 4.4.0
node version         : 8.16.0
node path            : ./pm2
argv                 : /usr/bin/node,/var/www/html/admin/modules/pm2/node/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : root
uid                  : 0
gid                  : 0
uptime               : 1min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 4.4.0
node version         : 8.16.0
node path            : ./pm2
argv                 : /usr/bin/node,/var/www/html/admin/modules/pm2/node/node_modules/.bin/pm2,report
argv0                : node
user                 : root
uid                  : 0
gid                  : 0
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz
cpus nb              : 1
freemem              : 151498752
totalmem             : 1927471104
home                 : /root
===============================================================================
--- PM2 list -----------------------------------------------
┌─────┬───────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name      │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
└─────┴───────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
/root/.pm2/pm2.log last 20 lines:
PM2        | 2020-07-14T22:25:36: PM2 log: ===============================================================================
PM2        | 2020-07-14T22:25:36: PM2 log: --- New PM2 Daemon started ----------------------------------------------------
PM2        | 2020-07-14T22:25:36: PM2 log: Time                 : Tue Jul 14 2020 22:25:36 GMT+0000 (UTC)
PM2        | 2020-07-14T22:25:36: PM2 log: PM2 version          : 4.4.0
PM2        | 2020-07-14T22:25:36: PM2 log: Node.js version      : 8.16.0
PM2        | 2020-07-14T22:25:36: PM2 log: Current arch         : x64
PM2        | 2020-07-14T22:25:36: PM2 log: PM2 home             : /root/.pm2
PM2        | 2020-07-14T22:25:36: PM2 log: PM2 PID file         : /root/.pm2/pm2.pid
PM2        | 2020-07-14T22:25:36: PM2 log: RPC socket file      : /root/.pm2/rpc.sock
PM2        | 2020-07-14T22:25:36: PM2 log: BUS socket file      : /root/.pm2/pub.sock
PM2        | 2020-07-14T22:25:36: PM2 log: Application log path : /root/.pm2/logs
PM2        | 2020-07-14T22:25:36: PM2 log: Worker Interval      : 30000
PM2        | 2020-07-14T22:25:36: PM2 log: Process dump file    : /root/.pm2/dump.pm2
PM2        | 2020-07-14T22:25:36: PM2 log: Concurrent actions   : 2
PM2        | 2020-07-14T22:25:36: PM2 log: SIGTERM timeout      : 1600
PM2        | 2020-07-14T22:25:36: PM2 log: ===============================================================================
@mbrooks mbrooks changed the title Running reloadLogs while run as a non-root user causing process to not release and hang Running reloadLogs while run as a non-root user causes process to not release and hang Jul 15, 2020
mbrooks added a commit to mbrooks/pm2 that referenced this issue Jul 15, 2020
@mbrooks
Prevent reloadLogs command from hanging
7a4664a 
Previously, when the pm2 reloadLogs command was running as a non-root
user, the process will not release and hangs. The reason for the hang
is that the forked process never closes because the object is empty
and doesn't contain the expected object. This commit checks if an
object exists first, before trying to execute methods on it.

Related to: Unitech#4781

Co-authored-by: Franck Danard <fdanard@sangoma.com>
@mbrooks mbrooks mentioned this issue Jul 15, 2020
Merged
mbrooks added a commit to mbrooks/pm2 that referenced this issue Jul 15, 2020
@mbrooks
Prevent reloadLogs command from hanging
08ae19e 
Previously, when the pm2 reloadLogs command was running as a non-root
user, the process will not release and hangs. The reason for the hang
is that the forked process never closes because the object is empty
and doesn't contain the expected object. This commit checks if an
object exists first, before trying to execute methods on it.

Related to: Unitech#4781

Co-authored-by: Franck Danard <fdanard@sangoma.com>
@mbrooks
Copy link
Contributor Author

mbrooks commented Jul 15, 2020

I submitted a patch for this, but unsure if it's the correct fix. It also seems to have broken tests which is concerning.

@stale
Copy link

stale bot commented Aug 14, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 14, 2020
@mbrooks
Copy link
Contributor Author

mbrooks commented Aug 20, 2020

I think it is a mistake to label this as stale, I'm still waiting on a response to my questions. I would like to help out if I can!

@Unitech
Copy link
Owner

Unitech commented Sep 29, 2020

published in pm2@4.5.0

npm install pm2@latest -g

@Unitech Unitech closed this as completed Sep 29, 2020
mendrix pushed a commit to mendrix/pm2 that referenced this issue Dec 29, 2020
@mbrooks @remz
Prevent reloadLogs command from hanging
82a3ad8 
Previously, when the pm2 reloadLogs command was running as a non-root
user, the process will not release and hangs. The reason for the hang
is that the forked process never closes because the object is empty
and doesn't contain the expected object. This commit checks if an
object exists first, before trying to execute methods on it.

Related to: Unitech#4781

Co-authored-by: Franck Danard <fdanard@sangoma.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mbrooks @Unitech