Amazon EC2 : saved PM2 configuration is not recognized at startup (when running pm2 in user mode) #1100

Closed
ObjectIsAdvantag opened this Issue Mar 16, 2015 · 12 comments

Comments

6 participants
@ObjectIsAdvantag

Scenario

// create an EC2 instance, install node, install pm2
// create an app (install sails for example)
// add your app to pm2
[ec2-user]$ pm2 start app.js -n <myapp> -x -- --prod
// configure PM2 startup as running in user mode (-u <user>)
[ec2-user]$ sudo env PATH=$PATH:/usr/bin pm2 startup amazon -u ec2-user
// save configuration
[ec2-user]$ pm2 save
=> OK :  writes current PM2 configuration (my app) into  ec2-user home dir, 
=> OK the file ~/.pm2/dump.pm2 is generated
// go to your AWS console, reboot your EC2 instance
=> FAILS : after restart, <myapp> is not started

@ObjectIsAdvantag ObjectIsAdvantag changed the title from Amazon EC2 : saved PM2 configuration is not recognized at startup to Amazon EC2 : saved PM2 configuration is not recognized at startup (when running pm2 in user mode) Mar 16, 2015

@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 16, 2015

Analysis

  • the startup script (see /etc/init.d/pm2-init.sh) does not use the USER variable to build PM2_HOME, but HOME_PATH which make it point to "/root/.pm2"
  • "/root/.pm2" contains an empty dump.pm2 file, as the configuration was saved in ec2-user home dir

Analysis

  • the startup script (see /etc/init.d/pm2-init.sh) does not use the USER variable to build PM2_HOME, but HOME_PATH which make it point to "/root/.pm2"
  • "/root/.pm2" contains an empty dump.pm2 file, as the configuration was saved in ec2-user home dir
@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 16, 2015

Fix proposal

  • modify the "pm2-init-amazon.sh" script to initialize the PM2_HOME env variable depending on the user specified at pm2 startup configuration (-u ec2-user for Amazon EC2)

Fix proposal

  • modify the "pm2-init-amazon.sh" script to initialize the PM2_HOME env variable depending on the user specified at pm2 startup configuration (-u ec2-user for Amazon EC2)
@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 16, 2015

Here it is

NAME=pm2
PM2=%PM2_PATH%
USER=%USER%

export PATH=$PATH:%NODE_PATH%
if [[ "$USER" == "root" ]]; then
    export PM2_HOME="%HOME_PATH%"
else
    export PM2_HOME=/home/$USER/.pm2
fi

lockfile="/var/lock/subsys/pm2-init.sh"

=> restarting the Amazon EC2 instance

Here it is

NAME=pm2
PM2=%PM2_PATH%
USER=%USER%

export PATH=$PATH:%NODE_PATH%
if [[ "$USER" == "root" ]]; then
    export PM2_HOME="%HOME_PATH%"
else
    export PM2_HOME=/home/$USER/.pm2
fi

lockfile="/var/lock/subsys/pm2-init.sh"

=> restarting the Amazon EC2 instance

@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 16, 2015

Argggg, I am now getting the EACCES error

> restart
> login as ec2-user
> pm2 list 
=> EACCES error

Several opened issues mention this error => seems I am not the first one to take this path...
Will need more time ...

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect EACCES
    at errnoException (net.js:905:11)
    at Object.afterConnect [as oncomplete] (net.js:896:19)

Argggg, I am now getting the EACCES error

> restart
> login as ec2-user
> pm2 list 
=> EACCES error

Several opened issues mention this error => seems I am not the first one to take this path...
Will need more time ...

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect EACCES
    at errnoException (net.js:905:11)
    at Object.afterConnect [as oncomplete] (net.js:896:19)
@soyuka

This comment has been minimized.

Show comment
Hide comment
@soyuka

soyuka Mar 16, 2015

Collaborator

I don't have a direct access right now but search through issues I remember
seeing something like this with a solution ;).
Le lun. 16 mars 2015 à 18:40, Stève Sfartz notifications@github.com a
écrit :

Argggg, I am now getting the EACCES error
Several opened issues, seems I am not the first one to take this path...
Will need more time ...

events.js:72
throw er; // Unhandled 'error' event
^
Error: connect EACCES
at errnoException (net.js:905:11)
at Object.afterConnect as oncomplete


Reply to this email directly or view it on GitHub
#1100 (comment).

Collaborator

soyuka commented Mar 16, 2015

I don't have a direct access right now but search through issues I remember
seeing something like this with a solution ;).
Le lun. 16 mars 2015 à 18:40, Stève Sfartz notifications@github.com a
écrit :

Argggg, I am now getting the EACCES error
Several opened issues, seems I am not the first one to take this path...
Will need more time ...

events.js:72
throw er; // Unhandled 'error' event
^
Error: connect EACCES
at errnoException (net.js:905:11)
at Object.afterConnect as oncomplete


Reply to this email directly or view it on GitHub
#1100 (comment).

@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 20, 2015

Diagnostic :

  • the EACCES error is due to the fact that 3 files are owned by root in the ec2-user .pm2 directory
    • interactor.sock
    • pub.sock
    • rpc.sock

[ec2-user ~]$ ls -alg .pm2/

drwxrwxr-x  4 ec2-user   4096 Mar 16 17:36 .
drwx------ 11 ec2-user   4096 Mar 16 15:08 ..
-rw-rw-r--  1 ec2-user    216 Mar 16 17:36 agent.json5
-rw-rw-r--  1 ec2-user 513021 Mar 20 21:06 agent.log
-rw-rw-r--  1 ec2-user      4 Mar 16 17:36 agent.pid
-rw-rw-r--  1 ec2-user   1478 Mar 15 14:58 conf.js
-rw-rw-r--  1 ec2-user      2 Mar 16 17:10 dump.pm2
srwxr-xr-x  1 root          0 Mar 16 17:36 interactor.sock
drwxrwxr-x  2 ec2-user   4096 Mar 16 13:00 logs
-rw-rw-r--  1 ec2-user      2 Mar 15 14:58 module_conf.json
drwxrwxr-x  2 ec2-user   4096 Mar 16 17:10 pids
-rw-rw-r--  1 ec2-user  28320 Mar 16 17:36 pm2.log
-rw-r--r--  1 root          4 Mar 16 17:36 pm2.pid
srwxr-xr-x  1 root          0 Mar 16 17:36 pub.sock
srwxr-xr-x  1 root          0 Mar 16 17:36 rpc.sock

Diagnostic :

  • the EACCES error is due to the fact that 3 files are owned by root in the ec2-user .pm2 directory
    • interactor.sock
    • pub.sock
    • rpc.sock

[ec2-user ~]$ ls -alg .pm2/

drwxrwxr-x  4 ec2-user   4096 Mar 16 17:36 .
drwx------ 11 ec2-user   4096 Mar 16 15:08 ..
-rw-rw-r--  1 ec2-user    216 Mar 16 17:36 agent.json5
-rw-rw-r--  1 ec2-user 513021 Mar 20 21:06 agent.log
-rw-rw-r--  1 ec2-user      4 Mar 16 17:36 agent.pid
-rw-rw-r--  1 ec2-user   1478 Mar 15 14:58 conf.js
-rw-rw-r--  1 ec2-user      2 Mar 16 17:10 dump.pm2
srwxr-xr-x  1 root          0 Mar 16 17:36 interactor.sock
drwxrwxr-x  2 ec2-user   4096 Mar 16 13:00 logs
-rw-rw-r--  1 ec2-user      2 Mar 15 14:58 module_conf.json
drwxrwxr-x  2 ec2-user   4096 Mar 16 17:10 pids
-rw-rw-r--  1 ec2-user  28320 Mar 16 17:36 pm2.log
-rw-r--r--  1 root          4 Mar 16 17:36 pm2.pid
srwxr-xr-x  1 root          0 Mar 16 17:36 pub.sock
srwxr-xr-x  1 root          0 Mar 16 17:36 rpc.sock
@ObjectIsAdvantag

This comment has been minimized.

Show comment
Hide comment
@ObjectIsAdvantag

ObjectIsAdvantag Mar 20, 2015

My guess :

  • the PM2 daemon is launched as root at instance startup whereas it should be launched as ec2-user
  • wierd : the PM2 daemon is launched 3 times (twice as root and once as ec2-user)

Next :

  • reinstall PM2 on a fresh EC2 instance and check who launches the PM2 process after PM2 boot configuration
[ec2-user]$ sudo env PATH=$PATH:/usr/bin pm2 startup amazon -u ec2-user
// save configuration
[ec2-user]$ pm2 save

My guess :

  • the PM2 daemon is launched as root at instance startup whereas it should be launched as ec2-user
  • wierd : the PM2 daemon is launched 3 times (twice as root and once as ec2-user)

Next :

  • reinstall PM2 on a fresh EC2 instance and check who launches the PM2 process after PM2 boot configuration
[ec2-user]$ sudo env PATH=$PATH:/usr/bin pm2 startup amazon -u ec2-user
// save configuration
[ec2-user]$ pm2 save
@spidie

This comment has been minimized.

Show comment
Hide comment
@spidie

spidie Mar 26, 2015

Same problem for me on Ubuntu. I run my pm2 as the "web" user and rely on pm2 config stored in /home/web/.pm2. However, when I run "pm2 startup ubuntu -u web" as root my script has PM2_HOME set as /root/pm2 and my config is not found.

ObjectIsAdvantag's fix proposal looks reasonable to me.

spidie commented Mar 26, 2015

Same problem for me on Ubuntu. I run my pm2 as the "web" user and rely on pm2 config stored in /home/web/.pm2. However, when I run "pm2 startup ubuntu -u web" as root my script has PM2_HOME set as /root/pm2 and my config is not found.

ObjectIsAdvantag's fix proposal looks reasonable to me.

@EgoAleSum

This comment has been minimized.

Show comment
Hide comment
@EgoAleSum

EgoAleSum Apr 8, 2015

I found the solution. The script for centos and amazon linux are very similar, but the latter had not been updated to switch to "ec2-user" before any command.

Here's my solution:

  1. Instead of using "amazon" with pm2 startup, use "centos"
  2. Edit the /etc/init.d/pm2-init.sh script to replace /root with /home/ec2-user

I found the solution. The script for centos and amazon linux are very similar, but the latter had not been updated to switch to "ec2-user" before any command.

Here's my solution:

  1. Instead of using "amazon" with pm2 startup, use "centos"
  2. Edit the /etc/init.d/pm2-init.sh script to replace /root with /home/ec2-user
@kelp404

This comment has been minimized.

Show comment
Hide comment
@kelp404

kelp404 Sep 27, 2015

It is work!

nice

kelp404 commented Sep 27, 2015

It is work!

nice

@Unitech

This comment has been minimized.

Show comment
Hide comment
@Unitech

Unitech Dec 6, 2016

Owner

may worth checking new upstart system. I recommend systemd though (available on ubuntu 16.x by default)

#2559

Owner

Unitech commented Dec 6, 2016

may worth checking new upstart system. I recommend systemd though (available on ubuntu 16.x by default)

#2559

@Unitech

This comment has been minimized.

Show comment
Hide comment
@Unitech

Unitech Dec 16, 2016

Owner

Check #2559, available on PM2 2.2.x

Owner

Unitech commented Dec 16, 2016

Check #2559, available on PM2 2.2.x

@Unitech Unitech closed this Dec 16, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment