pm2 fails to return correctly in ansible #88

Closed
jpmerritt opened this Issue Sep 24, 2013 · 19 comments

Comments

@jpmerritt

When pm2 commands are run through ansible they operate correctly but fail to return in a way that ansible recognises causing ansible to be stuck at that location waiting for a correct exit.

For example a command like
$ pm2 start pm2-start.json
will run the script but hang afterwards. If you kill that script in a different terminal session the still running script in the other session will finally continue. You can also work around the problem by attaching >& /dev/null to the end of the script so the ansible command might end up looking like this.

shell: pm2 start pm2-start.json >& /dev/null chdir=~/ executable=/bin/bash

@Unitech

This comment has been minimized.

Show comment
Hide comment
@Unitech

Unitech Sep 30, 2013

Owner

This is strange, I do the classical process.exit(SUCCESS_EXIT); (SUCCESS_EXIT = 1) when the CLI is exited

Owner

Unitech commented Sep 30, 2013

This is strange, I do the classical process.exit(SUCCESS_EXIT); (SUCCESS_EXIT = 1) when the CLI is exited

@arabold

This comment has been minimized.

Show comment
Hide comment
@arabold

arabold Oct 29, 2013

I'm invoking pm2 through SSH on my CI server (via Atlassian Bamboo) and are stuck in a similar situation. My call to pm2 is pm2 start -x /var/www/my-app/app.js --name my-app.

arabold commented Oct 29, 2013

I'm invoking pm2 through SSH on my CI server (via Atlassian Bamboo) and are stuck in a similar situation. My call to pm2 is pm2 start -x /var/www/my-app/app.js --name my-app.

@jonhatalla

This comment has been minimized.

Show comment
Hide comment
@jonhatalla

jonhatalla Jan 15, 2014

i'm also invoking pm2 via ssh
ssh hostname "cd folder; pm2 kill; git pull; npm install; pm2 start index.js"

this won't return from the ssh command - but if i run it locally it returns with an exit code of 0 ($?)

i'm also invoking pm2 via ssh
ssh hostname "cd folder; pm2 kill; git pull; npm install; pm2 start index.js"

this won't return from the ssh command - but if i run it locally it returns with an exit code of 0 ($?)

@soyuka soyuka added the bug label Mar 1, 2014

@mehulved

This comment has been minimized.

Show comment
Hide comment
@mehulved

mehulved Mar 5, 2014

I've run into this bug too. Ansible hangs when it reaches pm2 start even though the command has been executed on the server.

mehulved commented Mar 5, 2014

I've run into this bug too. Ansible hangs when it reaches pm2 start even though the command has been executed on the server.

@rlidwka

This comment has been minimized.

Show comment
Hide comment
@rlidwka

rlidwka Mar 6, 2014

Collaborator

isn't it an Ansible bug?

Collaborator

rlidwka commented Mar 6, 2014

isn't it an Ansible bug?

@dmourati

This comment has been minimized.

Show comment
Hide comment
@dmourati

dmourati Mar 14, 2014

It seems more a factor of the text output from pm2.

It seems more a factor of the text output from pm2.

@cmwelsh

This comment has been minimized.

Show comment
Hide comment
@cmwelsh

cmwelsh Mar 27, 2014

I think this is related to the bug described in #235. I used Ansible's async fire and forget to start pm2 before running any other commands.

- name: Ensure pm2 is running
  command: "pm2 ping"
  async: 30
  poll: 0

- name: Check list of Node.js apps running
  command: "pm2 list"
  register: pm2_list
  changed_when: false

- name: Start application
  command: "chdir={{appPath}} pm2 start app.js --name my-website"
  when: "pm2_list.stdout.find('my-website') == -1"

- name: Reload application
  command: "chdir={{appPath}} pm2 reload my-website"

cmwelsh commented Mar 27, 2014

I think this is related to the bug described in #235. I used Ansible's async fire and forget to start pm2 before running any other commands.

- name: Ensure pm2 is running
  command: "pm2 ping"
  async: 30
  poll: 0

- name: Check list of Node.js apps running
  command: "pm2 list"
  register: pm2_list
  changed_when: false

- name: Start application
  command: "chdir={{appPath}} pm2 start app.js --name my-website"
  when: "pm2_list.stdout.find('my-website') == -1"

- name: Reload application
  command: "chdir={{appPath}} pm2 reload my-website"
@dmourati

This comment has been minimized.

Show comment
Hide comment
@dmourati

dmourati Mar 27, 2014

  - name: Setup pm2 init
    copy: src=pm2-init.sh dest=/etc/init.d/ owner=root group=root mode=0755
  - name: Service pm2 setup
    service: name=pm2-init.sh enabled=yes
  - name: app.coffee start
    command: chdir=/data/repos/myrepo pm2 start app.coffee
    sudo_user: nobody
    async: 1
    poll: 0
  - name: app.coffee dump 
    command: chdir=/data/repos/myrepo pm2 dump
    sudo_user: nobody
    async: 1
    poll: 0
  - name: app.coffee stop
    command: chdir=/data/repos/myrepo pm2 stop app.coffee
    sudo_user: nobody
    async: 1
    poll: 0

  - name: Setup pm2 init
    copy: src=pm2-init.sh dest=/etc/init.d/ owner=root group=root mode=0755
  - name: Service pm2 setup
    service: name=pm2-init.sh enabled=yes
  - name: app.coffee start
    command: chdir=/data/repos/myrepo pm2 start app.coffee
    sudo_user: nobody
    async: 1
    poll: 0
  - name: app.coffee dump 
    command: chdir=/data/repos/myrepo pm2 dump
    sudo_user: nobody
    async: 1
    poll: 0
  - name: app.coffee stop
    command: chdir=/data/repos/myrepo pm2 stop app.coffee
    sudo_user: nobody
    async: 1
    poll: 0

@soyuka

This comment has been minimized.

Show comment
Hide comment
@soyuka

soyuka Jun 13, 2014

Collaborator

Could it be related to #338 or #281?

Collaborator

soyuka commented Jun 13, 2014

Could it be related to #338 or #281?

@kompot

This comment has been minimized.

Show comment
Hide comment
@kompot

kompot Jun 15, 2014

Same is true for Salt stack. First time I encounter bug of this kind with Salt.
Fixed that by this workaround

start pm2 daemon:
  cmd.run:
    - name: pm2 ping >/dev/null 2>&1

kompot commented Jun 15, 2014

Same is true for Salt stack. First time I encounter bug of this kind with Salt.
Fixed that by this workaround

start pm2 daemon:
  cmd.run:
    - name: pm2 ping >/dev/null 2>&1
@ryanlelek

This comment has been minimized.

Show comment
Hide comment
@ryanlelek

ryanlelek Jul 15, 2014

Same here with Ansible. Hangs on "start", even when wrapped in a Bash script that exits. Tried "pm2 kill" beforehand to try and restart the whole process, but no luck.

Same here with Ansible. Hangs on "start", even when wrapped in a Bash script that exits. Tried "pm2 kill" beforehand to try and restart the whole process, but no luck.

@luckychild305

This comment has been minimized.

Show comment
Hide comment
@luckychild305

luckychild305 Jul 15, 2014

usually "start" hangs when the process is already running or when you try to start it from a different directory that is messing up your lib imports. This is what I use successfully:

  1. Don't kill but delete process instead. Will give you ugly message if no process is running so set ignore_errors.
- name: delete existing pm2 processes if running
  command: pm2 delete all
  ignore_errors: True
  1. Start process by defining the full path to file and switch into the relative directory (had this issue for my node.js application, where require didn't work correctly without changing dir first).
- name: start pm2 process
  command: pm2 --run-as-user user start /home/user/full/path/to/file/file.js --name processname
    chdir=/home/full/path/to/file/
  sudo: yes
  sudo_user: user

usually "start" hangs when the process is already running or when you try to start it from a different directory that is messing up your lib imports. This is what I use successfully:

  1. Don't kill but delete process instead. Will give you ugly message if no process is running so set ignore_errors.
- name: delete existing pm2 processes if running
  command: pm2 delete all
  ignore_errors: True
  1. Start process by defining the full path to file and switch into the relative directory (had this issue for my node.js application, where require didn't work correctly without changing dir first).
- name: start pm2 process
  command: pm2 --run-as-user user start /home/user/full/path/to/file/file.js --name processname
    chdir=/home/full/path/to/file/
  sudo: yes
  sudo_user: user
@luckychild305

This comment has been minimized.

Show comment
Hide comment
@luckychild305

luckychild305 Jul 15, 2014

to install PM2 I use:

- name: install pm2 globally
  npm: name=pm2 state=present global=yes
  sudo: yes
  sudo_user: root

and

- name: configure pm2 to restart on startup
  shell: pm2 startup ubuntu >& /dev/null chdir=~/ executable=/bin/bash
  sudo: yes
  sudo_user: root

to install PM2 I use:

- name: install pm2 globally
  npm: name=pm2 state=present global=yes
  sudo: yes
  sudo_user: root

and

- name: configure pm2 to restart on startup
  shell: pm2 startup ubuntu >& /dev/null chdir=~/ executable=/bin/bash
  sudo: yes
  sudo_user: root

soyuka added a commit that referenced this issue Jul 16, 2014

@soyuka

This comment has been minimized.

Show comment
Hide comment
@soyuka

soyuka Jul 16, 2014

Collaborator

👍

Closing for now, feel free to re-open.

Collaborator

soyuka commented Jul 16, 2014

👍

Closing for now, feel free to re-open.

@soyuka soyuka closed this Jul 16, 2014

@ryanlelek

This comment has been minimized.

Show comment
Hide comment
@ryanlelek

ryanlelek Jul 17, 2014

Thank you for the tips @luckychild305

I'll give it a go :)

Thank you for the tips @luckychild305

I'll give it a go :)

@sjwoodr

This comment has been minimized.

Show comment
Hide comment
@sjwoodr

sjwoodr Jun 25, 2015

I have the same problem with running any "pm2" command via SaltStack. After struggling with it for some time, I finally found a workaround:

init-pm2:
    cmd.run:
      - name: env PATH=$PATH:/opt/node/node-v0.12.4-linux-x64/bin pm2 startup linux -u someuser
      - require:
        - cmd: install-npm-pm2
      - user: root

Even though the salt-minion process runs as root, without that explicit "- user: root" in there, the command will hang. This can be a highstate or even a cmd.run for the specific state that executes pm2 commands. Any pm2 command would hang: ping, startup, show, etc... I theorize that having the -user option in the state causes salt to spawn a new shell and that shell is sufficiently different than the one running the command by default. (shrug)

sjwoodr commented Jun 25, 2015

I have the same problem with running any "pm2" command via SaltStack. After struggling with it for some time, I finally found a workaround:

init-pm2:
    cmd.run:
      - name: env PATH=$PATH:/opt/node/node-v0.12.4-linux-x64/bin pm2 startup linux -u someuser
      - require:
        - cmd: install-npm-pm2
      - user: root

Even though the salt-minion process runs as root, without that explicit "- user: root" in there, the command will hang. This can be a highstate or even a cmd.run for the specific state that executes pm2 commands. Any pm2 command would hang: ping, startup, show, etc... I theorize that having the -user option in the state causes salt to spawn a new shell and that shell is sufficiently different than the one running the command by default. (shrug)

@jschementi

This comment has been minimized.

Show comment
Hide comment
@jschementi

jschementi Aug 31, 2015

@sjwoodr Confirmed (on pm2 v0.14.7 and salt v2015.5.3) that pm2 now works properly when invoked from salt. Thanks!

@sjwoodr Confirmed (on pm2 v0.14.7 and salt v2015.5.3) that pm2 now works properly when invoked from salt. Thanks!

@artursenk

This comment has been minimized.

Show comment
Hide comment
@artursenk

artursenk Nov 20, 2015

Big thanks @sjwoodr for sharing this!
I've been debugging like crazy for hours. ;)

Big thanks @sjwoodr for sharing this!
I've been debugging like crazy for hours. ;)

@sjwoodr

This comment has been minimized.

Show comment
Hide comment
@sjwoodr

sjwoodr Nov 20, 2015

Awesome. Glad its working for you guys!

sjwoodr commented Nov 20, 2015

Awesome. Glad its working for you guys!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment