Please update the wiki managing with upstart with the following suggestion

[innovia]

Hi

We found out that using the suggested upstart on the wiki is not good enough.

when you issue the stop command, the services that eye was monitoring are staying up while eye got killed by the upstart, which in turn cause the issue of trying to start services that are already up like unicorn etc...

in order to fix this and allow eye to stop all services we added some modifications (tested) to the upstart config file:

description "Eye Monitoring System"

start on runlevel [2345]
stop on runlevel [016]

expect fork
kill timeout 60 # when upstart issued a stop, send SIGTERM, wait 60 sec before sending SIGKILL

setuid deploy
setgid deploy

respawn

# ensure eye home folder is set (stores in .eye the pid, the states history, and the eye socket file
env EYE_HOME=/home/deploy

# log stdout and stderr to /var/log/upstart/eye
console log

# important for unicorn to create a socket folder & set permissions before run 
pre-start script
  mkdir -p "/var/run/unicorn"
  chown -R deploy:deploy "/var/run/unicorn"
end script

# load all eye services - upstart will monitor eye, and eye will monitor its own processes
script
  exec /usr/local/bin/eye load /etc/eye/*.eye
end script

# this section is to ensure services won't stay up while upstart kills (when issued a stop) the actual eye process

pre-stop script
  /usr/local/bin/eye stop all   # Stop all eye services
  /bin/sleep 15s # wait 15sec before issue SIGTERM to eye
end script

[kostya]

Thanks. When you quit eye and then up, it should not start again processes (unicorn,...) if it was configured correctly (pid_file). You can add this to wiki, just to another page for example.

[innovia]

what do you mean?
the issue here was upstart killing eye before it had a chance to gracefully send quit to all
so sleeping for 30s gave it enough time to do so and then exit

with the previous upstart script you had, it would kill eye bur processes would still be running

[kostya]

which in turn cause the issue of trying to start services that are already up like unicorn etc...
i mean, it should no be like this, eye just finds all running processes.