Added systemd based service-runner in python (#1025)

* Added systemd based service-runner in python

Added a new python based service-runner and .service file to enable it to run as
a service under systemd.
This should replace the service-runner.sh bash script included in other repos
that require service-runner.

* Fixed path to script in the .service file

* Tidied up the documentation

* Try and appease the automated code review

scripts/services/install-service-runner-update.md

@@ -0,0 +1,37 @@
+# Service Runner
+
+The service runner is used to trigger scrips (e.g update / backup) from emoncms, it needs to be running continuously. 
+
+Service runner is a bridge between the web application and update bash scripts.
+
+The process is as follows:
+
+1. Web application triggers an update by setting a flag in redis
+2. Service runner continuously polls redis for an update flag
+3. Service runner starts the update and logs to a file which the web application reads
+
+## Install python systemd service
+
+If you are not running EmonCMS on Raspbian, modify the .service file to run the service
+as an appropriate user. The service is configured to run as the user 'pi' by default.
+Install the service using the following commands:
+```
+sudo pip install redis
+sudo ln -s /var/www/emoncms/scripts/services/service-runner/service-runner.service /lib/systemd/system
+sudo systemctl daemon-reload
+sudo systemctl enable service-runner.service
+sudo systemctl start service-runner.service
+systemctl status service-runner.service
+```
+
+View the log with:
+`journalctl -f -u service-runner`
+
+Tested on Raspiban Stretch
+
+Prior to September 2018 the service runner ran as a bash script triggered by cron. The
+bash script had to connect to redis every iteration of the loop which on a RPi 3 caused
+service runner to consume 100% of the CPU.
+This version was written by @greeebs using python and systemd instead of bash and cron, see
+https://github.com/emoncms/emoncms/pull/1025 for the discussion.
+The python service is far more efficient as a constant connection to redis can be kept open.

scripts/services/service-runner/service-runner.py

@@ -0,0 +1,83 @@
+#!/usr/bin/python
+
+## Used to run arbitrary commands from the EmonCMS web interface
+# EmonCMS submits commands to redis where this service picks them up
+# Used in conjunction with:
+# - Admin module to run service-runner-update.sh
+# - Backup module
+# - Others??
+
+import sys
+import redis
+import subprocess
+import time
+import signal
+
+def handle_sigterm(sig, frame):
+  print("Got Termination signal, exiting")
+  sys.exit(0)
+
+# Setup the signal handler to gracefully exit
+signal.signal(signal.SIGTERM, handle_sigterm)
+signal.signal(signal.SIGINT, handle_sigterm)
+
+def connect_redis():
+  while True:
+    try:
+      server = redis.Redis()
+      if server.ping():
+        print("Connected to redis-server")
+        sys.stdout.flush()
+        return server
+    except redis.exceptions.ConnectionError:
+      print("Unable to connect to redis-server, sleeping for 30s")
+      sys.stdout.flush()
+    time.sleep(30)
+
+print("Starting service-runner")
+sys.stdout.flush()
+
+server = connect_redis()
+
+while True:
+  try:
+    # Check for the existence of a redis 'service-runner' key
+    if server.exists('service-runner'):
+      # We've got one, now to turn it into a cmdline
+      flag = server.lpop('service-runner')
+      print("Got flag: %s\n" % flag)
+      sys.stdout.flush()
+      script, logfile = flag.split('>')
+      cmdstring = "{s} > {l} 2>&1".format(s=script, l=logfile)
+      print("STARTING: " + cmdstring)
+      sys.stdout.flush()
+      # Got a cmdline, now run it.
+      try:
+        subprocess.call(cmdstring, shell=True)
+      except SystemExit:
+        # If the sys.exit(0) from the interrupt handler gets caught here,
+        # just break from the while True: and let the script exit normally.
+        break
+      except:
+        # if an error occurs running the subprocess, add the error to
+        #  the specified logfile
+        f = open(logfile, 'a')
+        f.write("Error running [%s]" % cmdstring)
+        f.write("Exception occurred: %s" % sys.exc_info()[0])
+        f.close()
+        raise # Now pass the exception upwards
+      print("COMPLETE: " + cmdstring)
+      sys.stdout.flush()
+  except redis.exceptions.ConnectionError:
+    print("Connection to redis-server lost, attempting to reconnect")
+    sys.stdout.flush()
+    server = connect_redis()
+  except SystemExit:
+    # If the sys.exit(0) from the interrupt handler gets caught here,
+    # just break from the while True: and let the script exit normally.
+    break
+  except:
+    print("Exception occurred", sys.exc_info()[0])
+    sys.exit(1)
+  time.sleep(0.2)
+

scripts/services/service-runner/service-runner.service

@@ -0,0 +1,68 @@
+# Systemd unit file for mqtt input script
+
+# ***** NOTE: RUNS AS USER "pi" BY DEFAULT *****
+#  If running on a non-Raspbian environment, change "User=pi" in the [Service] section
+#  to the user of your choice (user must exist and should be the "emoncms" admin account)
+
+# INSTALL:
+# sudo ln -s /var/www/emoncms/scripts/services/service-runner/service-runner.service /lib/systemd/system
+
+# RUN AT STARTUP
+# sudo systemctl daemon-reload
+# sudo systemctl enable service-runner.service
+
+# START / STOP With:
+# sudo systemctl start service-runner
+# sudo systemctl stop service-runner
+
+# VIEW STATUS / LOG
+# If Using Syslog:
+# systemctl status service-runner -n50
+# where -nX is the number of log lines to view
+# journalctl -f -u service-runner
+# Otherwise:
+# Specify
+#StandardOutput=file:/var/log/service-runner.log
+# tail -f /var/log/service-runner.log
+
+###
+#
+#  All Emoncms code is released under the GNU Affero General Public License.
+#  See COPYRIGHT.txt and LICENSE.txt.
+#
+#  ---------------------------------------------------------------------
+#  Emoncms - open source energy visualisation
+#  Part of the OpenEnergyMonitor project:
+#  http://openenergymonitor.org
+###
+
+[Unit]
+Description=Emoncms service-runner Input Script
+Wants=redis-server.service
+After=redis-server.service
+StartLimitIntervalSec=5
+#Documentation=https://github.com/emoncms/emoncms/blob/master/docs/service-runner.md
+
+# Uncomment this line to use a dedicated log file for StdOut and StdErr.
+# NOTE: only works in systemd v236+
+#       Debain "stretch" includes v232, "buster" includes v239
+#StandardOutput=file:/var/log/service-runner.log
+
+[Service]
+Type=idle
+ExecStart=/usr/bin/python /var/www/emoncms/scripts/services/service-runner/service-runner.py
+User=pi
+
+# Restart script if stopped
+Restart=always
+# Wait 60s before restart
+RestartSec=30s
+
+# Tag things in the log
+# If you want to use the journal instead of the file above, uncomment SyslogIdentifier below
+# View with: sudo journalctl -f -u service-runner -o cat
+SyslogIdentifier=service-runner
+
+[Install]
+WantedBy=multi-user.target
+