Fixes: #2902 - Implements systemd (#3017)

* Closes #2902 - Migrate to systemd from supervisord
* Closes #2902 - Migrate to systemd from supervisord

* Update systemd unit and environment file
* Add gunicorn.conf
* Update documentation and CHANGELOG.  Moved parameters around on service file
* Update Gitignore

.gitignore

@@ -12,6 +12,9 @@
 fabfile.py
 *.swp
 gunicorn_config.py
+gunicorn.conf
+netbox.log
+netbox.pid
 .DS_Store
 .vscode
 .coverage

CHANGELOG.md

@@ -1,6 +1,9 @@
-<<<<<<< HEAD
 v2.7.0 (FUTURE)
 
+## Enhancements
+
+* [#2902](https://github.com/digitalocean/netbox/issues/2902) - Replace supervisord with systemd
+
 ## Housekeeping
 
 * [#3407](https://github.com/netbox-community/netbox/issues/3407) - Added code coverage reporting to the CI pipeline
@@ -51,7 +54,7 @@ v2.6.2 (2019-08-02)
 
 ## Bug Fixes
 
-* [#3018](https://github.com/netbox-community/netbox/issues/3018) - Components connected via a cable must have an equal number of positions
+* [#3018](https://github.com/netbox-community/netbox/issues/301 8) - Components connected via a cable must have an equal number of positions
 * [#3289](https://github.com/netbox-community/netbox/issues/3289) - Prevent position from being nullified when moving a device to a new rack
 * [#3293](https://github.com/netbox-community/netbox/issues/3293) - Enable filtering device components by multiple device IDs
 * [#3315](https://github.com/netbox-community/netbox/issues/3315) - Enable filtering devices/interfaces by multiple MAC addresses

contrib/gunicorn.conf

@@ -0,0 +1,22 @@
+# Bind is the ip and port that the Netbox WSGI should bind to
+#
+bind='127.0.0.1:8001'
+
+# Workers is the number of workers that GUnicorn should spawn.
+# Workers should be: cores * 2 + 1.  So if you have 8 cores, it would be 17.
+#
+workers=3
+
+# Threads
+#     The number of threads for handling requests
+#
+threads=3
+
+# Timeout is the timeout between gunicorn receiving a request and returning a response (or failing with a 500 error)
+#
+timeout=120
+
+# ErrorLog
+#     ErrorLog is the logfile for the ErrorLog
+#
+errorlog='/opt/netbox/netbox.log'

contrib/netbox-rq.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=Netbox RQ Worker
+Documentation=https://netbox.readthedocs.io/en/stable/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+
+EnvironmentFile=/etc/sysconfig/netbox.env
+
+User=www-data
+Group=www-data
+
+WorkingDirectory=${WorkingDirectory}
+
+ExecStart=/usr/bin/python3 ${WorkingDirectory}/netbox/manage.py rqworker
+
+Restart=on-failure
+RestartSec=30
+PrivateTmp=true
+
+[Install]
+WantedBy=multi-user.target

contrib/netbox.env

@@ -0,0 +1,15 @@
+# Name is the Process Name
+#
+Name = 'Netbox'
+
+# ConfigPath is the path to the gunicorn config file.
+#
+ConfigPath=/opt/netbox/gunicorn.conf
+
+# WorkingDirectory is the Working Directory for Netbox.
+#
+WorkingDirectory=/opt/netbox/
+
+# PidPath is the path to the pid for the netbox WSGI
+#
+PidPath=/opt/netbox/netbox.pid

contrib/netbox.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=Netbox WSGI
+Documentation=https://netbox.readthedocs.io/en/stable/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+
+EnvironmentFile=/etc/sysconfig/netbox.env
+
+User=www-data
+Group=www-data
+PIDFile=${PidPath}
+WorkingDirectory=${WorkingDirectory}
+
+ExecStart=/usr/local/bin/gunicorn --pid ${PidPath} --pythonpath ${WorkingDirectory}/netbox --config ${ConfigPath} netbox.wsgi
+
+Restart=on-failure
+RestartSec=30
+PrivateTmp=true
+
+[Install]
+WantedBy=multi-user.target

contrib/netbox@.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=Netbox WSGI
+Documentation=https://netbox.readthedocs.io/en/stable/
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+
+EnvironmentFile=/etc/sysconfig/netbox.%i.env
+
+User=www-data
+Group=www-data
+PIDFile=${PidPath}
+WorkingDirectory=${WorkingDirectory}
+
+ExecStart=/usr/local/bin/gunicorn --pid ${PidPath} --pythonpath ${WorkingDirectory}/netbox --config ${ConfigPath} netbox.wsgi
+
+Restart=on-failure
+RestartSec=30
+PrivateTmp=true
+
+[Install]
+WantedBy=multi-user.target

docs/installation/3-http-daemon.md

@@ -1,4 +1,4 @@
-We'll set up a simple WSGI front end using [gunicorn](http://gunicorn.org/) for the purposes of this guide. For web servers, we provide example configurations for both [nginx](https://www.nginx.com/resources/wiki/) and [Apache](http://httpd.apache.org/docs/2.4). (You are of course free to use whichever combination of HTTP and WSGI services you'd like.) We'll also use [supervisord](http://supervisord.org/) to enable service persistence.
+We'll set up a simple WSGI front end using [gunicorn](http://gunicorn.org/) for the purposes of this guide. For web servers, we provide example configurations for both [nginx](https://www.nginx.com/resources/wiki/) and [Apache](http://httpd.apache.org/docs/2.4). (You are of course free to use whichever combination of HTTP and WSGI services you'd like.) We'll use systemd to enable service persistence.
 
 !!! info
     For the sake of brevity, only Ubuntu 18.04 instructions are provided here, but this sort of web server and WSGI configuration is not unique to NetBox. Please consult your distribution's documentation for assistance if needed.
@@ -108,42 +108,99 @@ Install gunicorn:
 # pip3 install gunicorn
 ```
 
-Save the following configuration in the root netbox installation path as `gunicorn_config.py` (e.g. `/opt/netbox/gunicorn_config.py` per our example installation). Be sure to verify the location of the gunicorn executable on your server (e.g. `which gunicorn`) and to update the `pythonpath` variable if needed. If using CentOS/RHEL, change the username from `www-data` to `nginx` or `apache`.
+# systemd configuration
+
+Copy or link contrib/netbox.service and contrib/netbox-rq.service to /etc/systemd/system/netbox.service and /etc/systemd/system/netbox-rq.service
 
 ```no-highlight
-command = '/usr/bin/gunicorn'
-pythonpath = '/opt/netbox/netbox'
-bind = '127.0.0.1:8001'
-workers = 3
-user = 'www-data'
+# cp contrib/netbox.service to /etc/systemd/system/netbox.service
+# cp contrib/netbox-rq.service to /etc/systemd/system/netbox-rq.service
 ```
 
-# supervisord Installation
+Edit /etc/systemd/system/netbox.service and /etc/systemd/system/netbox-rq.service. Be sure to verify the location of the gunicorn executable on your server (e.g. `which gunicorn`).  If using CentOS/RHEL, change the username from `www-data` to `nginx` or `apache`:
 
-Install supervisor:
+```no-highlight
+/usr/local/bin/gunicorn --pid ${PidPath} --pythonpath ${WorkingDirectory}/netbox --config ${ConfigPath} netbox.wsgi
+```
 
 ```no-highlight
-# apt-get install -y supervisor
+User=www-data
+Group=www-data
 ```
 
-Save the following as `/etc/supervisor/conf.d/netbox.conf`. Update the `command` and `directory` paths as needed. If using CentOS/RHEL, change the username from `www-data` to `nginx` or `apache`.
+Copy contrib/netbox.env to /etc/sysconfig/netbox.env
 
 ```no-highlight
-[program:netbox]
-command = gunicorn -c /opt/netbox/gunicorn_config.py netbox.wsgi
-directory = /opt/netbox/netbox/
-user = www-data
+# cp contrib/netbox.env to /etc/sysconfig/netbox.env
+```
+
+Edit /etc/sysconfig/netbox.env and change the settings as required.  Update the `WorkingDirectory` variable if needed.
+
+```no-highlight
+# Name is the Process Name
+#
+Name = 'Netbox'
+
+# ConfigPath is the path to the gunicorn config file.
+#
+ConfigPath=/opt/netbox/gunicorn.conf
+
+# WorkingDirectory is the Working Directory for Netbox.
+#
+WorkingDirectory=/opt/netbox/
+
+# PidPath is the path to the pid for the netbox WSGI
+#
+PidPath=/var/run/netbox.pid
+```
 
-[program:netbox-rqworker]
-command = python3 /opt/netbox/netbox/manage.py rqworker
-directory = /opt/netbox/netbox/
-user = www-data
+Copy contrib/gunicorn.conf to gunicorn.conf
+
+```no-highlight
+# cp contrib/gunicorn.conf to gunicorn.conf
+```
+
+Edit gunicorn.conf and change the settings as required.
+
+```
+# Bind is the ip and port that the Netbox WSGI should bind to
+#
+bind='127.0.0.1:8001'
+
+# Workers is the number of workers that GUnicorn should spawn.
+# Workers should be: cores * 2 + 1.  So if you have 8 cores, it would be 17.
+#
+workers=3
+
+# Threads
+#     The number of threads for handling requests
+#     Threads should be: cores * 2 + 1.  So if you have 4 cores, it would be 9.
+#
+threads=3
+
+# Timeout is the timeout between gunicorn receiving a request and returning a response (or failing with a 500 error)
+#
+timeout=120
+
+# ErrorLog
+#     ErrorLog is the logfile for the ErrorLog
+#
+errorlog='/opt/netbox/netbox.log'
+```
+
+Then, restart the systemd daemon service to detect the netbox service and start the netbox service:
+
+```no-highlight
+# systemctl daemon-reload
+# systemctl start netbox.service
+# systemctl enable netbox.service
 ```
 
-Then, restart the supervisor service to detect and run the gunicorn service:
+If using webhooks, also start the Redis worker:
 
 ```no-highlight
-# service supervisor restart
+# systemctl start netbox-rq.service
+# systemctl enable netbox-rq.service
 ```
 
 At this point, you should be able to connect to the nginx HTTP service at the server name or IP address you provided. If you are unable to connect, check that the nginx service is running and properly configured. If you receive a 502 (bad gateway) error, this indicates that gunicorn is misconfigured or not running.

docs/installation/index.md

@@ -12,3 +12,5 @@ The following sections detail how to set up a new instance of NetBox:
 If you are upgrading from an existing installation, please consult the [upgrading guide](upgrading.md).
 
 NetBox v2.5 and later requires Python 3.5 or higher. Please see the instructions for [migrating to Python 3](migrating-to-python3.md) if you are still using Python 2.
+
+Netbox v2.5.9 and later moved to using systemd instead of supervisord.  Please see the instructions for [migrating to systemd](migrating-to-systemd.md) if you are still using supervisord.

docs/installation/migrating-to-systemd.md

@@ -0,0 +1,105 @@
+# Migration
+
+Migration is not required, as supervisord will still continue to function.
+
+## Ubuntu
+
+### Remove supervisord:
+
+```no-highlight
+# apt-get remove -y supervisord
+```
+
+### systemd configuration:
+
+Copy or link contrib/netbox.service and contrib/netbox-rq.service to /etc/systemd/system/netbox.service and /etc/systemd/system/netbox-rq.service
+
+```no-highlight
+# cp contrib/netbox.service /etc/systemd/system/netbox.service
+# cp contrib/netbox-rq.service /etc/systemd/system/netbox-rq.service
+```
+
+Edit /etc/systemd/system/netbox.service and /etc/systemd/system/netbox-rq.service. Be sure to verify the location of the gunicorn executable on your server (e.g. `which gunicorn`).  If using CentOS/RHEL.  Change the username from `www-data` to `nginx` or `apache`:
+
+```no-highlight
+/usr/local/bin/gunicorn --pid ${PidPath} --pythonpath ${WorkingDirectory}/netbox --config ${ConfigPath} netbox.wsgi
+```
+
+```no-highlight
+User=www-data
+Group=www-data
+```
+
+Copy contrib/netbox.env to /etc/sysconfig/netbox.env
+
+```no-highlight
+# cp contrib/netbox.env /etc/sysconfig/netbox.env
+```
+
+Edit /etc/sysconfig/netbox.env and change the settings as required.  Update the `WorkingDirectory` variable if needed.
+
+```no-highlight
+# Name is the Process Name
+#
+Name = 'Netbox'
+
+# ConfigPath is the path to the gunicorn config file.
+#
+ConfigPath=/opt/netbox/gunicorn.conf
+
+# WorkingDirectory is the Working Directory for Netbox.
+#
+WorkingDirectory=/opt/netbox/
+
+# PidPath is the path to the pid for the netbox WSGI
+#
+PidPath=/var/run/netbox.pid
+```
+
+Copy contrib/gunicorn.conf to gunicorn.conf
+
+```no-highlight
+# cp contrib/gunicorn.conf to gunicorn.conf
+```
+
+Edit gunicorn.conf and change the settings as required.
+
+```
+# Bind is the ip and port that the Netbox WSGI should bind to
+#
+bind='127.0.0.1:8001'
+
+# Workers is the number of workers that GUnicorn should spawn.
+# Workers should be: cores * 2 + 1.  So if you have 8 cores, it would be 17.
+#
+workers=3
+
+# Threads
+#     The number of threads for handling requests
+#
+threads=3
+
+# Timeout is the timeout between gunicorn receiving a request and returning a response (or failing with a 500 error)
+#
+timeout=120
+
+# ErrorLog
+#     ErrorLog is the logfile for the ErrorLog
+#
+errorlog='/opt/netbox/netbox.log'
+```
+
+Then, restart the systemd daemon service to detect the netbox service and start the netbox service:
+
+```no-highlight
+# systemctl daemon-reload
+# systemctl start netbox.service
+# systemctl enable netbox.service
+```
+
+If using webhooks, also start the Redis worker:
+
+```no-highlight
+# systemctl start netbox-rq.service
+# systemctl enable netbox-rq.service
+```