How To Setup A Production Agora Server.
In this guide, all commands will be represented by using the "$" to show you are only running it as a user. Commands that need root access will be done with the sudo command. This means if you are connecting using SSH then you will need to either run as root or add a user to the sudoer group.
Install Debian using this tutorial or however you like, you can even use another distribution, but I prefer Debian and that's what this tutorial is based on.
https://www.youtube.com/watch?v=TIuOFqQ-XTk
When you get to the point where you are choosing packages, at roughly 7:52 What you need is SSH and Standard system utilities. If you are only testing this and don't quite plan on using production you may want to consider installing the Debian Desktop Environment.
At this point you should then setup ssh. To do this, you will want to do the following command.
$ sudo nano /etc/ssh/sshd_config
Here you want to change various settings to allow you to access the SSH server. If you are on a residential internet service provider you may need to use an obscure port, as most ISPs block standard SSH ports. You may also need to setup your router for port forwarding. Since this is probably specific process for your router, I would recommend googling something to the effect of:
setup port forwarding on 'router-model-here'
While you're setting up port forwarding you might as well chose a port you want to use for your web server as well. These will need to be separate ports. I recommend numbers with 4 digits. Here are some you can use.
SSH: 9942 HTTPS:4433 HTTP:8081
You can use other ports if you want, just make sure they aren't going to conflict with any other processes running on that server.
Once you have port forwarding setup and your config file setup. If you plan on logging into ssh using a normal user rather than a root account, I recommend adding that user to the "sudoer" group and the "ssh" group using the following commands.
$ sudo adduser 'username' sudo
$ sudo adduser 'username' ssh
Make sure your locales are setup by doing
$ sudo dpkg-reconfigure locales
Then set the locale you want it to be set to use. There are several prerequisites you need to install and get working before you're able to get Agora-Ciudadana running in production. The things you'll need to install are partly listed in installation file located here.
Agore-Ciudadana github INSTALL.md
There are also several others that you'll need to install as well. To give a pretty solid start and get all the prerequisites out of the way run the following commands:
$ sudo apt-get install python2.7 python2.7-dev virtualenvwrapper rabbitmq-server git gettext git-core curl build-essential openssl nginx libexpat1 ssl-cert postgresql gunicorn
$ sudo apt-get build-dep python-psycopg2
$ sudo apt-get install python-psycopg2
$ sudo apt-get -y install libmemcached-dev
$ sudo easy_install python-psycopg2
$ sudo easy_install virtualenv
$ sudo easy_install virtualenvwrapper
$ export WORKON_HOME=$HOME/.virtualenvs
$ pip install psycopg2
At this point you will need to install Node JS. Refereing to the following website you can install it using the following commands.https://gist.github.com/clemherreman/1869145
Node JS Prerequisites
$ sudo apt-get update && sudo apt-get install git-core curl build-essential openssl libssl-dev
Using any newer versions of node has given me problems, so I am posting this version. As it is known to work, at least in my case.
$ wget https://github.com/joyent/node/archive/v0.10.7.tar.gz
$ tar -zxvf v0.10.7.tar.gz
$ cd node
$ ./configure --openssl-libpath=/usr/lib/ssl
$ make
$ sudo make install
$ node -v # it's alive!
$ npm -v # it's alive!
Next we want to download the git file so run the command:
$ git clone https://github.com/agoraciudadana/agora-ciudadana.git
That will download the Agora-Ciudadana master to the current directory you're in and create a folder called "agora-ciudadana" This is going to be the location of all your webserver files, so make sure you're in the directory you want these downloaded to before executing that command, or be prepared to move it to the location you desire.From here we want to navigate to this folder by executing the following command:
$ cd ./agora-ciudadana
To setup a database you have to install a database server application. I use postgresql so I'll be explaining how to set that up. Since we already installed the software in earlier steps lets jump into setting it up. To create a database and user for postgresql you need to issue the following command.
$ sudo su postgres
You will now be logged in as the postgres user. Now issue the following command and you'll be able to work with postgresql to create and modify databases and database roles.
$ psql
To create a database you issue the following command.
$ CREATE DATABASE agoradb;
To create a database user.
$ CREATE ROLE agora WITH PASSWORD 'password';
To allow the user to be able to login.
$ ALTER ROLE agora WITH login;
Now we want to allow the user to modify the database. A simple way to do so, but also would be considered an insecure way of doing so is to run the command.
$ GRANT ALL PRIVILEGES ON DATABASE agoradb TO agora;
While it is recommended that you create a custom_settins.py file, I have had issues with that. So you can either create a custom_settings.py file which should be read and shouldn't cause issues. Otherwise, you can backup the settings.py file and edit a version of it. To backup the settings.py file issue the following command.
$ cp ~/agora-ciudadana/agora_site/settings.py ~/agora-ciudadana/agora_site/settings.py.bak
Now lets create a custom_settings.py file.
$ nano ~/agora-ciudadana/agora-site/custom_settings.py
So going through the various settings, here are some things we're looking for. You can even copy and paste the following code chunks, and edit it as needed into a custom_settings.py file.
# The DEBUG setting allows you to see the various errors that pop up while running the server within the
# browser. This way you will see errors right in your browser rather than having to search for them or
# follow your shell. The problem with DEBUG is it has memory leaks and will cause issues if run for to
# long. Don't use DEBUG on production, or at least try not to.
DEBUG = False
# There is a setting called ADMINS. This will list the administrators and send emails to those listed
# regarding issues the server is having.
ADMINS = (
('Admin Name', 'admin.email@email.com'),
)
# So lets go through the databases section one part at a time. The comments do provide a pretty decent
# explanation, but lets get a little more clear.
DATABASES = (
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'sqlite.db',
'USER': 'dbusername',
'PASSWORD': 'secret',
'HOST': '',
'PORT': '',
# ENGINE is where we put what type of database we're going to use. By default it uses sqlite3, but that
# is not nearly powerful enough, nor is it readily useable beyond testing. I personally use
# postgresql_psycopg2. It's extremely easy to use, and works great thus far.
# NAME is where you put the name # of your database. If you haven't created a database as of yet, then
# it doesn't matter what you put here as your database will be create, but remember the name so you can
# reference it late when creating other files later on.
# USER is the username for the database you created. if you use a username that is linked to a linux
# account that is running agora, then you don't need to put in a password.
# HOST is where you would put the name of the server that holds your database. If your database is on
# the computer you're running agora from, then you can just leave it blank.
# PORT if you plan on running your database on a port other than the default, then you should put that
# port number here.
# TIME_ZONE is pretty simple, just check the site listed in the settings file and find the timezone that
# best matches the one you want to use. If you're running this in linux than you can also just put
# "none" and django will use the system settings.
TIME_ZONE = 'America/Chicago'
#Next is language, pretty simple again, just go to the site listed in the settings file
LANGUAGE_CODE = 'en'
LANGUAGES = (
('es', _('Spanish')),
('en', _('English')),
('gl', _('Galician')),
)
#After you finish language settings you should change the SECRET_KEY setting, this is something that you
#wont really have to remember, so make it super long and jumbled up like this.
SECRET_KEY = 'G3jv$r9vfw#2Fjf9u32o2|#]fo34nJerV{$4krfj0vf29v2}#$RG^&HN^Rwouv04wCX$#Cwobbj54Hgj2fvV%^REv-+etn4lt3l5m,6y'
#Under INSTALLED_APPS you will want to add any additional applications you have for django.
INSTALLED_APPS = ( \# make sure to include at the very least gunicorn
'gunicorn'
#You will also want to set your TWITTER_CONSUMER_KEY and TWITTER_CONSUMER_SECRET if you plan on allowing
#people to login using twitter.
#You will also want to add the CRISPY_TEMPLATE_PACK setting as well, as without it the server will not
#function properly.
CRISPY_TEMPLATE_PACK = 'bootstrap'
#Continuing further on you'll find SITE_NAME where you will make your site a little prettier by giving
#it it's own special name.
#Then the AGORA_CREATION_PERMISSIONS is also something you may want to look at. If you don't want just
#anyone being able to create an agora,then you should set it to "superusers-only" There is a blurb of
#documentation in the settings.py file that should help you greatly in understanding that though. If you
#plan on allowing anyone to create an agora, then feel free to leave this out of the custom_settings.py
#file.
AGORA_CREATION_PERMISSIONS = 'superusers-only'
#If you want to have AGORA_USE_HTTPS and be a bit more secure, then set that setting to True.
#Then since we decided to customize the settings.py file we can comment out all the rest of the file,
#as it's trying to pull settings from a file that doesn't exist.
AGORA_USE_HTTPS = True`
Now you will want to run through the following commands in order to sync your database up with agora. This is quoted from the agora INSTALL.md
First we need to create the virtual environment where dependencies will be installed:
$ mkvirtualenv agora-ciudadanaNow everytime we want to use the installed virtualenv, we can do the following:
$ workon agora-ciudadanaNow we will install the dependencies:
$ pip install -r requirements.txt --upgradeAfter that, we need to configure the database (we use sqlite by default):
$ ./manage.py syncdb --allAnd mark all migration scripts as applied (because we have created the database with --all option):
$ ./manage.py migrate --fakeWe use django haystack for searching, so we need to create the initial index:
$ ./manage.py rebuild_index --noinputSometimes permissions are not created by django correctly. Running the following command fixes this problem:
$ ./manage.py check_permissionsWe use celery and rabbitmq for programmed tasks, so you need to setup it correctly in your server. Usually >you just need to install it and run it as a system daemon with:
$ sudo /etc/rc.d/rabbitmq startIn some occasions, for example if you are using ubuntu or Opensuse, the command above won't work. You need >use this one instead:
$ sudo service rabbitmq-server start
Before we install the software onto the server, we need to make sure that the make file has the proper install files needed. So what we need to do is edit the Makefile.
$ cd ~/agora-ciudadana
$ nano ./Makefile
In the Makefile on line 5 'less' is listed. You will want to change it to read:
npm install uglify-js less@1.3.3 jshint
Once that is finished save and close the file. then install the server software.
$ make dependencies && make all
Next you want to create a self signed certificate for the site, or get a certificate from a certificate authority. You can get them for free at startssl.com, or purchasing from comodo.com or another authority. To create a selfsigned certificate, go into the directory you want to store them and enter this command.
$ openssl req -newkey rsa:4096 -x509 -days 730 -nodes -out agora.pem -keyout agora.key
You'll now want to create some files to start agora itself. Here are a couple of examples. First you'll want to create the files by issuing these commands.
$ touch ./agora_launch.sh
$ touch ./agora_celery_launch.sh
$ sudo chmod 775 ./agora_launch.sh
$ sudo chmod 775 ./agora_celery_launch.sh
Now you want to copy and paste the following examples into the files we just created.
agora_launch.sh
#!/bin/bash
PIDFILE=/home/agora/agora-supervisord.pid
ROOTD=/home/agora/.virtualenvs
cd $ROOTD
. $ROOTD/agora/bin/activate
if [ -f $PIDFILE ]; then
kill \`cat -- $PIDFILE\
rm -f -- $PIDFILE
fi
python /home/agora/agora-ciudadana/manage.py run_gunicorn -b 0.0.0.0:3368 -w 3 -p $PIDFILE --user=agora --group=users --log-level=debug`
agora_celery_launch.sh
#!/bin/bash
PIDFILE=/home/agora/agora-celery.pid
ROOTD=/home/agora/.virtualenvs
cd $ROOTD
. $ROOTD/agora/bin/activate
if [ -f $PIDFILE ]; then
kill `cat -- $PIDFILE`
rm -f -- $PIDFILE
fi
/home/agora/agora-ciudadana/manage.py celery worker --pidfile=$PIDFILE -B -S djcelery.schedulers.DatabaseScheduler
You may need to alter the files in any way to make sure they work with your system. Now to start the agora services server, issue these two commands in separate terminals that are currently working on your virtual environments we created earlier.
$ ./agora_celery_launch.sh
$ sudo ./agora_launch.sh
Also it may be wise to create daemon services for these applications. I don't currently know how to do that yet, if you want to submit a daemon service file that would be great and I'll add it to the how to.
Now we want to allow folks to be able to view the site, so we have to setup a config file for nginx. Here is an example, feel free to copy and paste this thing too.
Agora nginx site file
server {
listen 80;
server_name blackbit.mnpirateparty.org;
return 301 https://blackbit.mnpirateparty.org$request_uri?;
}
# ssl
server {
listen 443 ssl;
server_name blackbit.mnpirateparty.org;
# HTTP Strict Transport Security, HSTS tells a browser that the website
# should only be accessed through a secure connection.
# Remember this setting for 365 days
# From: http://www.westphahl.net/blog/2012/01/03/setting-up-https-with-nginx-and-startssl/
add_header Strict-Transport-Security max-age=31536000;
ssl_certificate /etc/ssl/certs/agora.pem;
ssl_certificate_key /etc/ssl/private/agora.key;
# see https://support.comodo.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=1365&nav=0,1,95
#enables SSLv3/TLSv1, but not SSLv2 which is weak and should no longer be used.
ssl_protocols SSLv3 TLSv1;
#Disables all weak ciphers
ssl_ciphers ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM;
# serve directly - analogous for static/staticfiles
location /static/ {
alias /home/agora/agora-ciudadana/agora_site/static/;
autoindex off;
}
location /media/ {
autoindex off;
alias /home/agora/agora-ciudadana/agora_site/media/;
}
location / {
alias /home/agora/agora-ciudadana;
autoindex off;
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://localhost:3368/;
}
}
Be sure to modify this to allow your environment to run properly. After setting that up issue the command
$ sudo service nginx stop
$ sudo service nginx start
If you haven't already issued the following commands issued in 2 seperate windows that are working on the virtual environment we created earlier you should do so now.
$ ./agora_celery_launch.sh
$ sudo ./agora_launch.sh
You should now be able to see your agora running by going to your sites url.