Skip to content

Installation Guide

Jump to bottom Edit New page
Andrew Thompson edited this page Jul 29, 2014 · 41 revisions

If your intention is to set up OTM2 for development and contributing to the open source project, instead of using this guide, we recommend you use the otm2-vagrant project instead of following these steps. That will get you a "dev env" much more quickly!

This guide assumes use of 64-bit Ubuntu 12.04. IMPORTANT: OpenTreeMap2 will ONLY work on 64-bit Ubuntu. Do not attempt with 32-bit.

There are several config files, upstart scripts, and other small files that you'll need to copy/get from the otm2-vagrant project's "scripts" directory. We'll link to them here when you need them.

Table of Contents

  1. Install Basics
  2. Postgres and PostGIS
  3. Django and the OTM2 Django apps
  4. Ecobenefits project
  5. otm2-tiler
  6. nginx HTTP server
  7. Upstart scripts
  8. Start up services
  9. Create OTM User account
  10. Create treemap instance
  11. Visit your map!

Step 1: Install basics.

The vast majority of commands in this guide require superuser permissions. If you're comfortable, go ahead and sudo su to change to the root user to save yourself some repetitive sudo'ing.

apt-get update
apt-get install python-setuptools python-software-properties git

We'll add some aptitude repositories that will let us get Node.js and GIS components without building them from source, as well as some for mapnik (required by the otm2-tiler). Then run apt-get update again.

add-apt-repository ppa:mapnik/boost
add-apt-repository ppa:mapnik/v2.1.0
add-apt-repository ppa:chris-lea/node.js
add-apt-repository ppa:ubuntugis/ppa

apt-get update

Next, let's get node.js and Redis, both required by the otm2-tiler.

apt-get install nodejs redis-server

Finally, let's get the GIS package dependencies required by Geodjango and PostGIS, other image processing packages, pip (for our virtual environment we'll be using with the OTM2 Django app), and sendmail (a mailserver, needed by OTM2).

apt-get install gettext libgeos-dev libproj-dev libgdal1-dev build-essential python-pip python-dev
apt-get install libfreetype6-dev zlib1g-dev libpq-dev libxml2-dev
apt-get install sendmail
pip install virtualenv

Optional: For running tests If you're setting up a development environment for OTM2, you'll want xvfb and firefox to be able to run the test suites:

apt-get install xvfb firefox

Step 2: Install the PostgreSQL database and PostGIS

apt-get install postgresql postgresql-server-dev-9.1 postgresql-contrib postgresql-9.1-postgis-2.0

Optional: Compile PostGIS from source: If you'd prefer to do it this way, omit the "postgresql-9.1-postgis-2.0" from the last line, and execute the following instead:

#download and compile PostGIS:

mkdir -p /usr/local/postgis/download
cd /usr/local/postgis/download
wget http://postgis.org/download/postgis-2.0.1.tar.gz
tar -xzf postgis-2.0.1.tar.gz
cd postgis-2.0.1
./configure --without-topology
make
make install

Now that we have Postgres and PostGIS installed, however it was done, we need to restart Postgres, and create a user, database, and extensions for OTM2 to use. This will need to be done in the psql prompt as the "postgres" system user.

service postgresql restart
su postgres
psql

The following commands should be done in the psql prompt. If you want all the config files from the otm2-vagrant project to "just work" without any editing, use "otm" as your database name, "otm" as your database user, and "password" as your database user's password. However, don't use these values in a production environment!

CREATE USER pick_an_otm_username SUPERUSER PASSWORD ‘pick_a_password’; --Use superuser for django tests to work, but don’t use this if you're not building a development environment. Otherwise, use “ENCRYPTED” in place of "SUPERUSER".
\c template1; --connect to the template database to add the hstore extension
CREATE EXTENSION IF NOT EXISTS hstore; --must be created in template1
CREATE DATABASE pick_an_otm_db_name OWNER pick_an_otm_username; --the actual database 
\c pick_an_otm_db_name
CREATE EXTENSION IF NOT EXISTS postgis; --The PostGIS extension *must* be created in the OTM database, and NOT the template1 database. Django tests try to create postgis in the template db and will fail if it’s already there
\q --quit the psql prompt

Now, exit from the postgres user back to root or your system user:

exit

Step 3: Install Django and the OTM2 Django apps

Now we'll clone the OTM2 repo, make a virtualenv for it, activate the environment for the duration of this install process, and install the requirements (for production, development, and test) in that environment.

mkdir /usr/local/otm
cd /usr/local/otm
git clone https://github.com/OpenTreeMap/OTM2.git
mv OTM2 app
virtualenv env
source env/bin/activate

cd /usr/local/otm/app
pip install -r requirements.txt
pip install -r test-requirements.txt
pip install -r dev-requirements.txt

We'll need to create/add and edit a /usr/local/otm/app/opentreemap/opentreemap/local_settings.py file like the default in the otm2-vagrant project.

cd /usr/local/otm/app/opentreemap/opentreemap
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/usr/local/otm/app/opentreemap/opentreemap/local_settings.py

If you chose the same database info (database name, user, password) as we did above, you won't need to edit the downloaded file. If not, edit local_settings.py accordingly.

Client side assets

Next, we'll have npm and grunt build our client-side assets (CSS and Javascript):

cd /usr/local/otm/app
npm install
npm install -g grunt-cli@0.1.9
grunt --dev #OPTIONAL - Only run for a development environment

Database migrations

Now, we'll have to use south to sync and apply the OTM database migrations. We will also run the create_system_user management command at this time:

cd /usr/local/otm/app/opentreemap
python manage.py syncdb
python manage.py migrate
python manage.py create_system_user

Static files

Now let's set up the static and media folders (defined in local_settings.py) and run Django's collectstatic command:

cd /usr/local/otm
mkdir static
mkdir media

cd /usr/local/otm/app/opentreemap
python manage.py collectstatic

Step 4: Install the ecobenefits project

The ecobenefits repo is written in the Go language, so we'll need to install Go as well. The ecobenefits service is what OTM2 uses to calculate the ecosystem benefits of trees.

apt-get install libgeos-dev #should already be installed, but just check
cd /usr/local
wget "https://go.googlecode.com/files/go1.2.linux-amd64.tar.gz" -O go.tar.gz
tar -C /usr/local -xzf go.tar.gz
export PATH="$PATH:/usr/local/go/bin"
export GOPATH="/usr/local/ecoservice"
mkdir ecoservice
cd /usr/local/ecoservice
go get -v github.com/OpenTreeMap/ecobenefits # -v flag for verbose optional
go build github.com/OpenTreeMap/ecobenefits

Create/add a /usr/local/ecoservice/config.gcfg file like the default in the otm2-vagrant project. If you used different ones, you'll need to edit the file and update your values for database user, name, and password:

wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/usr/local/ecoservice/config.gcfg

Get the data

The ecobenefits repo has a "data" folder with the data the ecobenefits service needs to run. Let's clone the repo and move the data folder over to where the config.gcfg file expects it:

cd /usr/local
git clone https://github.com/OpenTreeMap/ecobenefits.git
cp -r ecobenefits/data ecoservice/data/

Step 5: Install the otm2-tiler

First, we'll need to clone the otm2-tiler repository. But we'll move it to a simpler folder name:

cd /usr/local
git clone https://github.com/OpenTreeMap/otm2-tiler.git
mv otm2-tiler/ tiler

Now, we need to install a few dependencies, and then run "npm install" to install the tiler:

apt-get install libsigc++-2.0-dev libmapnik-dev mapnik-utils
cd /usr/local/tiler
npm install

Finally, as described in the otm2-tiler readme, we'll need a settings.json file in the /usr/local/tiler directory. This can be from the otm2-vagrant project - just make sure the values reflect those for your own Postgres database:

wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/usr/local/tiler/settings.json

Step 6: Install and configure the nginx HTTP server

apt-get install nginx

We'll need to create/add an /etc/nginx/sites-available/otm.conf file like the default in the otm2-vagrant project. It should define a server for the OTM app and some other details like the below:

include includes/upstreams/*;

server {
  listen 80 default_server;
  
  client_max_body_size 20M;

  include includes/locations/*;
}

You can use the one from otm2-vagrant directly too...

cd /etc/nginx/sites-available
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/nginx/sites-available/otm.conf

Once we've added the file in the sites-available directory, we need to symlink it to the sites-enabled directory to enable it to be served by nginx:

cd /etc/nginx
rm sites-enabled/default
ln -s /etc/nginx/sites-available/otm.conf /etc/nginx/sites-enabled/otm.conf #must provide the full paths here for symlinking to work right

Location and Upstream definitions

Now we need to create some directories.

mkdir -p /etc/nginx/includes/upstreams
mkdir -p /etc/nginx/includes/locations
chown www-data:www-data /etc/nginx/includes
chown www-data:www-data /etc/nginx/includes/upstreams
chown www-data:www-data /etc/nginx/includes/locations

And add the relevant files from the otm2-vagrant project:

cd /etc/nginx/includes/locations
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/nginx/includes/locations/app
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/nginx/includes/locations/tiler

cd ../upstreams
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/nginx/includes/upstreams/app
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/nginx/includes/upstreams/tiler

Step 7: Add upstart scripts in /init

We'll need to add upstart scripts for ecobenefits service, the Gunicorn app server, the celery daemon, and the otm2-tiler.

If you've been following this guide, you can grab these right from otm2-vagrant without editing them. If you've used different directories for any of the above services, be mindful of that and edit these files appropriately:

cd /etc/init
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/init/celeryd.conf
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/init/ecoservice.conf
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/init/otm-unicorn.conf
wget https://raw.githubusercontent.com/OpenTreeMap/otm2-vagrant/master/configs/etc/init/tiler.conf

It shouldn't be necessary, but you may want upstart to reload these scripts if for some reason the changes don't take immediately:

initctl reload-configuration

Step 8: All systems go!

Let's start all 4 services we just set up!

service otm-unicorn restart
service tiler restart
service ecoservice restart
service nginx restart

Step 9: Create an OpenTreeMap account

With any luck, things should be humming along now. Visit your OTM server in a browser (if you have a GUI environment on your server, you can go straight to http://localhost/ in Firefox). Do you see the OpenTreeMap skeleton page? Great!

In the top right corner, there's a button to "sign up".

OpenTreeMap Sign up button

Click it and fill out the form to create an account. Take note of your username, and enter a valid email so you can use the activation link that will be sent out (via sendmail):

Sign up form

Step 10: Create a treemap instance

Treemaps are created via the create_instance Django management command. For example:

cd /usr/local/otm/app/opentreemap
python manage.py create_instance Philadelphia --url_name=philly --user=your_user_here --center=-75.1,40.0

Full documentation of the options available can be seen by running:

python manage.py create_instance -h

Step 11: Visit your map!

Using the url_name parameter you entered in the create_instance command in the last step, visit your new map! Fire up a browser and go to http://localhost/philly/ (or whatever your url_name was), and you should see a treemap like below!

treemap_example

Congrats!

Add a custom footer
Add a custom sidebar
Clone this wiki locally