A repo reborn

.coveragerc

@@ -0,0 +1,6 @@
+[run]
+include =
+    toolbox/*
+
+omit =
+    toolbox/tests.py

.gitignore

@@ -0,0 +1,11 @@
+.coverage
+*.pyc
+django.log
+django.log*
+{{ project_name }}/settings_dev.py
+chef/cookbooks/palewire/files/default/id_rsa
+chef/cookbooks/palewire/files/default/id_rsa.pub
+chef/cookbooks/palewire/files/default/authorized_keys
+*.egg-*
+.static/*
+.media/*

.travis.yml

@@ -0,0 +1,16 @@
+language: python
+python:
+- '2.7'
+install:
+- pip install django
+- mkdir repo
+- django-admin.py startproject --extension=py,.gitignore --template=https://github.com/datadesk/django-project-template/archive/master.zip project repo
+- cd repo
+- pip install -r requirements.txt
+- cp project/settings_test.template project/settings_dev.py
+script:
+- pep8 toolbox
+- pyflakes toolbox
+- coverage run setup.py test
+after_success:
+  - coveralls

README.md

@@ -0,0 +1,132 @@
+<pre><code>     _  _                                    _        _     _                  _      _       
+  __| |(_)__ _ _ _  __ _ ___   _ __ _ _ ___ (_)___ __| |_  | |_ ___ _ __  _ __| |__ _| |_ ___ 
+ / _` || / _` | ' \/ _` / _ \ | '_ \ '_/ _ \| / -_) _|  _| |  _/ -_) '  \| '_ \ / _` |  _/ -_)
+ \__,_|/ \__,_|_||_\__, \___/ | .__/_| \___// \___\__|\__|  \__\___|_|_|_| .__/_\__,_|\__\___|
+     |__/          |___/      |_|         |__/                           |_|                  
+</code></pre>
+
+A custom template for initializing a new Django project the Data Desk way. 
+
+Uses the [built-in](https://docs.djangoproject.com/en/dev/ref/django-admin/#startproject-projectname-destination) Django ``startproject`` templating system. Includes a number of small modifications favored by the [Los Angeles Times Data Desk](http://datadesk.latimes.com). Assumes you already have experience hacking around on Django and PostGIS.
+
+Still experimental, so don't get your hopes up.
+
+[![Build Status](https://travis-ci.org/datadesk/django-project-template.png?branch=master)](https://travis-ci.org/datadesk/django-project-template)
+[![Coverage Status](https://coveralls.io/repos/datadesk/django-project-template/badge.png?branch=master)](https://coveralls.io/r/datadesk/django-project-template?branch=master)
+
+* Issues: [https://github.com/datadesk/django-project-template/issues](https://github.com/datadesk/django-project-template/issues)
+* Testing: [https://travis-ci.org/datadesk/django-project-template](https://travis-ci.org/datadesk/django-project-template)
+* Coverage: [https://coveralls.io/r/datadesk/django-project-template](https://coveralls.io/r/datadesk/django-project-template)
+
+Features
+--------
+
+* A split of ``settings.py`` that allows for different values in [development](https://github.com/datadesk/django-project-template/blob/master/project_name/settings_dev.template) versus [production](https://github.com/datadesk/django-project-template/blob/master/project_name/settings_prod.py)
+* Preinstallation of Django's [automatic administration panel](https://docs.djangoproject.com/en/dev/ref/contrib/admin/)
+* Preconfiguration of [urls.py](https://github.com/datadesk/django-project-template/blob/master/project_name/urls.py) to serve static, media and Munin files
+* Preconfiguration of [logging options](https://github.com/datadesk/django-project-template/blob/master/project_name/settings.py#L104)
+* Preconfiguration of [GeoDjango](https://docs.djangoproject.com/en/dev/ref/contrib/gis/) for [PostGIS](http://postgis.net/)
+* Preinstallation of [django-debug-toolbar](https://github.com/django-debug-toolbar/django-debug-toolbar)
+* Preinstallation of [django-greeking](https://github.com/palewire/django-greeking)
+* [Fabric functions](https://github.com/datadesk/django-project-template/blob/master/fabfile/) for local development and production deployment
+* Preinstallation of [tools for interacting with Amazon Web Services](https://code.google.com/p/boto/)
+* Preconfiguration of [New Relic server and Python monitoring services](https://github.com/datadesk/django-project-template/blob/master/chef/cookbooks/datadesk/recipes/newrelic.rb)
+* Preconfiguration of our preferred caching options for [development](https://github.com/datadesk/django-project-template/blob/master/project_name/settings_dev.template#L14) and [production](https://github.com/datadesk/django-project-template/blob/master/project_name/settings_prod.py#L14)
+* [Chef cookbook](https://github.com/datadesk/django-project-template/tree/master/chef) with scripted production server configuration routines
+* Management commands for scheduling [database backups](https://github.com/datadesk/django-project-template/blob/master/toolbox/management/commands/backupdb.py) to be stored in a bucket on Amazon S3 and [retrieving them](https://github.com/datadesk/django-project-template/blob/master/toolbox/management/commands/loadbackupdb.py) for local installation.
+* Custom context processors that provide the [current site](https://github.com/datadesk/django-project-template/blob/master/toolbox/context_processors/sites.py) and [environment](https://github.com/datadesk/django-project-template/blob/master/toolbox/context_processors/env.py).
+* A number of goofball utilities, like a [unicode CSV reader](https://github.com/datadesk/django-project-template/blob/master/toolbox/unicodecsv.py)
+
+Requirements
+------------
+
+* [Django](https://www.djangoproject.com/download/)
+* [PostGIS](https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installation)
+* [virtualenv](http://www.virtualenv.org/en/latest/)
+
+Getting started
+---------------
+
+Create a virtual enviroment to work inside.
+
+```bash
+$ virtualenv my-environment
+```
+
+Jump in and turn it on.
+
+```bash
+$ cd my-environment
+$ . bin/activate
+```
+
+Install Django.
+
+```bash
+$ pip install django
+```
+
+Create a new Git repository.
+
+```bash
+$ git init repo
+```
+
+Download and install a project in there using this template.
+
+```bash
+$ django-admin.py startproject --extension=py,.gitignore --template=https://github.com/datadesk/django-project-template/archive/master.zip project repo
+```
+
+Now that the template has landed, jump in and install the project's Python dependencies.
+
+```bash
+$ cd repo
+$ pip install -r requirements.txt
+```
+
+Generate a secret key.
+
+```bash
+$ fab makesecret
+```
+
+Copy the key. Open the settings file and drop it near the top. While you're there, you can also customize any of the other top level configuration options.
+
+```bash
+$ vim project/settings.py
+```
+
+Create a PostGIS database to connect with. This may vary depending on your PostGIS configuration. 
+
+The command below assumes you have it running and want to make the database with a user named ``postgres``. Please modify it to suit your needs. If you don't have PostGIS installed, try following [the GeoDjango installation instructions](https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installation).
+
+```bash
+$ createdb -U postgres -E UTF8 -T template_postgis mydatabasename
+```
+
+Make a copy of the development settings template.
+
+```bash
+$ cp project/settings_dev.template project/settings_dev.py
+```
+
+Open it and put in the credentials for the database you just made.
+
+```bash
+$ vim project/settings_dev.py
+```
+
+Sync the database.
+
+```bash
+$ python manage.py syncdb
+```
+
+Fire up the test server.
+
+```bash
+$ fab rs
+```
+
+Get to work. Once you have something worth saving you can replace this README with a description of your new project. 

chef/cookbooks/datadesk/definitions/virtualenv.rb

@@ -0,0 +1,31 @@
+define :virtualenv, :action => :create, :owner => "root", :group => "root", :mode => 0755, :packages => {} do
+    path = params[:path] ? params[:path] : params[:name]
+    if params[:action] == :create
+        # Manage the directory.
+        directory path do
+            owner params[:owner]
+            group params[:group]
+            mode params[:mode]
+        end
+        execute "create-virtualenv-#{path}" do
+            user params[:owner]
+            group params[:group]
+            command "virtualenv --no-site-packages #{path}"
+            not_if "test -f #{path}/bin/python"
+        end
+        params[:packages].each_pair do |package, version|
+            pip = "#{path}/bin/pip"
+            execute "install-#{package}-#{path}" do
+                user params[:owner]
+                group params[:group]
+                command "#{pip} install #{package}==#{version}"
+                not_if "[ `#{pip} freeze | grep #{package} | cut -d'=' -f3` = '#{version}' ]"
+            end
+        end
+    elsif params[:action] == :delete
+        directory path do
+            action :delete
+            recursive true
+        end
+    end
+end

chef/cookbooks/datadesk/files/default/apache/apache2.conf

@@ -0,0 +1,28 @@
+ServerName datadesk
+Mutex file:${APACHE_LOCK_DIR} default
+PidFile ${APACHE_PID_FILE}
+Timeout 60
+KeepAlive Off
+StartServers                2
+MinSpareThreads             2
+MaxSpareThreads             4 
+ThreadLimit                 10
+ThreadsPerChild             10
+MaxRequestWorkers           160
+MaxConnectionsPerChild      10000
+User ${APACHE_RUN_USER}
+Group ${APACHE_RUN_GROUP}
+AccessFileName .htaccess
+HostnameLookups Off
+ErrorLog ${APACHE_LOG_DIR}/error.log
+LogLevel warn
+Include mods-enabled/*.load
+Include mods-enabled/*.conf
+Include ports.conf
+Include sites-enabled/*
+LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
+LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
+LogFormat "%h %l %u %t \"%r\" %>s %b" common
+LogFormat "%v %l %u %t \"%r\" %>s %b" comonvhost
+LogFormat "%{Referer}i -> %U" referer
+LogFormat "%{User-agent}i" agent

chef/cookbooks/datadesk/files/default/init.d/pgpool2

@@ -0,0 +1,99 @@
+#! /bin/sh
+
+### BEGIN INIT INFO
+# Provides:          pgpool2
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Should-Start:      postgresql
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: start pgpool-II
+# Description: pgpool-II is a connection pool server and replication
+#              proxy for PostgreSQL.
+### END INIT INFO
+
+
+PATH=/sbin:/bin:/usr/sbin:/usr/bin
+DAEMON=/usr/sbin/pgpool
+#PIDFILE=/var/run/postgresql/pgpool.pid
+PIDFILE=/var/run/pgpool/pgpool.pid
+
+test -x $DAEMON || exit 5
+
+# Include pgpool defaults if available
+if [ -f /etc/default/pgpool2 ] ; then
+        . /etc/default/pgpool2
+fi
+
+OPTS=""
+if [ x"$PGPOOL_LOG_DEBUG" = x"yes" ]; then
+        OPTS="$OPTS -d"
+fi
+
+. /lib/lsb/init-functions
+
+
+is_running() {
+        pidofproc -p $PIDFILE $DAEMON >/dev/null
+}
+
+
+d_start() {
+        if is_running; then
+                :
+        else
+                su -c "$DAEMON -n $OPTS 2>&1 </dev/null | logger -t pgpool -p ${PGPOOL_SYSLOG_FACILITY:-local0}.info >/dev/null 2>&1 &" - postgres
+        fi
+}
+
+
+d_stop() {
+        killproc -p $PIDFILE $DAEMON -INT
+        status=$?
+        [ $status -eq 0 ] || [ $status -eq 3 ]
+        return $?
+}
+
+
+case "$1" in
+    start)
+        log_daemon_msg "Starting pgpool-II" pgpool
+        d_start
+        log_end_msg $?
+        ;;
+    stop)
+        log_daemon_msg "Stopping pgpool-II" pgpool
+        d_stop
+        log_end_msg $?
+        ;;
+    status)
+        is_running
+        status=$?
+        if [ $status -eq 0 ]; then
+                log_success_msg "pgpool-II is running."
+        else
+                log_failure_msg "pgpool-II is not running."
+        fi
+        exit $status
+        ;;
+    restart|force-reload)
+        log_daemon_msg "Restarting pgpool-II" pgpool
+        d_stop && sleep 1 && d_start
+        log_end_msg $?
+        ;;
+    try-restart)
+        if $0 status >/dev/null; then
+                $0 restart
+        else
+                exit 0
+        fi
+        ;;
+    reload)
+        exit 3
+        ;;
+    *)
+        log_failure_msg "Usage: $0 {start|stop|status|restart|try-restart|reload|force-reload}"
+        exit 2
+        ;;
+esac
+

chef/cookbooks/datadesk/files/default/memcached/memcached.conf

@@ -0,0 +1,6 @@
+-d
+logfile /var/log/memcached.log
+-m 256
+-p 11211 
+-u nobody
+-l 127.0.0.1

chef/cookbooks/datadesk/files/default/motd/caw.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+
+printf '
+                                          _..                                              
+  ____     _   _____   _     ___..-"""-.  `)^|   .-"""-..___    ____   _____  ____   _  __ 
+ |  _ \   / \ |_   _| / \   `-...___ `=.`-.`  \-`.=` ___...-`  |  _ \ | ____|/ ___| | |/ / 
+ | | | | / _ \  | |  / _ \          `\  ` ##### `  /`          | | | ||  _|  \___ \ | ` /  
+ | |_| |/ ___ \ | | / ___ \           `--;|||||;--`            | |_| || |___  ___) || . \  
+ |____//_/   \_\|_|/_/   \_\             /\|||/\               |____/ |_____||____/ |_|\_\ 
+_______________________________________ ( /;-;\ ) _________________________________________
+                                         `-...-`                                           
+'