Automated deploys for CKAN
Ansible roles for automating deploys and rollbacks for CKAN custom installations that need to manage several extensions on active development, and that need to be deployed into several environments.
This set of roles and scripts provides a framework when working on custom developments of CKAN which involve setting up, developing and deploying several extensions. The main issues it tries to solve are:
- Deploying releases of several CKAN extensions in a capistrano-like way. Each of them coming from different origins:
- Open source extensiones developed outisde your organization
- Forked open source extensions adapted for our use case
- Private extensions developed by your organization
- Ability to easily rollback a failed deploy
- Encapsulating Python packages dependencies for each of the deploys (we used virtualenv for this), avoiding possible conflicts when releasing a new version.
- Deploying some minor changes to the CKAN core we needed, without forking CKAN.
- Deploying to several environments (production and staging), without duplicating the tasks for each of this roles.
The repository is organized as follows:
bin/ # easy-to-run scripts for each action production_deploy.sh production_first_deploy.sh staging_deploy.sh staging_first_deploy.sh sites/ ckan/ files/ tasks/ # tasks for cold deploy, deploy and rollback deploy.yml first_deploy.yml main.yml rollback.yml templates/ # CKAN config file, parameterized for each environment ckan_config.ini.j2 vars/ # encrypted vaults for each environment production_secrets.yml secrets.yml.example staging_secrets.yml ckan_production.yml ckan_staging.yml inventory.example inventory
We assume you have installed CKAN following the official guide. In our case, we did it with CKAN 2.7.2 on a CentOS 7 machine, following this guide.
We assume you have the following users and groups in your server:
Clone the repo:
git clone https://github.com/populatetools/ckan-deploys
inventory.example and edit it with your server IPs:
cd ckan-deploys cp inventory.example inventory vim inventory
Fill in the vault files inside
sites/ckan/vars according to your own needs and encrypt them. You can copy the
secrets.yml.example file to use it as a template:
cd sites/ckan/vars cp secrets.yml.example production_secrets.yml cp secrets.yml.example staging_secrets.yml
Edit the variables in
sites/ckan/vars/ckan_config.yml and give them the values that match your needs. Some of them are parameters for the CKAN config file
first_deploy.yml tasks prepare the default CKAN instalation to support the workflow of deploying releases. A summary of the steps is:
- Creates the releases directory
- Creates the symlink
/usr/lib/ckan/currentthat now points to the default CKAN virtualenv, but that later will point to the current release.
- Copies the CKAN configuration file inside the default virtualenv.
- Modifies the
/etc/ckan/default/apache.wsgifile to point to the
currentsymlink, instead of
- Sets the domain in the virtualhost file
bin/production_first_deploy.sh you should have a working CKAN application accessible via web, and prepare for the next deploys.
deploy.yml tasks deploys a new release of CKAN, with the specified extensions and the changes performed in the CKAN configuration file. You can run it with
bin/production_deploy.sh and a summary of its steps is:
- Clones the default virtualenv into the releases directory, inside a timestamped directory (ex.
- Clones the specified extensions directory inside the release virutalenv, install the extensions in the virtualenv and their dependencies (presumed to be in a
- Updates the release CKAN configuration file (ex.
- Changes the
currentsymlink to point to the last release.
- If the amount of kept releases was exceeded, removes the oldest release.
The rollback procedure can be run with
bin/production_rollback.sh. A summary of the steps is the following:
- Points the
currentsymlink to the previous release.
- Removes the broken release from
This repository contains an adaptation of the code we used. We've tested it but not extensively, and probably you'll need to adapt some parts of it to your own needs.
This tools have been developed by Populate, a studio which designs and builds products around civic engagement, data journalism and open data.