Our collection of opinionated Capistrano recipes
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib
.gitignore
Gemfile
LICENSE.txt
README.md
Rakefile
matross.gemspec

README.md

matross

Code Climate Dependency Status

Matross is our collection of opinionated Capistrano recipes. We made a bunch of additions and customizations. Below we list the most relevant ones.

  • Foreman by default: we use foreman to environment variables, init scripts, task definitions and more.
  • Custom foreman upstart template: we also leverage foreman's templates to build a custom upstart template that enables console log, allowing logrotate to work properly.

Usage

Put matross in the :development group of your Gemfile:

group :development do
  gem 'matross'
end

Run bundle exec capify . in the project root folder:

$ bundle exec capify .

Find a full example down this README.

Overriding default templates

We have our opinions, but don't know everything. What works for us, may not fit your needs since each app is a unique snowflake. To take care of that, matross allows you to define your own templates instead of the built in ones. Look at the included ones in lib/matross/templates to see how we think things should go.

Managing application daemons with Foreman

Foreman has freed us of the tedious task of writing init and Upstart scripts. Some of our matross recipes automatically add processes - such as the unicorn server - to the Procfile.

If you have an application Procfile with custom daemons defined, such as Rake task, they will be concatenated with all the processes defined in matross, resulting in one final Procfile-matross file that will be used to start your application and export init scrips.

You can specify the number of each instance defined in Procfile-matross using the foreman_procs variable. Suppose you have a process called dj and want to export 3 instances of it:

set :foreman_procs, {
    dj: 3
}

We also modified the default upstart template to log through upstart instead of just piping stdout and stderr into files. Goodbye nocturnal logexplosion. (Like all templates you can override it!).

If you have custom tasks that should also be started, simply list them in the Procfile in the root of your application. They will be appended to the recipe's task definitions (eg.: unicorn).

custom_task: bundle exec rake custom_task

If there are any environment variables that you want to use, just set them in a .env file in the root of your application. Please note that RAILS_ENV is properly set during foreman tasks.

CUSTOM_TASK_ENV=boost

If you require different values for these environment variables across deployment stages, define them separately in .env-#{stage} files, for example .env-production.

Recipes

Foreman

Requires having foreman available in the application. As mentioned before, we use foreman in production to save us from generating upstart init scripts. As a bonus we get sane definition of environment variables.

Overwritable template: process.conf.erb

Variables

Variable Default value Description
:foreman_user { user } - The user defined in Capistrano The user which should run the tasks defined in the Procfile
:foreman_bin 'bundle exec foreman' The foreman command
:foreman_procs {} - Defaults to one per task definition Number of processes for each task definition in the Procfile

Tasks

Task Description
foreman:pre_setup Creates the upstart folder in the shared_path
foreman:setup Merges all partial Procfiles and .envs, including the appropriate RAILS_ENV
foreman:export Export the task definitions as Upstart scripts
foreman:symlink Symlink .env-matross and Procfile-matross to current_path
foreman:log Symlink Upstart logs to the log folder in shared_path
foreman:stop Stop all of the application tasks
foreman:restart Restart or start all of the application tasks
foreman:remove Remove all of the application tasks from Upstart

Unicorn

Requires having unicorn available in the application. By loading our unicorn recipe, you get our default configuration.

Overwritable template: unicorn.rb.erb Procfile task: web: bundle exec unicorn -c <%= unicorn_config %> -E <%= rails_env %>

Variables

Variable Default value Description
:unicorn_config "#{shared_path}/config/unicorn.rb" Location of the configuration file
:unicorn_log "#{shared_path}/log/unicorn.log" Location of unicorn log
:unicorn_workers Number of cores specified by /proc/cpuinfo Number of unicorn workers

Tasks

Task Description
unicorn:setup Creates the unicorn.rb configuration file in the shared_path
unicorn:procfile Defines how unicorn should be run in a temporary Procfile

Nginx

This recipes creates and configures the virtual_host for the application. [This virtual host] has some sane defaults, suitable for most of our deployments (non-SSL). The file is created at /etc/nginx/sites-available and symlinked to /etc/nginx/sites-enabled. These are the defaults for the Nginx installation in Ubuntu. You can take a look at our general nginx.conf.

Overwritable template: nginx_virtual_host_conf.erb

Variables

Variable Default value Description
:htpasswd None htpasswd user:password format
:nginx_default_server false Sets the vhost for the specified stage as the default

Tasks

Task Description
nginx:setup Creates the virtual host file
nginx:reload Reloads the Nginx configuration
nginx:lock Sets up the a basic http auth on the virtual host
nginx:unlock Removes the basic http auth

MySQL

Requires having mysql2 available in the application. In our MySQL recipe we dynamically generate a database.yml based on the variables that should be set globally or per-stage.

The backup routine requires s3cmd installed and properly configured. The key pair must have write access to the backup bucket on S3.

Overwritable template: database.yml.erb

Variables

Variable Default value Description
:database_config "#{shared_path}/config/database.yml" Location of the configuration file
:mysql_host None MySQL host address
:mysql_database None MySQL database name. Dashes - are gsubed for underscores _
:mysql_user None MySQL user
:mysql_passwd None MySQL password
:mysql_backup_script "#{shared_path}/matross/mysql_backup.sh" MySQL backup script location
:mysql_backup_cron_schedule '30 3 * * *' Cron schedule for the backup script
:mysql_backup_bucket None Bucket used to store the dumps
:mysql_backup_prefix None Flat file prefix for the backup files

Tasks

Task Description
mysql:setup Creates the database.yml in the shared_path
mysql:symlink Creates a symlink for the database.yml file in the current_path
mysql:create Creates the database if it hasn't been created
mysql:schema_load Loads the schema if there are no tables in the DB
mysql:backup:setup Creates the backup script and configures the user's cron - Not hooked to any other task
mysql:dump:do Dumps the application database
mysql:dump:get Downloads a copy of the last generated database dump
mysql:dump:apply Applies the latest dump generated stored in 'dumps' locally
mysql:dump:post Uploads a copy of the last generated database dump
mysql:dump:apply_remotely Applies the latest uploaded dump in the remote server

PostgreSQL

Requires having pg available in the application. In our PostgreSQL recipe we dynamically generate a database.yml based on the variables that should be set globally or per-stage.

Please note that we rely on local peer connection, hence, remote connections should use a custom template for database.yml.erb. The PG cluster must have UTF8 encoding.

Overwritable template: database.yml.erb

Variables

Variable Default value Description
:database_config "#{shared_path}/config/database.yml" Location of the configuration file
:postgresql_database None PostgreSQL database name. Dashes - are gsubed for underscores _
:postgresql_user user PostgreSQL user

Tasks

Task Description
postgresql:setup Creates the database.yml in the shared_path, and the user if needed
postgresql:symlink Creates a symlink for the database.yml file in the current_path
postgresql:create Creates the database if it hasn't been created and loads the schema

Mongoid

Requires having mongoid available in the application. In our Mongoid recipe we dynamically generate a mongoid.yml based on the variables that should be set globally or per-stage.

Overwritable template: mongoid.yml.erb

Variables

Variable Default value Description
:mongoid_config "#{shared_path}/config/mongoid.yml" Location of the mongoid configuration file
:mongo_hosts None List of MongoDB hosts
:mongo_database None MongoDB database name
:mongo_user None MongoDB user
:mongo_passwd None MongoDB password

Tasks

Task Description
mongoid:setup Creates the mongoid.yml in the shared_path
mongoid:symlink Creates a symlink for the mongoid.yml file in the current_path

Delayed Job

Requires having delayed_job available in the application.

Procfile task: dj: bundle exec rake jobs:work or dj_<%= queue_name %>: bundle exec rake jobs:work QUEUE=<%= queue_name %>

Variables

Variable Default value Description
:dj_queues None List of queues

Tasks

Task Description
delayed_job:procfile Defines how delayed_job should be run in a temporary Procfile

Fog (AWS)

Requires having fog available in the application. When we use fog, it is for interacting with Amazon services, once again very opinionated.

Overwritable template: fog_config.yml.erb

The configuration generated may be used by other gems, such as carrierwave. Here is an example of how we use it:

# config/initializers/carrierwave.rb
CarrierWave.configure do |config|
  fog_config = YAML.load(File.read(File.join(Rails.root, 'config', 'fog_config.yml')))
  config.fog_credentials = {
    :provider               => 'AWS',
    :aws_access_key_id      => fog_config['aws_access_key_id'],
    :aws_secret_access_key  => fog_config['aws_secret_access_key'],
    :region                 => fog_config['region']
  }
  config.fog_directory  = fog_config['directory']
  config.fog_public     = fog_config['public']
end

Variables

Variable Default value Description
:fog_config "#{shared_path}/config/fog_config.yml" Location of the fog configuration file
:fog_region 'us-east-1' AWS Region
:fog_public false Bucket policy
:fog_aws_access_key_id None AWS Access Key Id
:fog_aws_secret_access_key None AWS Secret Access Key

Tasks

Task Description
fog:setup Creates the fog_config.yml in the shared_path
fog:symlink Creates a symlink for the fog_config.yml file in the current_path

Faye

Requires having faye available in the application.

Overwritable templates: faye.ru.erb and faye_server.yml Procfile task: faye: bundle exec rackup <%= faye_ru %> -s thin -E <%= rails_env %> -p <%= faye_port %>

Variables

Variable Default value Description
:faye_config "#{shared_path}/config/faye_config.yml" Location of the faye parameters configuration file
:faye_ru "#{shared_path}/config/faye.ru" Location of the faye configuration file
:faye_port None Which port faye should listen on

Tasks

Task Description
faye:setup Creates faye_config.yml and faye.ru in the shared_path
faye:symlink Creates a symlink for the faye_config.yml file in the current_path

Local Assets

In order to deal with memory or CPU constrained production servers, this recipe overwrites the default assets precompilation by compiling them locally and then uploading the result to the server.

Rake precompiles assets in a production to properly generate file names with hashes. One of the gotchas to this is that by default Rails initializes the entire applicaction when executing the assets:precompile task. If you use different databases in production and development and run into database connection errors with this task you can override this behavior.

config/application.rb

module AwesomeApplication
  class Application < Rails::Application
    ...
    config.assets.initialize_on_precompile = false if Rails.env.production?
    ...
  end
end

Full Example

Below is a full example of how to use matross exhaustively. Do note that this would be an edge case as, for example, you don't normally run mongoid with mysql.

config/deploy.rb

set :stages, %w(production staging)
set :default_stage, 'staging'
require 'capistrano/ext/multistage'
require 'bundler/capistrano'
require 'matross'
load 'matross/local_assets'
load 'matross/nginx'
load 'matross/unicorn'
load 'matross/faye'
load 'matross/delayed_job'
load 'matross/fog'
load 'matross/mongoid'
load 'matross/mysql'
load 'matross/foreman'

set :application,           'awesome_application'
set :repository,            'git@github.com:innvent/awesome_application.git'
set :ssh_options,           { :forward_agent => true }
set :scm,                   :git
set :scm_verbose,           true
set :deploy_via,            :remote_cache
set :shared_children,       %w(public/system log tmp/pids public/uploads)
default_run_options[:pty]   = true

logger.level = Capistrano::Logger::DEBUG

after 'deploy:update', 'deploy:cleanup'

config/deploy/production.rb

set :user,                      'ubuntu'
set :group,                     'ubuntu'
set :use_sudo,                  false
set :branch,                    'master'
set :rails_env,                 'production'
set :deploy_to,                 "/home/#{user}/#{application}"
set :server_name,               'example.com'

set :mongo_hosts,               [ 'localhost' ]
set :mongo_database,            "#{application}_#{rails_env}"

set :mysql_host,                'localhost'
set :mysql_database,            "#{application}_#{rails_env}"
set :mysql_user,                "#{user}"
set :mysql_passwd,              ''

set :faye_port,                 '9292'
set :faye_local,                true

set :htpasswd,                  'admin:$apr1$twOdKBdh$okL.giy91y9LzXsD5swUb0'

set :dj_queues,                 [ 'queue1', 'queue2' ]

set :fog_aws_access_key_id,     'AKIACS5E2GU9NND0NMD4'
set :fog_aws_secret_access_key, 'rNB38h5Y4ysUM3r10F3oehrnp2ZaBcUPtiOnJyLn'
set :fog_directory,             'awesome_application_production'

set :foreman_procs,             { 'dj_queue1' => 2, 'dj_queue2' => 3 }

server '192.168.1.1', :app, :web, :dj, :faye, :db, :primary => true

set :default_environment, {
  'PATH' => "/home/#{user}/.rbenv/shims:/home/#{user}/.rbenv/bin:$PATH"
}

Capfile

load 'deploy'
load 'deploy/assets'
load 'config/deploy'