Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Experiment Driven Development for Ruby
Ruby JavaScript
Fetching latest commit...
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Vanity is an Experience Driven-Development framework for Rails.

Requires Ruby 1.9 and Redis 1.0 or later.

A/B Testing with Rails (in 5 easy steps)

Add Vanity to your Rails app:

class ApplicationController < ActionController::Base
  use_vanity :current_user

Define an A/B test. This test compares three pricing options:

experiment "Price options" do
  description "Mirror, mirror on the wall, who's the better price of them all?"
  alternatives 19, 25, 29

Present different options to the user:

<h2>Get started for only $<%= ab_test :pricing_options %> a month!</h2>

Measure conversion:

class SignupController < ApplicationController
  def signup
    @account =[:account])
      ab_goal! :pricing_options # <- conversion
      redirect_to @acccount
      render action: :offer

Check the report:


A/B Tests

Each A/B experiment represents several (two or more) alternatives. Use the ab_test method to choose an alternative. Call ab_test without a block to return the value of the chosen alternative. Call ab_test with a block to yield with the value.

Here are some examples:

def index
  if ab_test(:new_page) # classic true/false test
    render action: "new_page"
    render action: "index"

def index
  # alternatives are names of templates 
  render template: ab_test(:new_page)

<%= if ab_test(:banner) %>100% less complexity!<% end %>

<%= ab_test(:greeting) %> <%= %>

<% ab_test :features do |count| %>
  <%= count %> features to choose from!
<% end %>

To measure conversion, call ab_goal! with the experiment's name. Typically, you would do that from a controller action, for example:

def create
  ab_goal! :new_page

To measure conversion, simply call ab_goal! with the experiment name. From the Vanity identity set by the filter we know which alternative was presented by ab_test, and can correlate conversions to alternative. It's that simple!

Managing Identity

For effective A/B tests, you want to:

  • Randomly show different alternatives to different people

  • Consistently show the same alternatives to the same person

  • Know which alternative caused a conversion

  • When running multiple tests at once, keep them independent

If you don't use any other mechanism, Vanity will assign a random value to a persistent cookie and use it to track the same visitor on subsequent visits. Cookie tracking is enabled by use_vanity.

If you keep track of users, you would want to use the user's identity instead. Using user identity is more reliable than a cookie tied to a single Web browser.

To do that, call use_vanity with the name of a method which returns an object with the desired id attribute. Alternatively, you can use a proc. These two examples are equivalent:

use_vanity :current_user
use_vanity { |controller| }

There are times when you would want to use a different identity to distinguish test alternatives. For example, your application may have groups and you may want to A/B test an option that will be available (or not) to all people in the same group.

You can tell Vanity to use a different identity on a particular controller using use_vanity. Alternatively, you can configure the experiment to extract the identity. The following example will apply to all controllers that have a project attribute (without affecting other experiments):

example "New feature" do
  description "New feature only available to some groups"
  identify { |controller| }

Configuring Vanity

Vanity will work out of the box on a default configuration. Assuming you're using Redis on localhost, post 6379, there's nothing special to do.

If you run a different setup, use the playground object to configure Vanity. For example: = "redis.local"
Vanity.playground.password = "supersecret"


EDD was all Nathaniel Talbott's idea, I had experience tests to finish for Apartly, there was coffee involved and out came the idea for Vanity.

First experiment, A/B tests, heavily influenced by Patrick McKenzie's awesome A/Bingo (

Pain points courtesy of Google Analytics's stylish graphs and too-many-clicks goal tracking process.


Vanity, copyright (C) 2009 Assaf Arkin, released under the “Use for good, not evil” license (

Something went wrong with that request. Please try again.