Vanity is an Experiment Driven Development framework for Rails.
All about Vanity: vanity.labnotes.org
Original version on github: github.com/assaf/vanity
The verticalbrands fork: github.com/verticalbrands/vanity
Changes in the verticalbrands fork
To update your vanity database for the features in the verticalbrands fork, run:
$ rails generate vanity --skip $ rake db:migrate
Enabling/disabling an experiment through the cpanel. Specify a “default” alternative in the experiment specification; disabled experiments will always show that option. e.g.
ab_test "Price options" do description "Mirror, mirror on the wall, who's the better price of all?" alternatives 19, 25, 29 default 25 metrics :signups end
While disabled, experiments will not be tracked as participants and metrics will not be recorded.
Experiments start off disabled. You will have to enable them, either programmatically or through the cpanel. This will affect currently running experiments (i.e. they will be disabled).
If collecting is off, experiments are always seen as enabled. A finished experiment will always be disabled. The following table summarizes experiment states:
collecting | active | enabled | showing -------------------------------------------- T | T | T | alternative <- only case when data is collected T | T | F | default T | F | T | (not possible, !active => !enabled) T | F | F | outcome F | T | T | alternative F | T | F | (not possible, !collecting => enabled && active) F | F | T | (not possible, !collecting => enabled && active) F | F | F | (not possible, !collecting => enabled && active)
Clicking the 'reset experiment' button associated with an experiment on the dashboard will delete all collected data and restart the experiment from scratch. Complete experiments can be restarted this way (although all previously collected data will be deleted; if you want to simply pause an experiment, use the enable/disable feature).
Manually complete experiments
Clicking the 'complete' button next to an option will complete the associated AB Test and set that option as the outcome.
Tracking multiple metrics for each alternative
AB Tests now keeps track of metrics for each alternative in addition to conversions. Unlike conversions, metrics are tracked multiple times per participant. Note: this feature is only intended for metric tracking using the track! method.
A/B Testing With Rails (In 5 Easy Steps)
Step 1: Start using Vanity in your Rails application:
Rails 2.x configuration
Rails::Initializer.run do |config| gem.config "vanity" config.after_initialize do require "vanity" end end
Rails 3 configuration
Add to your Gemfile:
If using a relational database, run the generator and migrations to create the database schema:
$ rails generate vanity $ rake db:migrate
Add to your application controller:
class ApplicationController < ActionController::Base use_vanity :current_user end
Step 2: Define your first A/B test. This experiment goes in the file experiments/price_options.rb:
ab_test "Price options" do description "Mirror, mirror on the wall, who's the better price of all?" alternatives 19, 25, 29 default 25 metrics :signups end NOTE: If using a metric as above ("signups"), there needs to be a corresponding ruby file for that metric. Inside the "experiments" directory create a "metrics" directory with a file called "signups.rb". The contents of the file can describe the signup metric, refer to the "Metrics" Vanity documentation page for an example.
Step 3: Present the different options to your users:
<h2>Get started for only $<%= ab_test :price_options %> a month!</h2>
Step 4: Measure conversion:
class SignupController < ApplicationController def signup @account = Account.new(params[:account]) if @account.save track! :signups redirect_to @acccount else render action: :offer end end end
Step 5: Check the report:
vanity report --output vanity.html
There is currently an issue with report generation. The vanity-talk Google Group has a couple posts that outline the issue for now. This is one of the posts: groups.google.com/group/vanity-talk/browse_thread/thread/343081a72a0cefb6
If you are collecting data (in development you need to opt-in to this by setting Vanity.playground.collecting = true in environments/development.rb) you can view experiment results with the vanity dashboard instead of the report.
The vanity dashboard setup instructions with Vanity work for Rails 3.x except the route is different. A Rails 3.x-style route would look like this:
`match '/vanity(/:action(/:id(.:format)))', :controller=>:vanity`
Set Vanity.playground.add_participant_path = '/path/to/vanity/action', this should point to the add_participant path that is added with Vanity::Rails::Dashboard, make sure that this action is available to all users
Add <%= vanity_js %> to any page that needs uses an ab_test. vanity_js needs to be included after your call to ab_test so that it knows which version of the experiment the participant is a member of. The helper will render nothing if the there are no ab_tests running on the current page, so adding vanity_js to the bottom of your layouts is a good option. Keep in mind that if you call use_js! and don't include vanity_js in your view no participants will be recorded.
Fork the project
Please use a topic branch to make your changes, it's easier to test them that way
To set up the test suite run bundler, then run `rake appraisal:install` to prepare the test suite to run against multiple versions of Rails
Fix, patch, enhance, document, improve, sprinkle pixie dust
At minimum run `rake appraisal test`, if possible, please run rake test:all
Tests. Please. Run rake test, of if you can, rake test:all
Send a pull request on GitHub
Original code, copyright of Assaf Arkin, released under the MIT license.
Documentation available under the Creative Commons Attribution license.
For full list of credits and licenses: vanity.labnotes.org/credits.html.