Rack based split testing framework
Split is a rack based ab testing framework designed to work with Rails, Sinatra or any other rack based app.
Split is heavily inspired by the Abingo and Vanity rails ab testing plugins and Resque in its use of Redis.
Split uses redis as a datastore.
If you're on OS X, Homebrew is the simplest way to install Redis:
$ brew install redis $ redis-server /usr/local/etc/redis.conf
You now have a Redis daemon running on 6379.
If you are using bundler add split to your Gemfile:
Otherwise install the gem:
gem install split
and require it in your project:
Split is autoloaded when rails starts up, as long as you've configured redis it will 'just work'.
To configure sinatra with Split you need to enable sessions and mix in the helper methods. Add the following lines at the top of your sinatra app:
class MySinatraApp < Sinatra::Base enable :sessions helpers Split::Helper get '/' do ... end
To begin your ab test use the
ab_test method, naming your experiment with the first argument and then the different variants which you wish to test on as the other arguments.
ab_test returns one of the alternatives, if a user has already seen that test they will get the same alternative as before, which you can use to split your code on.
It can be used to render different templates, show different text or any other case based logic.
finished is used to make a completion of an experiment, or conversion.
<% ab_test("login_button", "/images/button1.jpg", "/images/button2.jpg") do |button_file| %> <%= img_tag(button_file, :alt => "Login!") %> <% end %>
def register_new_user #See what level of free points maximizes users' decision to buy replacement points. @starter_points = ab_test("new_user_free_points", 100, 200, 300) end
Example: Conversion tracking (in a controller!)
def buy_new_points #some business logic finished("buy_new_points") #Either a conversion named with :conversion or a test name. end
Example: Conversion tracking (in a view)
Thanks for signing up, dude! <% finished("signup_page_redesign") >
Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.
You can mount this inside your app using Rack::URLMap in your
require 'split/dashboard' run Rack::URLMap.new \ "/" => Your::App.new, "/split" => Split::Dashboard.new
You may want to password protect that page, you can do so with
Split::Dashboard.use Rack::Auth::Basic do |username, password| username == 'admin' && password == 'p4s5w0rd' end
You may want to change the Redis host and port Split connects to, or set various other options at startup.
Split has a
redis setter which can be given a string or a Redis
object. This means if you're already using Redis in your app, Split
can re-use the existing connection.
Split.redis = 'localhost:6379'
Split.redis = $redis
For our rails app we have a
config/initializers/split.rb file where
config/split.yml by hand and set the Redis information
development: localhost:6379 test: localhost:6379 staging: redis1.example.com:6379 fi: localhost:6379 production: redis1.example.com:6379
And our initializer:
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..' rails_env = ENV['RAILS_ENV'] || 'development' split_config = YAML.load_file(rails_root + '/config/split.yml') Split.redis = split_config[rails_env]
If you're running multiple, separate instances of Split you may want to namespace the keyspaces so they do not overlap. This is not unlike the approach taken by many memcached clients.
This feature is provided by the [redis-namespace][rs] library, which Split uses by default to separate the keys it manages from other keys in your Redis server.
Simply use the
Split.redis.namespace = "resque:GitHub"
We recommend sticking this in your initializer somewhere after Redis is configured.
Special thanks to the following people for submitting patches:
- Lloyd Pick
- Jeffery Chupp
Note on Patches/Pull Requests
- Fork the project.
- Make your feature addition or bug fix.
- Add tests for it. This is important so I don't break it in a future version unintentionally.
- Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
- Send me a pull request. Bonus points for topic branches.
Copyright (c) 2011 Andrew Nesbitt. See LICENSE for details.
n.b don't pass the same alternative twice!