Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Rack::AB module for A/B testing #35

Closed
wants to merge 14 commits into from

2 participants

@erikeldridge

Default settings assign users to one of two buckets, "a" and "b", and store the bucket name in a cookie called "rack_ab". If a bucket has not yet been assigned for a user, one is randomly chosen. The bucket name is made available to the app via the env object.

Basic usage looks like this:

use Rack::AB
# ...
if 'a' == env['rack.ab.bucket_name']
  body = 'content for bucket a'
else
  body = 'content for bucket b'
end
# ...
[200, {}, body]

Configuration options allow:

  • custom cookie name, e.g., "bucket_name" instead of "rack_ab"
  • custom bucket names, e.g. "control" and "fancy_bucket" instead of "a" and "b"
  • custom cookie parameters such as path, domain, and expires.

Custom configuration looks like this:

use Rack::AB, :bucket_names => ['control', 'new_feature']
# ...

Any and all feedback welcome.

Erik Eldridge added some commits
Erik Eldridge Create AB testing middleware for Rack
Define basic functionality and initial spec tests
7890851
Erik Eldridge Simplify args passed to Rack::AB 54560f5
Erik Eldridge Correct assertion logic
Since Rack::AB can return 'a' or 'b' randomly, verify that
an array of correct response values includes the value we receive.
4a0c3c6
Erik Eldridge Write test to verify no cookie set if one exists
Rack::AB should only set a cookie if one has not been set before
3110683
Erik Eldridge Create test for custom cookie name ecab8e1
Erik Eldridge Update initialize to accept params hash
Simplify arg syntax for initializer by accepting a hash instead
of ordered args
d520de4
Erik Eldridge Create test for custom possible values
Developers should be able to define custom values to
use in the cookie. Create a test to verify
6c9be20
Erik Eldridge Write test to verify cookie expiration setting 200765f
Erik Eldridge Rename possible_values to bucket_names
bucket_names is more self-descriptive.
2551bbe
Erik Eldridge Add Rack::AB to autoload 48afa95
Erik Eldridge Add bucket name to env obj, and comment code 009acc1
Erik Eldridge Add test for splitting traffic 8117831
Erik Eldridge Remove .gitignore from repo ad9e0b4
Erik Eldridge Add Rack::AB to readme ceb20d9
@josh

Sorry, but we aren't accepting any new middleware.

I'd suggest staring your own rack-ab project on github.

@josh josh closed this
@erikeldridge

Thanks for the quick response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 6, 2011
  1. Create AB testing middleware for Rack

    Erik Eldridge authored
    Define basic functionality and initial spec tests
  2. Simplify args passed to Rack::AB

    Erik Eldridge authored
  3. Correct assertion logic

    Erik Eldridge authored
    Since Rack::AB can return 'a' or 'b' randomly, verify that
    an array of correct response values includes the value we receive.
  4. Write test to verify no cookie set if one exists

    Erik Eldridge authored
    Rack::AB should only set a cookie if one has not been set before
  5. Create test for custom cookie name

    Erik Eldridge authored
  6. Update initialize to accept params hash

    Erik Eldridge authored
    Simplify arg syntax for initializer by accepting a hash instead
    of ordered args
  7. Create test for custom possible values

    Erik Eldridge authored
    Developers should be able to define custom values to
    use in the cookie. Create a test to verify
  8. Write test to verify cookie expiration setting

    Erik Eldridge authored
Commits on Apr 7, 2011
  1. Rename possible_values to bucket_names

    Erik Eldridge authored
    bucket_names is more self-descriptive.
Commits on Apr 12, 2011
  1. Add Rack::AB to autoload

    Erik Eldridge authored
  2. Add bucket name to env obj, and comment code

    Erik Eldridge authored
  3. Add test for splitting traffic

    Erik Eldridge authored
  4. Remove .gitignore from repo

    Erik Eldridge authored
  5. Add Rack::AB to readme

    Erik Eldridge authored
This page is out of date. Refresh to see the latest.
View
4 .gitignore
@@ -1,4 +0,0 @@
-/pkg
-/doc
-/RDOX
-ChangeLog
View
1  README.rdoc
@@ -44,6 +44,7 @@ interface:
* Rack::AcceptFormat - Adds a format extension at the end of the URI when there is none, corresponding to the mime-type given in the Accept HTTP header.
* Rack::HostMeta - Configures /host-meta using a block
* Rack::Cookies - Adds simple cookie jar hash to env
+* Rack::AB - Splits traffic for A/B testing
* Rack::Access - Limits access based on IP address
* Rack::ResponseHeaders - Manipulates response headers object at runtime
* Rack::SimpleEndpoint - Creates simple endpoints with routing rules, similar to Sinatra actions
View
1  lib/rack/contrib.rb
@@ -7,6 +7,7 @@ def self.release
end
end
+ autoload :AB, "rack/contrib/ab"
autoload :AcceptFormat, "rack/contrib/accept_format"
autoload :Access, "rack/contrib/access"
autoload :BounceFavicon, "rack/contrib/bounce_favicon"
View
76 lib/rack/contrib/ab.rb
@@ -0,0 +1,76 @@
+module Rack
+
+ # The Rack::AB middleware splits users into buckets by selecting randomly from
+ # a collection of bucket names, setting this value in a cookie, and passing this
+ # value into the app via the env object.
+ #
+ # Example
+ #
+ # use Rack::AB,
+ # :cookie_name => 'new_cookie_name',
+ # :bucket_names => [1,2,3]
+ #
+ # creates three buckets, 1, 2, and 3, randomly assigning each bucket name to a user, and
+ # storing the bucket name in a cookie called 'new_cookie_name', e.g., new_cookie_name=1
+ #
+ # Usage
+ #
+ # 1) Use Rack::AB middleware via 'use' directive
+ # 2) Set options as desired
+ # 3) Check bucket value inside Rack app via the 'rack.ab.bucket_name' member of the
+ # env object.
+ # 4) Use this bucket name to split your traffic, eg if 'a' == env['rack.ab.bucket_name']:
+ # ...; elsif 'b' == env['rack.ab.bucket_name']: ... end
+ #
+
+ class AB
+
+ def initialize(app, options={})
+ @app = app
+
+ # Name of cookie for storing bucket name
+ @cookie_name = options[:cookie_name] || 'rack_ab'
+
+ # Collection of bucket names to pick randomly from
+ @bucket_names = options[:bucket_names] || ['a', 'b']
+
+ # Params to pass into Rack::Response::set_cookie
+ # http://rack.rubyforge.org/doc/classes/Rack/Response.src/M000179.html
+ @cookie_params = options[:cookie_params] || {}
+
+ end
+
+ def call(env)
+
+ req = Request.new(env)
+
+ # If user hasn't been assigned a bucket
+ if req.cookies[@cookie_name].nil?
+
+ bucket_name = @bucket_names[rand(@bucket_names.length)]
+
+ # Add bucket name to env so app can split traffic internally
+ env["rack.ab.bucket_name"] = bucket_name
+
+ status, headers, body = @app.call(env)
+
+ resp = Rack::Response.new(body, status, headers)
+
+ # Set cookie w/ bucket name
+ @cookie_params[:value] = bucket_name
+ resp.set_cookie(@cookie_name, @cookie_params)
+ resp.finish
+
+ else
+
+ env["rack.ab.bucket_name"] = req.cookies[@cookie_name]
+
+ status, headers, body = @app.call(env)
+
+ [status, headers, body]
+ end
+
+ end
+
+ end
+end
View
80 test/spec_rack_ab.rb
@@ -0,0 +1,80 @@
+require 'test/spec'
+require 'rack/mock'
+
+context "Rack::AB" do
+ specify "should return 'a' or 'b' if no cookie is set" do
+ app = lambda { |env|
+ [200, {'Content-Type' => 'text/plain'}, '']
+ }
+ app = Rack::AB.new(app)
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => '')
+ response.headers['Set-Cookie'].should =~ /rack_ab=[a,b]/
+ end
+
+ specify "should not set a cookie if one is already defined" do
+ app = lambda { |env|
+ [200, {'Content-Type' => 'text/plain'}, '']
+ }
+ app = Rack::AB.new(app)
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => 'rack_ab=a')
+ response.headers['Set-Cookie'].should == nil
+ end
+
+ specify "provides a way to set a custom cookie name" do
+ app = lambda { |env|
+ [200, {'Content-Type' => 'text/plain'}, '']
+ }
+ app = Rack::AB.new(app, :cookie_name => 'new_cookie_name')
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => '')
+ response.headers['Set-Cookie'].should =~ /new_cookie_name=[a,b]/
+ end
+
+ specify "provides a way to set a custom cookie values" do
+ app = lambda { |env|
+ [200, {'Content-Type' => 'text/plain'}, '']
+ }
+ app = Rack::AB.new(app, :bucket_names => [1,2,3])
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => '')
+ response.headers['Set-Cookie'].should =~ /rack_ab=[1,2,3]/
+ end
+
+ specify "provides a way to set cookie expiration" do
+
+ expiration = Time.now+24*60*60
+
+ app = lambda { |env|
+ [200, {'Content-Type' => 'text/plain'}, '']
+ }
+ app = Rack::AB.new(app, :cookie_params => { :expires => expiration})
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => '')
+
+ pattern = "rack_ab=[a,b]; expires=(.*)"
+ regexp = Regexp.new(pattern)
+ matches = regexp.match response.headers['Set-Cookie']
+
+ Time.parse(matches.captures.first).to_s.should.equal expiration.to_s
+
+ end
+
+ specify "provides a way to split traffic inside app" do
+ app = lambda { |env|
+ if 'a' == env['rack.ab.bucket_name']
+ body = 'content for bucket a'
+ elsif 'b' == env['rack.ab.bucket_name']
+ body = 'content for bucket b'
+ end
+ [200, {'Content-Type' => 'text/plain'}, body]
+ }
+ app = Rack::AB.new(app)
+
+ response = Rack::MockRequest.new(app).get('/', 'HTTP_COOKIE' => 'rack_ab=a')
+
+ response.body.should.equal 'content for bucket a'
+ end
+
+end
Something went wrong with that request. Please try again.