Bestcase is an A/B and multivariate testing library for Clojure.
Latest commit 464e261 Dec 10, 2015 @charlespwd Merge pull request #6 from crclark/redis-problems
Fix failing Redis tests
Failed to load latest commit information.
src/bestcase fix get-user-scored-count returning string instead of long Jul 7, 2015
test/bestcase Uncommented out redis tests from core_test. Tests are failing. Feb 19, 2015
.travis.yml Add booting the redis server to the travis build Feb 11, 2015 bump up version number Feb 19, 2015
LICENSE.html Initial commit. Nov 23, 2012 bump up version number Feb 19, 2015
project.clj bump up version number Feb 19, 2015


Build Status

Bestcase is an A/B and multivariate testing library for Clojure.

Version Number


  • Create an A/B test in one line of code
  • Measure multiple independent goals per test
  • See results and pick winners through a web interface (or programmatically)
  • It's very fast
  • Store tests and results (i) in-memory or (ii) in a Redis store
  • Written for programmers (yes, that's a feature!)


The easiest way to get started with bestcase is to add it to your leiningen dependencies in your project.clj file:

[com.cpclermont/bestcase "0.2.1"]

You can run the tests using lein with-profile dev midje. By default, the tests use the in-memory store.


Check out the User Guide and the Examples Page for detailed documentation.

(ns your-app
  (:require [bestcase :as bc]
            [ :as bcm]))

;; Use the in-memory store to keep track of tests
(bc/set-config! {:store (bcm/create-memory-store)})


;; Let's test how different purchase buttons colors affect conversions.
;; We have three alternatives to which users are allocated evenly:
(bc/alt :purchase-button-test
        :red-button   "/images/button1.jpg"
        :green-button "/images/button2.jpg"
        :blue-button  "/images/button3.jpg")


;; Let's track conversions whenever a user purchases something
(bc/score :purchase-button-test :purchase)

You can see results programmatically:

;; We use the :red-button as the control variable
(bc/results :purchase-button-test :red-button)
;; => {:test-name :purchase-button-test
;;     :test-type :ab-test
;;     :alternatives ({:alternative-name :red-button
;;                     :count 182
;;                     :control true
;;                     :goal-results ({:goal-name :purchase
;;                                     :score 35
;;                                     :z-score: 0.0})}
;;                    {:alternative-name :green-button
;;                     :count 180
;;                     :goal-results ({:goal-name :purchase
;;                                     :score 45
;;                                     :z-score: 1.3252611961151077})}
;;                    {:alternative-name :blue-button
;;                     :count 188
;;                     :goal-results ({:goal-name :purchase
;;                                     :score 61
;;                                     :z-score: 2.941015722492861})})}

or through your webapp by using bestcase's Ring handler:

(ns your-app
  (:require [bestcase.util.ring :as bcr]))

(bcr/dashboard-routes "/bestcase" {:css ["/css/custom.css"]
                                   :js ["/js/custom.js"]})

This returns a handler that matches the route /bestcase to provide you a dashboard that:

  • lists all active tests;
  • shows you results for those tests;
  • allows you to pick a winner for any active test without having to change any code or restart your server; and
  • can be customized through css and javascript if you desire it.

Learning Bestcase

Bestcase has a lot more functionality which you can learn about in the User Guide. With bestcase, you can:

  • end a test and choose a winner programmatically;
  • put different weights on alternatives so that some are tested more often than others;
  • toggle whether a user can count as a test participate or complete a goal more than once;
  • change backend store through (set-config! ...) and (with-config ...);
  • exclude bots;
  • force different alternatives from your browser using the query string (mainly for testing);
  • pretty-print results on the command-line; and
  • more.



The Performance Section of the User Guide contains arbitrary, non-rigorous performance non-benchmarks.

  • If you are handling hundreds (100s) of requests per second, you will be just fine.
  • If you are handling many thousands (1000s) of requests per second, benchmark bestcase first, but you should be fine.
  • If you are handling many tens of thousands (10000s) of requests per second you aren't reading this.

If you are encountering performance bottlenecks, let me know and I'll spend some time optimizing things and implementing a hybrid backend (in-memory + Redis).

Further Reading On A/B Testing

If you are looking to run multi-armed bandit tests in Clojure, check out Touchstone. Here are a few articles on A/B testing versus multi-armed bandit testing:

A/B versus bandit: either way, you'll be better off than not testing and that's the thing that matters most.


This project borrows from the A/Bingo Ruby testing framework by Patrick McKenzie. Thank you for your work.

I'd also like to thank the authors of the following Clojure libraries, which are used under the hood:

  • Ring (simple webapps in Clojure)
  • Compojure (routes for Ring)
  • Hiccup (library for representing html in Clojure)
  • Midje (a fun and powerful testing framework)


Copyright © 2012 Jean-Denis Greze, original author, github
Copyright © 2015 Charles-Philippe Clermont

Like Clojure, bestcase is distributed under the Eclipse Public License.