Skip to content
A source-controlled split testing framework that lets you build, launch and analyse experiments via Git/CI.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE
README.md
logo.png
modules.png
reports.png Init Jul 11, 2019

README.md

Mojito logo

Mojito

A source-controlled split testing framework that lets you build, launch and analyse experiments via Git/CI.

Our Delivery JS library is a heavily modified fork of the excellent jamesyu/cohorts lib. Meanwhile we employ heavy use of the Snowplow Analytics event pipeline for our Storage component and RStudio/Knitr for our Analytics reports.

Features

  • Under 5kb minified & gzipped
  • Create experiments with simple JSON or YAML
  • Self-hosted & git-controlled for familiar code review / merging
  • Expressive trigger system & utilities
  • Variant code JS/CSS minification
  • Track and handle JS errors caused by your variant code
  • Own your own experiments data

Mojito vs. [vendor]

Differentiating features between popular vendors' tools and Mojito out of the box:

Feature Optimizely X Google Optimize Mojito
Open-source license BSD3
Light front-end codebase * ~80kb ❗~25kb <5kb
Git source control & CI
Variant error-tracking/handling
Auto CSS/JS minification ❗(not custom code)
Self-hosted ❗ (for a fee) ❗(via API)
Data ownership ❗(yours via S3 export) ❗(yours via BigQuery)
Retroactively add new metrics ❗(360 only)
Server-side/App testing ❗(via API) ❗(via Storage)
WYSIWYG test editor

* Tested 2019-07-05

Getting started

Mojito consists of three components, which are often switched out in the course of Mint Metrics' client services:

  1. Delivery: Front-end libraries to reliably control which treatments users are exposed to. e.g. Mojito JS Delivery
  2. Storage: Data collection modules and data modelling steps to power your reports. e.g. Mojito Snowplow Storage
  3. Analytics: Tools to measure & report on the effects caused by your treatments. e.g. Mojito R Analytics

Get up and running quickly with the README files inside each section.

Mojito's 3 components

Example experiment

Using Mojito's CI tools, you can set up experiments in YAML & JS:

id: ex1
name: Example test 1
state: live
sampleRate: 0.75
trigger: trigger.js
recipes:
  0:
    name: Original
  1:
    name: Variant
    js: variant.js
    css: variant.css

Where trigger.js activates the experiment when a condition is met and a callback to activate is fired:

function trigger(test) {
    if (document.location.pathname === '/') test.activate();
}

Upon activation, the will include 75% of traffic (sampleRate: 0.75) and split it 50-50 between "Original" and "Variant" groups.

For users assigned to the "Variant" group, we execute a) variant.js and b) variant.css files to transform the page through a a) JS function and b) CSS stylesheet respectively.

After you've defined an experiment YAML...

Run the Gulp pipeline to lint/test/publish your container.

  1. Install the necessary NPM packages: npm install
  2. Build & publish your testing container: gulp scripts-local && gulp publish

Example analytics reports

If you use our Snowplow/Redshift & R Analytics component for reporting, all your metrics can be reported on with a simple array of metrics.

wave_params <- list(
  client_id = "mintmetrics",
  wave_id = "ex1",
  start_date = "2019-05-15 09:19:45",
  stop_date = "2019-06-05 14:29:00",
  time_grain = "hours",
  subject = "usercookie",
  recipes = c("Original", "Variant")
)

goalList <- list(
  list(
    title = "Transactions",
    goal = "purchase",
    operand = "="
  ),
  list(
    title = "Thankyou page views",
    goal = "page_view /contact/thank-you%",
    operand = "like"
  )
)
goalList <- mojitoFullKnit(wave_params, goal_list = goalList)

For this experiment, we'll report on transactions and page views:

Measuring the performance of a treatment relative to the control group in Mojito.

Support for other analytics back-ends

You don't exactly need Snowplow Analytics to use Mojito. You can also track experiments to wherever you like, via a custom storage adapter. E.g. To Google Tag Manager, Adobe etc.

You can even hook Mojito Delivery up to Google Optimize's reports for free.

Getting involved

We would love to see PRs! We're able to assist if you hit any snags getting set up.

Reach out to us via:

You can’t perform that action at this time.