Algorithms

There are a few common assignment algorithms included in TrailGuide, and it's easy to define your own and configure your experiments to use them. Algorithms can either be configured globally for all experiments in your initializer, or overridden individually per-experiment.

The following algorithms are available:

• :distributed (default) - totally even distribution across variants
• :weighted - allows favoring variants by assigning them weights
• :random - truly random sampling of variants on assignment
• :bandit - a "multi-armed bandit" approach to assignment

Distributed

This is the default algorithm, which ensures completely even distribution across all variants by always selecting from the variant(s) with the lowest number of participants.

experiment :my_experiment do |config|
  config.algorithm = :distributed
end

Weighted

The weighted algorithm allows weighted assignment to variants based on each variant's configuration. All things being equal (all variants having equal weights), it's essentially a random sampling that will provide mostly even distribution across a large enough sample size. The default weight for all variants is 1.

experiment :my_experiment do |config|
  config.algorithm = :weighted

  variant :a, weight: 2 # would be assigned roughly 40% of the time
  variant :b, weight: 2 # would be assigned roughly 40% of the time
  variant :c, weight: 1 # would be assigned roughly 20% of the time
end

Note that the weighted algorithm is the only one that takes variant weight into account, and the other algorithms will simply ignore it if it's defined.

Random

The random algorithm provides totally random distribution by sampling from all variants on assignment.

experiment :my_experiment do |config|
  config.algorithm = :random
end

Multi-Armed Bandit

The bandit algorithm in TrailGuide was heavily inspired by the split gem, and will automatically weight variants based on their performance over time. You can read more about this approach if you're interested.

experiment :my_experiment do |config|
  config.algorithm = :bandit
end

Static

The static algorithm is intended to be used for content-based experiments alongside the sticky_assignment = false experiment configuration. The algorithm will select a variant based on a match between configured metadata and the contextual metadata provided when choosing a variant. You must configure the static algorithm with a block, which will be provided with both sets of metadata, that provides the matching logic.

For example, to render a specific variant based on geographic location (in this case state):

experiment :my_experiment do |config|
  config.sticky_assignment = false # content-based experiments should not store/assign variants to participants
  config.algorithm = :static, -> (varmeta,ctxmeta) { varmeta[:states].include?(ctxmeta[:state]) }

  variant :alpha
  variant :bravo, metadata: {states: ['CA', 'OR', 'WA']}
  variant :charlie, metadata: {states: ['NV', 'NM', 'UT']}
end

When you want to render the experiment and choose a variant, just pass in the relevant metadata, and if the state matches a variant (based on your block) it will be returned:

case trailguide(:my_experiment, metadata: {state: @thing.state})
  when :alpha
    # ...
  when :bravo
    # ...
end

Custom

TODO - In the meantime, take a look at the included algorithms as a starting point. Essentially as long as you accept an experiment and return a variant, the rest is up to you.

experiment :my_experiment do |config|
  config.algorithm = MyCustom::AlgorithmClass
end