MultiArmedBandit
title: Multi-Armed Bandit Testing type: technique created: 2026-05-16 last_updated: 2026-05-16 related: ["Playradar"] sources: ["https://stevehanov.ca/blog/20-lines-of-code-that-will-beat-ab-testing-every-time"] radar_quadrant: Techniques radar_ring: Assess radar_position: center
An experimentation technique that replaces traditional A/B testing by continuously optimising toward the best-performing variant rather than holding fixed traffic splits for the duration of a test. Named after the slot machine analogy: a gambler with multiple "arms" must decide how often to exploit the known best arm versus explore others.
Traditional A/B testing divides traffic equally between variants for a predetermined period, then picks a winner. This means half the audience is exposed to the inferior variant for the entire test duration — a cost that grows with test length and traffic volume. A/B testing also requires statistical significance calculations upfront and provides no adaptive behaviour during the test.
The most common bandit algorithm for web experimentation. Operation:
- 90% of the time (exploitation): serve the variant with the highest observed reward rate.
- 10% of the time (exploration): select a variant at random, including potentially inferior ones, to avoid premature convergence.
The split (epsilon = 0.10) is a tunable parameter. The algorithm tracks wins and total attempts per variant and recalculates the best arm after every interaction.
import random
def choose(counts, values):
epsilon = 0.1
if random.random() < epsilon:
return random.randrange(len(values))
return values.index(max(values))
def update(counts, values, choice, reward):
counts[choice] += 1
n = counts[choice]
values[choice] = ((n - 1) / n) * values[choice] + (1 / n) * reward| Property | A/B Testing | Multi-Armed Bandit |
|---|---|---|
| Traffic to inferior variant | 50% throughout | Decreases over time |
| Handles 3+ variants | Requires separate tests | Native |
| Dynamic variant addition | Not supported | Supported |
| Ongoing management | Requires monitoring and cutover | Self-optimising |
The technique is most valuable when:
- A measurable reward signal exists per interaction (click, conversion, sign-up).
- Test duration is long enough that early optimisation reclaims meaningful traffic.
- The reward distribution is stationary (no time-of-day or seasonal confounding).
It is less appropriate when formal statistical significance guarantees are required (regulatory contexts, clinical settings) or when the reward signal is delayed beyond the request cycle.
Multi-Armed Bandit Testing sits in the Assess ring of the Techniques quadrant, at center position. First studied via Steve Hanov's article. Personal use confirmed in a side project context as of 2026-05-16. Technique is well-established in data science and growth engineering (Thoughtworks has covered bandit algorithms). Center position reflects that hands-on experience exists in a limited scope — applicability is contingent on having a product with measurable conversion events, and no production deployment has occurred, which holds the blip from moving inner.
