Two‐Armed Bandit

Two-Armed Bandit

Overview

The two-armed bandit paradigm was first formalized by Robbins (1952) as a mathematical framework for sequential decision-making under uncertainty. The problem was framed through the metaphor of a gambler faced with multiple slot machines ("one-armed bandits"), each with an unknown probability of payout, who must balance exploration of lesser-known options against exploitation of the most rewarding one. This framework was soon adapted into psychology experiments to study reinforcement-driven choice, and later into animal operant tasks where subjects select between two responses, each associated with probabilistic reward delivery.

In neuroscience, the task has become a central tool for studying reinforcement learning (Sutton & Barto, 1998) and the neural computations supporting flexible decision-making (Daw et al., 2006). In a FED-based closed economy design, the two-armed bandit can be continuously available in the home cage, allowing animals to earn all food through probabilistic choices and enabling long-timescale, naturalistic measurement of reinforcement learning with fixed reward probabilities. It is designed to assess decision-making under probabilistic reinforcement and measure learning.

Theory

Performance on the two-armed bandit reflects integration of reinforcement history, probabilistic outcome tracking, and the ability to flexibly adapt when contingencies shift. Central to this process is dopaminergic prediction error signaling: unexpected rewards generate positive errors that reinforce chosen actions, while unexpected omissions generate negative errors that weaken choice tendencies. Striatal circuits encode value estimates of each option, while prefrontal regions support exploration–exploitation balance and adaptive switching. Pharmacological manipulations of dopamine transmission bias these dynamics—dopamine-enhancing drugs favor exploitation of high-value options, while antagonists or lesions increase perseveration or random choice. Thus, the bandit task provides a sensitive assay of reinforcement learning and cognitive flexibility.

Considerations

• Reward probabilities: Probabilities should be chosen to ensure sufficient sampling of both options while maintaining discriminability (e.g., 0.7 vs 0.3).
• Exploration vs exploitation: Animals must balance trying both options (exploration) against favoring the better option (exploitation).
• Session length: Longer sessions allow observation of learning dynamics and choice stabilization.
• Contingency reversals: Switching which option is more rewarding can assess behavioral flexibility and reversal learning.
• Statistical power: Probabilistic reinforcement requires more trials than deterministic schedules to reveal preferences.

Key Features

• Two response options (e.g., left vs right nose poke), each with independent reward probability.
• Measures reinforcement learning, choice behavior, and decision-making under uncertainty.
• Enables assessment of exploration–exploitation trade-offs.
• Sensitive to dopaminergic manipulations and striatal/prefrontal circuit function.
• Can incorporate reversals to measure cognitive flexibility.
• Supports continuous measurement in closed economy home cage environments.