Test Optimization Simulation

This is a repo that is accompanying this blog post and a couple of conference talks on the same topic.

TLDR: This code is used to simulate and show the differences between A/B tests and optimizing test groups using multi-armed bandit methods.

Installation

Use the package manager pip to install required libraries. Or if you are using anaconda, it should work out of the box.

pip install requirements.txt

Usage

For the imports to work properly either augment the provided example in experiment.ipynb or alternatively create a new notebook/python script that is on the same level in the project directory. The examples provided are very simple and can be extended to simulate more complex tests. All functions have provided docstrings for easier use.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.