iter8: Analytics-driven canary releases and A/B testing
What is iter8 about?
Iter8 supports cloud-native, automated canary releases and A/B testing, driven by analytics based on robust statistical techniques. It comprises two components:
iter8-analytics: A service that assesses the behavior of different microservice versions by analyzing metrics associated with each version using robust statistical techniques to determine which version is the best one with respect to the metrics of interest and which versions pass a set of success criteria. Multiple success criteria can be defined by users; each criterion refers to a metric and specifies absolute or relative thresholds which define how much a candidate version can deviate from a baseline (stable) version. The iter8-analytics service exposes a REST API; each time it is called, the service returns the result of the data analysis along with a recommendation for how the traffic should be split across all microservice versions. The iter8-analytics' REST API is used by iter8-controller, which is described next.
iter8-controller: A Kubernetes controller that automates canary releases and A/B testing by adjusting the traffic across different versions of a microservice as recommended by iter8-analytics. For instance, what happens in the case of a canary release is that the controller will shift the traffic towards the canary version if it is performing as expected, until the canary replaces the baseline (previous) version. If the canary is found not to be satisfactory, the controller rolls back by shifting all the traffic to the baseline version. Traffic decisions are made by iter8-analytics and honored by iter8-controller.
When iter8 is installed, a new Kubernetes CRD is added to your cluster. This CRD kind is
Experiment and it is documented here.
To assess the behavior of microservice versions, iter8 supports a few metrics out-of-the-box without requiring users to do any extra work. In addition, users can define their own custom metrics. Iter8's out-of-the-box metrics as well as user-defined metrics can be referenced in the success criteria of an experiment. More details about metrics are documented here.
The iter8-controller currently uses the Istio service mesh traffic management capabilities to automate the user traffic split across multiple microservice versions.
These instructions will guide you to install the two iter8 components (iter8-analytics and iter8-controller) on Kubernetes with Istio.
The following tutorials will help you get started with iter8:
- Automated canary releases with iter8 when specifying versions by Kubernetes deployment names
- Automated canary releases with iter8 when specifying versions by Kubernetes service names
Algorithms behind iter8
A key goal of this project is to introduce statistically robust algorithms for decision making during cloud-native canary releases and A/B testing experiments. We currently support four algorithms.
Iter8 can be integrated with Tekton Pipelines for an end-to-end CI/CD experience, and with KUI, for a richer Kubernetes command-line experience. Initial integrations with these two technologies already exist, but we are actively improving them.
The upcoming iter8 1.0.0 release will feature a deep integration with KUI. Stay tuned!
The current iter8 release is v0.2.1. This documentation always points to the current release. Documentation of older releases can be found here.
Release 1.0.0 is coming soon! Stay tuned!!