A\B Testing is meant to test a new version of something (new design, new features, etc) to the old version and how that affects the business to determine which performs better
It also can be more than 2 Groups a-b-c-d but the most common one is (a-b) because it's hard and not the most effective to test more than one group
To run an A/B test, you need to create two different versions of one piece of content, with changes to a single variable. Then, you'll show these two versions to two similarly sized audiences and analyze which one performed better over a specific period (long enough to make accurate conclusions about your results).
Explanation of what a/b testing is
- Understanding business problems & data
- Detect and resolve problems in the data (Missing Value, Outliers, Unexpected Value)
- Look at summary stats and plots
- Apply hypothesis testing and check assumptions
- Check Normality & Homogeneity
- Apply tests (Shapiro, Levene Test, T-Test, Welch Test, Mann Whitney U Test)
- Evaluate the results
- Make inferences
- Recommend business decisions to your customer/director/CEO etc.