The primary objective of this A/B testing was to determine the most effective marketing strategy between two campaigns: the Control Campaign and the Test Campaign. The opportunity identified was to improve marketing ROI, increase conversion rates, and optimise the overall cost-efficiency of marketing efforts. The decision to conduct this test stemmed from a need to identify which campaign variables (e.g., ad creatives, targeting criteria, spending allocation) contribute most significantly to campaign success.
The hypothesis was that the Test Campaign, with adjusted targeting criteria and ad creative strategies, would outperform the Control Campaign in terms of Conversion Rate, CTR, CPA, and ROAS.
- Primary Metrics: Conversion Rate, ROAS
- Guardrail Metrics: CTR, CPA
- Control Campaign: Utilised original ad creative and targeting criteria.
- Test Campaign: Implemented adjusted ad creative and refined targeting criteria based on preliminary data insights.
- Audience Targeting: Both campaigns were deployed across similar audience segments to ensure a fair comparison.
The test concluded with significant findings, providing clear insights into the effectiveness of both campaigns.
The Control Campaign emerged as the more cost-effective strategy, disproving the initial hypothesis.
The hypothesis was disproven; the Test Campaign did not outperform the Control Campaign in terms of the primary metrics.
- Conversion Rate: No significant difference was found between the campaigns.
- CTR: The Test Campaign achieved a higher CTR, suggesting better engagement but not necessarily higher conversion efficiency.
- CPA: The Control Campaign demonstrated higher cost-efficiency with a lower CPA.
- ROAS: The Control Campaign had a significantly higher ROAS, affirming its superior cost-effectiveness.
The results suggest that the Test Campaign's adjustments did not resonate as expected, despite higher initial engagement. The Control Campaign's strategies proved to be more effective in driving revenue relative to spend.
The discrepancy in performance could be attributed to various factors, including the effectiveness of ad creatives and targeting criteria.
Click here to
It's recommended to further analyze aspects of the Test Campaign's strategy to understand the lack of conversion efficiency. A hybrid strategy that combines elements from both campaigns could be explored.
This report provides a narrative from hypothesis through testing, analysis, and strategic recommendations, offering actionable insights based on quantitative data.