11. Lessons Learned

Challenges faced and how they were overcome

Subgroup A

Customer Segmentation model

One of the primary challenges in developing our customer segmentation model was the limited coverage of segmentation techniques in our lectures. While our course included a general overview of machine learning, the emphasis was largely on machine-based models rather than specifically on segmentation. The only machine learning model that seemed to be suitable for segmentation was unsupervised learning that was in the summary slide, with minimal guidance on how to apply these models for customer segmentation. This gap meant that I needed to independently research and study various unsupervised learning techniques to understand and implement an effective segmentation approach.

After testing several times with different unsupervised learning methods, I identified an optimal number of clusters as k=64, based on standard unsupervised clustering evaluation metrics. However, this result presented a second major challenge: from a business perspective, such a high number of clusters was impractical and extremely challenging to interpret. This outcome highlighted the need for alternative segmentation approaches that align better with business goals, which had not been covered in our lectures.

To overcome these challenges, I sought advice from our teaching assistant (TA), Hera, in a consultation session. Hera introduced me to a human knowledge-based approach for customer segmentation, where clusters could be designed around human-defined behavioral attributes. This approach provided a more practical, interpretable solution for our project and was highly effective in creating meaningful customer segments based on business insights.

Analysing impacts of marketing channels and promotional campaigns

One of the primary challenges we encountered during the project was the inadequacy of the dataset we found to thoroughly evaluate the impacts on both marketing channels and campaigns. This limitation forced us to utilize two separate datasets - one for campaigns and another for messages, rather than relying on a single comprehensive dataset.

The primary challenge was that the datasets did not provide a holistic view of the marketing efforts. The gaps in data left us unable to analyze key performance indicators effectively, hindering our assessment of impacts of marketing channels and campaign effectiveness. Furthermore, the messages dataset was considerably large, which caused our code to execute slowly and made it difficult to visualize the data efficiently. This performance issue significantly impacted our ability to conduct timely analysis and draw actionable insights.

To address these challenges, we implemented a solution that involved sampling a subset of campaigns from both the campaigns and messages tables. This approach allowed us to reduce the size of both datasets, facilitating a more manageable analysis without compromising the integrity of our findings.

However, after performing random sampling, we realized that the resulting tables were still insufficient for our analysis. We lacked essential product details, such as products, product costs and prices, which were critical for understanding the overall effectiveness of our marketing strategies.

To overcome this shortfall, we resorted to generating synthetic data to complement our analysis. We conducted thorough research in generating synthetic data, ensuring that the simulated product details accurately reflected real market patterns. We made sure to reference credible sources during this process to reduce potential biases and enhance the validity of the synthetic data.

By navigating these challenges with strategic solutions, we were able to refine our analysis, ultimately leading to more insightful conclusions regarding the impacts of marketing channels and campaigns. The use of synthetic data not only addressed the missing information but also helped create a more robust framework for evaluating marketing effectiveness.

Insights gained during the project

Reflections on the team's approach and process

In this project, sourcing datasets that could effectively address our research questions was a priority and required significant effort. One of the biggest challenges we encountered was merging datasets from different sources, which involved navigating inconsistencies in formats and missing connections between data tables. To address this, we applied database concepts, such as using primary and foreign keys, to establish meaningful links between tables.

However, due to limitations in both time and resources, we had to rely on some random data generation to connect certain tables. Although this approach is not ideal, it was the most practical solution given the time and resource constraints we faced. This experience highlighted the importance of having the creative problem-solving skills required to work with fragmented data.