05. Modelling

Subgroup A: Customer Behavior and Sales Analysis

1. What are the key factors influencing customer purchasing behavior?

Develop customer segmentation models based on purchasing behavior.

Modeling techniques considered

1. Machine learning models
   • Dimension reduction, comparing performance between
     • Principal Component Analysis (PCA) (Scikit-learn developers, n.d.-b) and
     • T-distributed Stochastic Neighbor Embedding (t-SNE) (Scikit-learn developers, n.d.-a) followed by
   • Clustering algorithms, comparing performance between
     • Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) (Scikit-learn developers, n.d.-d) and
     • K-means Clustering (Scikit-learn developers, n.d.-c)
2. Human knowledge-based model
   • Behavioral Segmentation of customers based on their actions and spending habits (Tarver, 2024)

Model Selection Criteria

In selecting a model for customer segmentation based on purchasing behavior, interpretability was a key consideration. The goal was to ensure that the customer segments were not only data-driven but also intuitive and actionable within a business context.

Despite the advanced clustering achieved with machine learning models, the optimal clustering parameter 𝑘 = 64 suggested a large number of segments. Such a high number of clusters posed challenges for business interpretability, as it would be difficult to discern and apply insights from so many distinct customer groups effectively.

Thus, I opted for the Human Knowledge-Based Model. This model allowed for more meaningful segmentation that aligned with real-world business practices, making the segments easier to interpret and more directly applicable to strategic decision-making.

Detailed description of the chosen model:

To segment customers based on their actions and spending habits, I calculated various features from our database, summarizing purchasing behavior at an individual level. These features provided insights into each customer's frequency, spending patterns, and product preferences, forming the basis for intuitive, human knowledge-based customer segments.

Calculated Features for Each Customer

The following columns were calculated for each customer to capture key aspects of their behavior:

1. ordered: Indicates order history:

   • 0: No orders made.
   • 1: Orders made on only one day.
   • 2: Orders made on multiple days.

2. total_order: Total number of orders a customer has ever made.

3. ave_monthly_orders: Average number of orders per month, calculated between the earliest and latest orders for customers with multiple order days. Set to 0 for customers with fewer than two order days.

4. days_last_order: Days since the customer’s last order to the final date in the dataset. Marked as NA if no orders were made.

5. only_promo_order: Binary indicator:

   • 1: Customer only ordered during promotional periods.
   • 0: Otherwise.

6. total spend: Total amount a customer has spent across all orders.

7. ave_monthly_spending: Average monthly spending between the earliest and latest order dates for customers with multiple order days, otherwise 0.

8. most_ordered_cat: The product category that the customer ordered the most.

9. moc_ratio = (number of orders a customer made for their most ordered category) / (total orders the customer made)

Customer Segment Labels

Based on the calculated features, I labeled each customer into one of six segments. These segments were derived using percentile-based thresholds (with the top 25% defined as "high" and bottom 25% as "low") established during exploratory data analysis (EDA). The feature distributions used in this labelling are documented in the Appendices.

1. Discount Seekers:

   • Identified by having only_promo_order = 1 or high values in CouponUsed (≥ 2) or Cashback Amount (≥ 205.45).

2. Loyal High-Spenders:

   • Characterized by high total spend (≥ 62.4), high ave_monthly_orders (≥ 0.32), and long tenure (≥ 14 months).

3. Occasional Shoppers:

   • Defined by low ave_monthly_orders (≤ 0.13) and low HourSpendOnApp (≤ 2).

4. Tech-Savvy Users:

   • Spend a high number of hours on the app (HourSpendOnApp ≥ 4) and have multiple registered devices (NumberOfDeviceRegistered ≥ 4).

5. Single-Category Shoppers:

   • Show a strong preference for a single product category, indicated by a high moc_ratio (≥ 1).

6. Long-Tenured Non-Buyers:

   • Customers who have not made any orders (ordered = 0) and have a long tenure (≥ 14 months).

Following the segmentation, I calculated the number of customers in each of the six segments to understand the distribution across these identified customer types.

Model Performance Metrics and Interpretation

Since this is a human knowledge-based model, traditional performance metrics (such as accuracy or clustering scores) do not apply. Instead, the model’s effectiveness is evaluated based on its interpretability and utility for strategic business insights. The segmentation was designed to align closely with real-world customer behaviors, allowing for intuitive and actionable insights.

Key interpretative metrics for this model includes:

1. Segment Size: By examining the number of customers within each segment, stakeholders gain a clear understanding of customer distribution, enabling them to focus efforts on larger segments.
2. Strategic Value: The different behaviours of each segment allows the business to identify specific customer needs and tailor engagement efforts accordingly.

2. Improving Customer Retention and Lifetime Value

Modeling techniques considered:

• Logistic Regression
• Decision Tree
• Random Forest

Model Selection criteria

Since objective is to predict and identify customers likely to churn (binary classification) and churn values were imbalanced, best model was selected based on its F1 score, ensuring that both false positives and false negatives are minimized.

Detailed description of the chosen model

Random Forest model had the highest F1 score among the 3 models. RandomizedSearchCV was then ran to find the best hyperparameters, which are:

• n_estimators : 300
• min_samples_split : 2
• min_samples_leaf : 1
• max_depth : 10
• bootstrap : True

Model Performance metrics and interpretation:

• Test Accuracy: 86.83%. While accuracy provides an overall measure of the model's performance, it may not fully represent its effectiveness in predicting the minority class (churned customers) due to the class imbalance in our dataset.

• Classification Report:

  • Precision for class 1 (churn) is 0.73, indicating that 73% of the customers the model predicted as churned were indeed correct.
  • Recall for class 1 is 0.63, meaning the model successfully identified 63% of actual churned customers.
  • F1-Score for class 1 is 0.68, balancing both precision and recall, making it a useful metric given the importance of accurately identifying churned customers.

Inventory Management and Pricing Optimization