05. Modelling

Subgroup A: Customer Behavior and Sales Analysis

1. What are the key factors influencing customer purchasing behavior?

Develop customer segmentation models based on purchasing behavior.

Modeling techniques considered

1. Machine learning models
   • Dimension reduction, comparing performance between
     • Principal Component Analysis (PCA) (Scikit-learn developers, n.d.-b) and
     • T-distributed Stochastic Neighbor Embedding (t-SNE) (Scikit-learn developers, n.d.-a) followed by
   • Clustering algorithms, comparing performance between
     • Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) (Scikit-learn developers, n.d.-d) and
     • K-means Clustering (Scikit-learn developers, n.d.-c)
2. Human knowledge-based model
   • Behavioral Segmentation of customers based on their actions and spending habits (Tarver, 2024)

Model Selection Criteria

In selecting a model for customer segmentation based on purchasing behavior, interpretability was a key consideration. The goal was to ensure that the customer segments were not only data-driven but also intuitive and actionable within a business context.

Despite the advanced clustering achieved with machine learning models, the optimal clustering parameter 𝑘 = 64 suggested a large number of segments. Such a high number of clusters posed challenges for business interpretability, as it would be difficult to discern and apply insights from so many distinct customer groups effectively.

Thus, I opted for the Human Knowledge-Based Model. This model allowed for more meaningful segmentation that aligned with real-world business practices, making the segments easier to interpret and more directly applicable to strategic decision-making.

Detailed description of the chosen model:

To segment customers based on their actions and spending habits, I calculated various features from our database, summarizing purchasing behavior at an individual level. These features provided insights into each customer's frequency, spending patterns, and product preferences, forming the basis for intuitive, human knowledge-based customer segments.

Calculated Features for Each Customer

The following columns were calculated for each customer to capture key aspects of their behavior:

1. ordered: Indicates order history:

   • 0: No orders made.
   • 1: Orders made on only one day.
   • 2: Orders made on multiple days.

2. total_order: Total number of orders a customer has ever made.

3. ave_monthly_orders: Average number of orders per month, calculated between the earliest and latest orders for customers with multiple order days. Set to 0 for customers with fewer than two order days.

4. days_last_order: Days since the customer’s last order to the final date in the dataset. Marked as NA if no orders were made.

5. only_promo_order: Binary indicator:

   • 1: Customer only ordered during promotional periods.
   • 0: Otherwise.

6. total spend: Total amount a customer has spent across all orders.

7. ave_monthly_spending: Average monthly spending between the earliest and latest order dates for customers with multiple order days, otherwise 0.

8. most_ordered_cat: The product category that the customer ordered the most.

9. moc_ratio = (number of orders a customer made for their most ordered category) / (total orders the customer made)

Customer Segment Labels

Based on the calculated features, I labeled each customer into one of six segments. These segments were derived using percentile-based thresholds (with the top 25% defined as "high" and bottom 25% as "low") established during exploratory data analysis (EDA). The feature distributions used in this labelling are documented in the Appendices.

1. Discount Seekers:

   • Identified by having only_promo_order = 1 or high values in CouponUsed (≥ 2) or Cashback Amount (≥ 205.45).

2. Loyal High-Spenders:

   • Characterized by high total spend (≥ 62.4), high ave_monthly_orders (≥ 0.32), and long tenure (≥ 14 months).

3. Occasional Shoppers:

   • Defined by low ave_monthly_orders (≤ 0.13) and low HourSpendOnApp (≤ 2).

4. Tech-Savvy Users:

   • Spend a high number of hours on the app (HourSpendOnApp ≥ 4) and have multiple registered devices (NumberOfDeviceRegistered ≥ 4).

5. Single-Category Shoppers:

   • Show a strong preference for a single product category, indicated by a high moc_ratio (≥ 1).

6. Long-Tenured Non-Buyers:

   • Customers who have not made any orders (ordered = 0) and have a long tenure (≥ 14 months).

Following the segmentation, I calculated the number of customers in each of the six segments to understand the distribution across these identified customer types.

Model Performance Metrics and Interpretation

Since this is a human knowledge-based model, traditional performance metrics (such as accuracy or clustering scores) do not apply. Instead, the model’s effectiveness is evaluated based on its interpretability and utility for strategic business insights. The segmentation was designed to align closely with real-world customer behaviors, allowing for intuitive and actionable insights.

Key interpretative metrics for this model includes:

1. Segment Size: By examining the number of customers within each segment, stakeholders gain a clear understanding of customer distribution, enabling them to focus efforts on larger segments.
2. Strategic Value: The different behaviours of each segment allows the business to identify specific customer needs and tailor engagement efforts accordingly.

2. Improving Customer Retention and Lifetime Value

Modeling techniques considered:

• Logistic Regression
• Decision Tree
• Random Forest

Model Selection criteria

Since objective is to predict and identify customers likely to churn (binary classification) and churn values were imbalanced, best model was selected based on its F1 score, ensuring that both false positives and false negatives are minimized.

Detailed description of the chosen model

Random Forest model had the highest F1 score among the 3 models. RandomizedSearchCV was then ran to find the best hyperparameters, which are:

• n_estimators : 300
• min_samples_split : 2
• min_samples_leaf : 1
• max_depth : 10
• bootstrap : True

Model Performance metrics and interpretation:

• Test Accuracy: 86.83%. While accuracy provides an overall measure of the model's performance, it may not fully represent its effectiveness in predicting the minority class (churned customers) due to the class imbalance in our dataset.

• Classification Report:

  • Precision for class 1 (churn) is 0.73, indicating that 73% of the customers the model predicted as churned were indeed correct.
  • Recall for class 1 is 0.63, meaning the model successfully identified 63% of actual churned customers.
  • F1-Score for class 1 is 0.68, balancing both precision and recall, making it a useful metric given the importance of accurately identifying churned customers.

Subgroup B: Inventory Management and Pricing Optimization

1. How can we optimize inventory levels to minimize costs while ensuring product availability?

2. What pricing strategies can we implement to maximize revenue?

In order to maximize revenue, we developed a dynamic pricing model which takes into account

Modeling techniques considered

Model Selection Criteria

Model Performance Metrics and Interpretation

3. How can we improve the efficiency of our supply chain?

Modeling techniques considered

• Random Forest Classifier
• Logistic Regression
• Gradient Boosting

Model Selection Criteria

To enhance supply chain efficiency by identifying bottlenecks and predicting delays, we required a model that provided robust interpretability and worked well with imbalanced data. Interpretability was especially important, as the insights needed to be actionable and understandable to stakeholders.

Given the class imbalance between delayed and on-time orders, we evaluated models primarily on their F1 score, aiming to balance precision and recall. This focus ensured that the model would effectively identify delays without excessive false positives, which could lead to unnecessary interventions.

We ultimately selected Logistic Regression due to its simplicity, interpretability, and reliable performance with binary classification. The model’s coefficients allowed us to clearly identify influential factors, such as scheduled shipping days and delivery risk, providing a straightforward foundation for targeted interventions in the order fulfillment process. This choice aligns with business requirements by offering clear insights into specific factors driving delays.

Detailed description of the chosen model:

Logistic Regression is a statistical model commonly used for binary classification tasks, where the objective is to predict one of two possible outcomes. In our supply chain project, we used logistic regression to predict the likelihood of delays in order fulfillment, a binary classification problem (i.e., delayed or on-time).

Logistic regression models the probability of a binary outcome by fitting a logistic function (also known as a sigmoid curve) to the data. This function transforms input features into a probability value between 0 and 1, where values closer to 1 indicate a higher likelihood of delay, while those near 0 indicate on-time fulfillment.

Key Features of Logistic Regression:

1. Interpretability: Logistic regression provides clear insights into how each feature contributes to the likelihood of an outcome. The model’s coefficients indicate the strength and direction (positive or negative) of each feature’s impact on the delay probability.

2. Probability Outputs: Unlike many other models, logistic regression outputs probabilities rather than simple class predictions. This probability-based output is valuable for decision-making as it allows us to prioritize high-probability delays for intervention.

3. Handling of Linearity: Although logistic regression assumes a linear relationship between the independent variables and the log-odds of the outcome, this assumption was acceptable for our dataset, given that the features selected were well-suited to the model’s capabilities.

4. Efficiency and Scalability: Logistic regression is computationally efficient, making it suitable for handling large datasets. This efficiency was particularly useful given the volume of records in our supply chain dataset.

Model Interpretation

The model’s coefficients provide direct insight into which factors are most predictive of delays. For example:

• A positive coefficient for Scheduled Days for Shipping implies that orders with a higher number of scheduled days are more likely to be delayed.
• A high positive coefficient for Late Delivery Risk indicates that as the perceived risk of late delivery increases, so does the probability of an actual delay.

These insights guide strategic interventions, such as refining scheduling practices or adjusting supplier coordination to minimize delays. Logistic regression’s straightforward output and interpretability make it an ideal choice for driving improvements in supply chain efficiency while keeping the model easily explainable for business applications.