This project was developed for the Vammo challenge, focusing on improving operational efficiency and customer satisfaction through the analysis of service and churn data.
Based on ticket and client data, I sought to:
- Understand patterns and bottlenecks in customer service
- Identify factors associated with dissatisfaction and churn
- Propose solutions for improvement and automation
The project is divided into two main notebooks:
| Notebook | Description |
|---|---|
vammo_case_eda.ipynb |
Data exploration and initial processing. Includes descriptive analysis of tickets and clients, verification of null values, ticket types, service hours, etc. |
vammo_case_metrics.ipynb |
Development of efficiency and satisfaction metrics and churn analysis. Includes a proposal for a simple churn risk model. |
vammo_case_hypothesis_testing.ipynb |
Hypothesis testing to assess the difference between satisfaction rating averages. |
- Service Tickets (December 2024 and January 2025): information about requests, attendants, schedules, and notes.
- Churn customers (February 2025): identification of customers who have terminated their contract, with variables such as contract type and estimated weekly kilometers.
-
Volume of tickets concentrated during business hours (9 a.m. – 7 p.m.) → indicates the need for the team to concentrate during these hours and/or automate simple tasks during these hours.
-
A significant portion of tickets were not initially assigned to an agent (25%), suggesting bottlenecks in automatic routing.
-
Average first response time is 47 minutes — among customers who churned, it was 41 minutes.
-
The satisfaction rate is directly linked to the attendant. On average, it is 1.6 per attendant, which is very low if we consider a range of 0 to 5 — some even reached an average of zero.
-
The most frequent tickets are related to motorcycle support and marketing communications. The latter occurred due to the sending of Retrospective 2024 (98% of tickets).
- Among customers who churned, 21% traveled up to 50 km per week, which is three times the volume of customers who traveled between 50 and 100 km.
- Among these same customers, the type of contract was distributed almost equally.
| Contract Type | Percent |
|---|---|
| Annual | 54.3% |
| Monthly | 46.7% |
- Note: information on contract type and weekly kilometers traveled was obtained from customers who churned, making it impossible to compare with customers who maintained an active contract.
- I ran a hypothesis test to verify the relevance of satisfaction_rating. Although clients churned, they gave the highest satisfaction ratings on average, which seems contradictory.
| Group | Size | Avg satisfaction_rating |
|---|---|---|
| Churn | 1182 | 1.7893 |
| No churn | 9054 | 1.6160 |
- The permutation test resulted in a p-value = 0.0142, i.e., which means that the hypothesis that the means between the two groups are equal is rejected.
- I calculated the Confidence Interval (95%), the average difference in
satisfaction_ratingbetween clients who churned and those who did not churn is between +0.03 and +0.31 points. The difference exists, but it is too small to be considered decisive as a predictor of churn on its own.
-
A simple churn risk score was created by combining variables such as the number of messages exchanged between the customer and agent, the reason for contact, and the volume of tickets.
$$ \text{churn risk score} = \mathbf{1}_{\text{message count} \ge 25} + \mathbf{1}_{\text{outbound message count} \ge 10} + \mathbf{1}_{\text{contact reason = 'duvida plano geral'}} + \mathbf{1}_{\text{ticket count} > 4} $$ -
Clients with a churn risk score greater than or equal to 1.6 deserve proactive retention actions.
-
Note: the churn risk equation can be improved with more variables that allow comparison between churn and non-churn groups.
| Metric | Description | Value | Note |
|---|---|---|---|
| Average SLA | Average time between ticket opening and assignment to an agent. | 47 min | The shorter the average response time, the greater the client satisfaction. |
| Unresponded ticket rate | Ticket marked with satisfaction status 'unresponded'. | 50% | Indicates the percentage of tickets without a response (important for efficiency and client experience). |
| Undirected ticket rate | Ticket without timestamp directed to the attendant. | 25% | Indicates the percentage of tickets that the system opened but did not reach an agent. |
| Average ticket complexity | Quantity of messages from the client → attendant (the bigger, the more complex). | 24 | Prioritization of automations: topics with high incidence and low complexity → ideal candidates for chatbot/FAQ. |
| Average satisfaction | On a scale of 0 to 5. | 1.6 | The average overall satisfaction rating is quite low. |
| Churn Risk Score | Indicator composed of other metrics. | >= 1.6 | Metrics were used that made it possible to differentiate between churn and non-churn customers. |
- Implement smart ticket queuing, prioritizing customers at high risk of churn — even though customers who churned were attended to 7 minutes faster on average.
- Create automatic alerts when a ticket exceeds the average first response time.
- Due to the low satisfaction rate, I would recommend improvements in the training of attendants.
- Evaluate the best way to send communications so as not to generate a high volume of tickets, as occurred with the 2024 Retrospective.
- Chatbot with NLP for screening low-complexity requests.
- Use of automatic ticket classification to speed up routing and reduce backlog — despite the lack of teams specializing in a single topic.
- Retention program for monthly contracts, offering loyalty benefits—despite the proportion by contract type being similar between monthly and annual contracts among customers who churned.
- Dashboard for continuous monitoring of efficiency and satisfaction KPIs.
