Implement Emotion Classification Feature in Falcon Evaluate for Enhanced B2C Chat and Customer Interaction Analysis

Falcon evaluate now helps to compute user Analytics — emotions score for your customer interaction applications

Intro
Welcome to the world of Falcon Evaluate User Analytics — Emotion Module! This isn’t just any tool; it’s your new buddy in understanding what your customers are really feeling when they chat with your GenAI app. Think of it as having a superpower to see beyond words, getting to the heart of every 😊, 😠, or 😢 in your customer conversations.

Here’s the deal: We know that every chat your customer has with your AI is more than just words. It’s about feelings. That’s why we created the Emotion Module. It’s like having a smart friend who reads between the lines, telling you if your customers are happy, just okay, or maybe a bit upset. It’s all about making sure you really get what your customers are feeling, through emojis they use, like 👍 for ‘Great job!’ or 😞 for ‘Oh no!’.

We crafted this tool with one big goal: to make your chats with customers not just smarter, but more human and relatable. Imagine being able to know exactly how your customer feels and being able to respond just right. That’s what the Emotion Module is here for. It’s easy to use, integrates with your chat data like a charm, and gives you insights that are all about making your customer interactions better, one chat at a time.

So, get ready to transform your customer chats from just words on a screen to conversations filled with real, understood emotions. Falcon Evaluate’s Emotion Module is here to make every chat count!

Positive:

• 👍 approval
• 😊 joy
• 💕 caring
• 😍 desire
• 🥰 admiration
• 🌟 optimism
• ❤️ love
• 🤩 excitement
• 😂 amusement
• 🙏 gratitude
• 😇 pride

Neutral:

• 😐 neutral
• 😮 realization
• 🤔 curiosity
• 😯 surprise

Negative:

• 😞 disappointment
• 😢 sadness
• 😒 annoyance
• 🙅 disapproval
• 😰 nervousness
• 😠 anger
• 😳 embarrassment
• 😔 remorse
• 🤢 disgust
• 😭 grief
• 😕 confusion
• 😌 relief
• 😨 fear

!pip install falcon_evaluate -q

from falcon_evaluate.user_analytics import Emotions
import pandas as pd

Telecom - Customer Assistant Chatbot conversation

data = {"Session_ID":{"0":"47629","1":"47629","2":"47629","3":"47629","4":"47629","5":"47629","6":"47629","7":"47629"},"User_Journey_Stage":{"0":"Awareness","1":"Consideration","2":"Consideration","3":"Purchase","4":"Purchase","5":"Service/Support","6":"Service/Support","7":"Loyalty/Advocacy"},"Chatbot_Robert":{"0":"Robert: Hello! I'm Robert, your virtual assistant. How may I help you today?","1":"Robert: That's great to hear, Ramesh! We have a variety of plans that might suit your needs. Could you tell me a bit more about what you're looking for?","2":"Robert: I understand. Choosing the right plan can be confusing. Our Home Office plan offers high-speed internet with reliable customer support, which sounds like it might be a good fit for you. Would you like more details about this plan?","3":"Robert: The Home Office plan includes a 500 Mbps internet connection and 24/7 customer support. It's designed for heavy usage and multiple devices. Plus, we're currently offering a 10% discount for the first six months. How does that sound?","4":"Robert: Not at all, Ramesh. Our team will handle everything, ensuring a smooth setup process at a time that's convenient for you. Plus, our support team is here to help with any questions or concerns you might have.","5":"Robert: Fantastic choice, Ramesh! I can set up your account and schedule the installation right now. Could you please provide some additional details? [Customer provides details and the purchase is completed.] Robert: All set! Your installation is scheduled, and you'll receive a confirmation email shortly. Remember, our support team is always here to assist you. Is there anything else I can help you with today?","6":"","7":"Robert: You're welcome, Ramesh! We're excited to have you on board. If you love your new plan, don't hesitate to tell your friends or give us a shoutout on social media. Have a wonderful day!"},"Customer_Ramesh":{"0":"Ramesh: Hi, I've recently heard about your new internet plans and I'm interested in learning more.","1":"Ramesh: Well, I need a reliable connection for my home office, and I'm not sure which plan is the best fit.","2":"Ramesh: Yes, please.","3":"Ramesh: That sounds quite good. But I'm worried about installation and setup. Is it complicated?","4":"Ramesh: Alright, I'm in. How do I proceed with the purchase?","5":"","6":"Ramesh: No, that's all for now. Thank you for your help, Robert.","7":"Ramesh: Will do. Thanks again!"}}

Create the DataFrame

df = pd.DataFrame(data)

#Compute emotion score with Falcon evaluate module
remotions = Emotions()
result_df = emotions.evaluate(df.loc[['Chatbot_Robert','Customer_Ramesh']])
pd.concat([df[['Session_ID', 'User_Journey_Stage']],result_df], axis=1)

By mapping user emotions to the customer journey, the telecom customer assistance chatbot can provide a more empathetic, engaging, and effective service, ultimately enhancing customer satisfaction and loyalty.

Conclusion
In GenAI space , Falcon Evaluate’s User Analytics Emotion Module is a game-changer, transforming customer interactions into meaningful dialogues. This tool goes beyond mere analysis; it empathizes, making every customer’s emotion — each smile, frown, or tear — feel acknowledged and valued. It’s not just about solving problems; it’s about nurturing relationships and fostering trust. With Falcon Evaluate, we’re not just advancing technology; we’re enhancing our humanity, ensuring every chat is not just counted but felt, bringing us closer in a world that often feels distant. Let’s embrace this journey, where every interaction is not just heard but deeply understood.

Full Changelog: v0.1.12.0...v0.1.13.0

v0.1.12.0: Merge pull request #40 from Praveengovianalytics/feature/machine_ethics

Evaluating the alignment of LLM outputs with Shannon Vallor's 12 techno-moral values presents a unique approach to embedding ethical considerations into AI systems. Here are the benefits of using the Falcon-evaluate Machine_ethics module for LLM output evaluation:

Enhanced Ethical Oversight:

The module offers a systematic way to ensure that the outputs of language models align with predefined ethical values. This is particularly crucial in fields like journalism, education, and public communications, where the ethical implications of generated content are significant.

2. Automated Value Alignment Check:
   Manually checking LLM outputs for alignment with specific values can be time-consuming and prone to human error. An automated classifier Falcon-evaluate Machine_ethics module can provide quick and consistent assessments, making it a valuable tool for moderating content at scale.

🔍 Honesty - being truthful and transparent in one's interactions and intentions.
🧘 Self-control - the ability to regulate one's emotions, thoughts, and behaviors in the face of temptations and impulses.
🌱 Humility - recognizing and accepting one's limitations and the value and contributions of others.
⚖️ Justice - being committed to fairness and treating others with respect and equity.
🦁 Courage - the willingness to take risks or endure hardship to achieve a moral or worthwhile goal.
💞 Empathy - the capacity to understand and share the feelings of another.
🤲 Care - showing concern for the well-being of others and acting to promote and protect their interests.
🤝 Civility - showing respect for others, especially in the face of disagreement or conflict.
🔄 Flexibility - being willing and able to adapt one's beliefs and actions in response to changing circumstances or new information.
🌈 Tolerance - accepting and respecting differences and diversity in beliefs, values, and practices.
🔭 Perspective - the ability to understand and consider different viewpoints and the broader context of one's actions.
👑 Magnanimity - being generous, forgiving, and noble in spirit, especially towards rivals or those less powerful.

Fairness - Stereotype score

The primary goal of the Fairness Module is to evaluate LLMs for:

📊 Stereotypes: Assessing stereotypes through agreement on stereotypes, stereotype recognition, and stereotype query test.

🔍 Stereotypes Classes Identified

The Fairness Module - stereotypes identifies nine classes for evaluation:

🚫 unrelated: The text does not indicate any stereotype.
👫 stereotype_gender: The text indicates a gender stereotype.
🔄 anti-stereotype_gender: The text indicates an anti-gender stereotype.
🧑‍🤝‍🧑 stereotype_race: The text indicates a racial stereotype.
🔄 anti-stereotype_race: The text indicates an anti-racial stereotype.
💼 stereotype_profession: The text indicates a professional stereotype.
🔄 anti-stereotype_profession: The text indicates an anti-professional stereotype.
⛪ stereotype_religion: The text indicates a religious stereotype.
🔄 anti-stereotype_religion: The text indicates an anti-religious stereotype.

Security features

v0.1.10.0

#24 Version is updated

Issues fixed during volumetric testing

v0.1.9.2

Truncation issue resolved

Enhancement: Multi-Dimensional Evaluation Metrics Feature

Enhancement: Multi-Dimensional Evaluation Metrics Feature

1. Falcon Performance Quadrant
2. Documentation updated

Enhancement: Multi-Dimensional Evaluation Metrics Feature

#6 or GAFE-11 Enhancement: Multi-Dimensional Evaluation Metrics Feature

Description

In order to provide a more comprehensive and meaningful evaluation of our Language Models (LMs) through the Falcon Framework, I propose the addition of a multi-dimensional evaluation metrics feature. This feature should categorize and calculate various evaluation metrics organized into five distinct categories:

Readability and Complexity:

Automated Readability Index (ARI)
Flesch-Kincaid Grade Level
Language Modeling Performance:

Perplexity
Text Toxicity:

Toxicity Level
Text Similarity and Relevance:

BLEU Score
Cosine Similarity
Semantic Similarity
Jaccard Similarity
Information Retrieval:

Precision
Recall
F1-Score

Helps the ML engineers to come up with below quadrant to select the best model and config for production

Issue fixes and documentation

Issue fixes and documentation

v0.1.3

Merge pull request #3 from Praveengovianalytics/test

v0.1.3 python files added

v0.1.0

Initial release