Customer Sentiment and Trend Analysis

Overview

The Customer Sentiment Analysis project leverages natural language processing (NLP) techniques to analyze customer reviews from Amazon. By employing machine learning algorithms, this project aims to classify reviews as either positive or negative based on the sentiments expressed within the text. This analysis provides valuable insights into customer opinions, enabling businesses to understand customer satisfaction and improve their products and services.

Dataset

The dataset used in this project is the Amazon Reviews Dataset. The dataset includes the following columns:

• Sentiment: The target variable indicating the sentiment of the review (0 for negative, 1 for positive).
• Title: A brief summary or title of the review.
• Review: The full text of the customer's review.

Objectives

The primary objectives of this project are:

• To preprocess the text data for effective analysis, including tokenization, stopword removal, lemmatization, and spell checking.
• To apply different machine learning algorithms (Logistic Regression, Random Forest, SVM, and XGBoost) for sentiment classification.
• To evaluate the performance of each model using metrics such as accuracy, precision, recall, and F1-score.
• To visualize the distribution of sentiments in the dataset and the performance of different models using relevant graphs and charts.

Methodology

1. Data Collection: We sourced a comprehensive dataset of Amazon reviews that includes the sentiment labels, titles, and review texts.
2. Data Preprocessing: The data was cleaned and preprocessed to remove noise, such as HTML tags, punctuation, and stopwords. This involved:
   • Tokenization
   • Lemmatization
   • Handling of contractions and special characters
3. Exploratory Data Analysis (EDA): We performed EDA to visualize the distribution of sentiments, identify trends, and understand the dataset better. This included:
   • Creating visualizations to show the frequency of positive and negative reviews.
   • Analyzing the correlation between review length and sentiment.
4. Feature Engineering: Utilized TF-IDF vectorization to convert the review texts into numerical representations suitable for machine learning models.
5. Model Selection and Training: We experimented with various machine learning algorithms, including:
   • Logistic Regression
   • Random Forest
   • Support Vector Machine (SVM)
   • Decision Trees
   • XGBoost
6. Model Evaluation: The performance of each model was evaluated using accuracy, confusion matrix, and classification reports. We selected the best-performing model based on these metrics.
7. Deployment: We implemented a simple user interface to allow users to input reviews and receive sentiment predictions from the trained model.

Fine-Tuning and Model Optimization

To enhance the performance of our sentiment classification models, we employed fine-tuning techniques on pre-trained models. Fine-tuning involves taking an existing model trained on a similar task and adjusting it for our specific dataset. This approach leverages transfer learning, enabling us to achieve high accuracy even with limited training data. By adjusting hyperparameters and retraining the model on our dataset.

Generative Adversarial Networks (GANs)

In addition to traditional machine learning techniques, we explored the use of Generative Adversarial Networks (GANs) to augment our dataset. GANs consist of two neural networks, a generator and a discriminator, that work together to create synthetic data. In this project, we trained the GAN on the existing review data to generate additional samples.

Azure for Deployment and MLOps

To facilitate deployment and streamline our machine learning operations (MLOps), we utilized Azure's cloud services. Azure provides a robust platform for deploying machine learning models, managing resources, and monitoring performance. In our project, we leveraged Azure Machine Learning to:

• Deploy our trained sentiment analysis model as a web service, enabling users to interact with it via a REST API.
• Monitor model performance and implement iterative improvements based on user feedback.
• Securely store and manage the dataset in Azure Blob Storage.

By utilizing Azure, we ensured our sentiment analysis solution is scalable, accessible, and maintainable, setting the stage for future enhancements and updates.

Team

• Alaa Sayed
• Omnia Meabed
• Omnia Samir
• Asmaa Mohamed
• Rana Yasser