Transfer Learning and Fine Tuning in Sentiment Analysis - 42 Urduliz Bizkaia AI Project

Project Overview

This project introduces Transfer Learning in Sentiment Analysis, a key NLP technique used to determine whether a text conveys positive or negative sentiment. By leveraging pretrained LLM models, we aim to create a sentiment analysis system with minimal training data and computational resources.

The focus is on developing a binary sentiment analysis model using transfer learning techniques. Students will explore pretrained models, experiment with tokenization, and adjust hyperparameters to optimize performance while considering resource limitations.

Project Aim

The project provides hands-on experience in:

1. Transfer Learning: Adapt pretrained models like BERT, RoBERTa, or GPT-2 for sentiment analysis.
2. Sentiment Analysis: Create a binary classifier to categorize text (positive/negative).
3. Tokenization: Research and apply suitable tokenization methods.
4. Hyperparameter Tuning: Optimize hyperparameters like learning rate and batch size.
5. Resource Efficiency: Train models within limited computing resources.

Key Features

• Choose from a range of pretrained models (e.g., BERT, RoBERTa).
• Flexible resources: Use local or cloud computing (e.g., sgoinfre).
• Customize the project based on personal research and interests.

Important Clarification

The provided sample dataset is not intended for training but to give an idea of what sentiment analysis datasets look like. Please choose your own dataset (e.g., IMDB, Yelp, Amazon reviews) for training and evaluation.

Bonus Challenges

• Test your model on additional datasets for generalization.
• Compare tokenization techniques.
• Deploy your model for real-time sentiment prediction via a web interface.

Note on Fine-Tuning

Fine-tuning pretrained models is a critical part of this project. By adjusting hyperparameters and training on your specific dataset, you can improve model performance and efficiency.

Solution Notebook:

transfer_learning_AI.ipynb

A complete solution to the project is provided in the notebook transfer_learning_AI.ipynb. This notebook was developed and executed in Amazon SageMaker Studio Lab, a cloud-based environment designed for experimenting with machine learning workflows in a flexible and scalable way.

The solution reflects the project's core objective: to gain hands-on experience in cloud-based development using transfer learning and fine-tuning techniques in Natural Language Processing (NLP).

The notebook includes:

✅ The full implementation of a sentiment analysis model using transfer learning.

✅ Executed code cells, allowing immediate inspection of results and outputs.

✅ Additional code snippets and in-line explanations to support a deeper understanding of each phase, from tokenization to evaluation.

💡 This notebook is both a functional solution and a guided walkthrough for understanding how to apply transfer learning in a modern, cloud-based NLP project.

Before running the notebook in your own cloud environment (e.g., Google Colab or SageMaker Studio Lab), remember to update file paths if necessary.