Home
This project was completed as part of the "Twitter Sentiment Extraction" Kaggle competition in June 2020 (https://www.kaggle.com/c/tweet-sentiment-extraction). This project involved extracting the text related to emotions found in Tweets. Our team submitted this git hub project to Kaggle, we scored in top 1000 :)
We hope that this project will help you get a jump start on sentiment extraction!
Our code is largely based on the simple transformer work done by: xxx here http://google.com
Sentiment extraction takes sentiment analysis to the next level. In legacy NLP tasks it was enough to predict the emotion of a certain tweet, for example:
| Tweet | Predicted Sentiment |
|---|---|
| Shia LaBeouf is my hero! | Positive |
However, sentiment extraction is a more advanced form of NLP that extracts the words that contribute to the reason behind a particular sentiment. For example, see the example below:
| Tweet | Predicted Sentiment | Extracted Text |
|---|---|---|
| Shia LaBeouf is my hero! | Positive | hero! |
Sentiment extraction has important business applications such as identifying additional information from tweets, product reviews, customer service. If you have the words the cause positive, negative, or neutral sentiment than you can use that data to improve your business.
NLP is an exciting area of artificial intelligence and machine learning. There are numerous algorithms and models for completing traditional NLP tasks such as:
- Information extraction
- Topic modeling
- Summarizing
- Similarity
- Etc..
However, for the purpose of sentiment extraction transformers are the only way (that we know of) to extract the text related to the sentiment in these Tweets.
Transformers for machine learning are a relatively new and advanced AI technique for NLP tasks. Transformers are a deep learning technique, i.e. an advanced neural network architecture that is suited for identifying meaningful relationships in unstructured data (such as text).
The next section will go into detail on this.
- Advanced Techniques
- We used Huggingface pertained transformer models (Roberta-base-cased) and added customized question-answer head layers using TensorFlow to reach a better question answering result.
- Please see more details from this notebook Link to Tensorflow approach
- Key difference:
- TensorFlow approach can support more customized functions including customizing loss function, question answering structures, the number of inputs, and doing k-fold.
- TensorFlow approach can help you understand deeper on how transformers work by creating attention mask, input ids, and padding your train set.
- The training time takes longer for this approach as we are using the k-fold method to avoid overfitting
- Key Insights
- Lessons Learned and Best Practices
- Resources