Skip to content

This repo contains code for scraping tweets related to AI and Data Jobs from Twitter using snscrape and analyzing their sentiment using the "twitter-roberta-base-sentiment" model.

License

Notifications You must be signed in to change notification settings

hossam-elshabory/AI-and-Data-jobs-Twitter-Sentiment-Analysis

Repository files navigation

AI and Data Jobs Tweets Sentiment Analysis

project-banner

Introduction

AI is significantly impacting data jobs, such as data science, analysis, and engineering. With AI, these jobs are becoming more efficient and effective, leading to faster and more accurate analysis. However, as AI continues to develop rapidly with the emergence of new tools and implementations, some individuals may struggle to adapt to the changes. This may result in a scarcity of professionals, particularly as the new AI tools replace some jobs, leading to layoffs and unemployment.


Aim of this analysis

  • The goal of this project is to analyze the sentiment and public opinion surrounding AI and data jobs on Twitter.

Data

The data was scrapped by the snscrape python module, the scrapping code is on a google colab notebook, for some reason i keep getting network errors when i run the code inside a kaggle notebook. Please leave a comment below if you know how to address this issue.

Open In Colab


Conclusion

The sentiment analysis showed that most tweets' sentiments were neutral. However, after six months of decline, the sentiment score spiked following the release of ChatGPT. The analysis also revealed that the topic of AI and its tools is still relatively new, with people unsure of what to think about these emerging technologies. As AI continues to evolve, it will undoubtedly impact the data professions and jobs landscape, leading to significant changes in the field.

Findings Summary

  • Most of the tweets sentiments labels are Neutral by ~69%.

    Click to View Visulaization.

    Alt text


  • The topic of AI related to data jobs has seen a decline in the sentiment score from the beginning of July to December.

    Click to View Visulaization.

    Alt text


  • The release of Chat-GPT on November 30, 2022 spiked the sentiment score up again.

    Click to View Visulaization.

    Alt text


  • Data analysis has the most steady and consistent sentiment trend COMPARED to data science and data engineering.

    Click to View Visulaization.

    Alt text


  • Data engineering sentiment score didn't increase like the other job titles on the release date of chat-GPT, but a significant increase on the month of February.

    Click to View Visulaization.

    Alt text

Acknowledgments

About

This repo contains code for scraping tweets related to AI and Data Jobs from Twitter using snscrape and analyzing their sentiment using the "twitter-roberta-base-sentiment" model.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published