This project was part of an Independent Study during my MS in Computer Science Capstone Project at Rochester Institute of Technology, NY.
In data science projects, analysts usually have to work on data that is outside their field of domain. In this independent study, we apply techniques from data science to the domain of employer compensation, using standard and some novel techniques. This project is a study of cognitive biases that affect an employees’ compensation. Exploratory data analysis is performed on salary information obtained from the datasets available on the United Kingdom and the United States government websites. Additional data is gathered by scraping websites providing relevant information and Twitter tweets are taken to perform sentiment analysis. We explore how factors like location of the company, composition of board of directors, size of the company, occupation etc. affect the compensation of both the genders. We explore Twitter tweets by performing sentiment analysis by using VADER, to observe what the social media has to say about the issue. Various reports and statistics regarding the wage gap issue are studied. Tools used to analyze and explore this data include Python, Plotly, R, Tableau, Seaborn etc.
All of the findings, important code snippets, data sources and references are described in the report.