Demographics-of-Best-CS-Scientists-Worldwide

Problem Statement

The goal of this project is to gather information of best 1000 Computer Science researchers from this website.
Later we utilized the scraped data to understand the following demographics and correlations using Tableau Dashboard:

1. A barchart of countries with average publications.
2. European countries with the number of scientists in a map (excluding Russia)
3. Which Middle Eastern universities are good at research? (using citations as metric)
4. Which column is directly correlated with World Rank column? We wanted to understand how the ranking was done.

You can visit the public dashboard here.

Findings and Observations from the Dashboard

1. Brazilian scientists have the highest average publications.
2. Researchers from King Abdullah University of Science and Technology (KAUST), Saudi Arabia have the highest number of average citations.
3. Among European countries, United Kingdom (UK) has the highest number of scientists among the top 1000.
4. The ranking was done most probably using H-Index.

Build From Sources and Run the Selenium Scraper

1. Clone the repo

git clone https://github.com/msi1427/Demographics-of-Best-CS-Scientists-Worldwide.git

2. Intialize and activate virtual environment

virtualenv --no-site-packages  venv
source venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

4. Download Chrome WebDrive from https://chromedriver.chromium.org/downloads
5. Run the scraper

python selenium_scraper/scraper.py --chromedriver_path <path_to_chromedriver>

6. You will get a file named best_cs_scientist_details.csv containing all the required fields. Alternatively, check our scraped data here: https://github.com/msi1427/Demographics-of-Best-CS-Scientists-Worldwide/blob/main/selenium_scraper/best_cs_scientist_details.csv