Scraping Google Scholar Data of University Professors using Python

Overview

This project is aimed at scraping the data of university professors from Google Scholar, including their citation count, h-index, and other relevant information. The scraped data will be stored in a CSV file for further analysis. This project will be implemented using Python, BeautifulSoup, and the requests library. The project will be executed in an IPython Notebook.

Prerequisites

Before starting this project, you will need to have the following software installed on your machine:

• Python 3.x
• Jupyter Notebook
• pip
• virtualenv (optional)

Installation

1. Clone the repository to your local machine:

• git clone https://github.com/Anas1108/Scraping-Google-Scholar-Data-of-University-Professors.git

2. Change into the project directory:

• cd <repository_name>

3. Create a virtual environment (optional):

• virtualenv env

4. Activate the virtual environment (if you created one in step 3):

• source env/bin/activate

5. Install the required packages:

• pip install -r requirements.txt

Usage

1. Start Jupyter Notebook:

• jupyter notebook

2. Open the IPython Notebook file google_scholar_scraping.ipynb.

3. Follow the instructions in the notebook to scrape the data from Google Scholar.

4. The scraped data will be stored in a CSV file named professors_data.csv.

Note

Please note that Google Scholar may have restrictions on scraping its data. Use this project at your own risk and make sure to respect the terms of service of Google Scholar.