Web Scraping with Python

Getting data from the original website of greek super league about each game from seasons 2006-2007 to 2021-2022. My purpose is to do an exploratory data analysis on these data at a later point.

Prerequisites

• Requests
• BeautifulSoup
• pandas

Walk-through of the project

Considering the schedule page, there are two things to consider at first:

Seasons:

until ...

Super league games (and not play off's / play out's):

Here we get a list of links, about each season and only the Super league games:

In each of these links, in order to get inside each game, we should get access in the circled arrows. Moreover, we should take care of the days that the games happened.

In each game there were different tables, but I wanted the statistics table as shown in the picture below. What needed was to add in the end of each game link the following string "/statistics".

In each page, I used the following sources for information (specific fixture and statistics).

From the statistics table, I got the information about the teams and the game stats

Considering the game stats, each trow was related to a specific category (goals, assists, etc.)

Inside each "trow" there was info about the category, home_team statistic and away_team statistic

Then I noticed that for each statistic, the "pattern" of the html code was the same

For goals

For headers

The final step before scraping the information, was to create an empty dataframe with the appropriate columns

The code

The obtained dataframe