Skip to content

A multithreaded web scraper that collects all player data from fbref.com for the top 5 European soccer leagues and stores it in a MySQL database.

Notifications You must be signed in to change notification settings

biniyamyohannes/fbref-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 

Repository files navigation

fbref-scraper

A multithreaded web scraper that collects all player data from fbref.com for the top 5 European soccer leagues and stores it in a MySQL database.

Crawler/Scraper

A multithreaded web scraper using BeautifulSoup. Iteratively crawls through teams from the top 5 European soccer leagues and scrapes the player performance data for their players.
Sample run with 8 worker processes:

animated

Database

A MySQL database modeled after the format of tables from fbref. PyMySQL is used to connect to and query the database.
A look at the database and a sample query. Select all players who have averaged more than 15 goals per season. No surprises here...

animated

Dataset

You can find the final dataset here: https://www.kaggle.com/biniyamyohannes/soccer-player-data-from-fbrefcom

About

A multithreaded web scraper that collects all player data from fbref.com for the top 5 European soccer leagues and stores it in a MySQL database.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Languages