Minutes estimation dataset for Hackweek

[rchan26]

Creating dataset for minutes estimation task for hack week.

• Fixed an error in TransferMarkt scraper where it wouldn't catch that a player was in the youth or reserve team for a Premier League club (e.g. wouldn't recognise that playing for Arsenal U18 means you'd be able to play for Arsenal)
• Re-run TransferMarkt scraper to update absence CSVs
• Create dataset with minutes played for each player and any absence reason (according to TransferMarkt) in airsenal/data/minutes_estimation_challenge.csv
  • Notebook to create this dataset is in notebooks/minutes_estimation_data.ipynb
• Made changes to get_player_history_df function in airsenal/framework/prediction_utils.py to allow getting the player history for all players (if all_players=True) and to not create blank entries (if fill_blank=False). By default, all_players=False and fill_blank=True to keep the original behaviour
  • Dataframe now queries the absence table to check if there was an absence for a game (according to data scraped from TransferMarkt)

Warning: May have some errors in airsenal/data/minutes_estimation_challenge.csv if TransferMarkt data was incorrect, or how we collect that data was incorrect - have found a few and deleted but there probably is more...

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[rchan26]

can we merge this @jack89roberts? it didn't get used in the hackweek, but can be useful for future work

[jack89roberts]

Sure 👍