Skip to content

blnkpagelabs/nflscraPy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Welcome to nflscraPy

This package was inspired by the creators of nflscrapR and nflfastR and the tremendous influence they have had on the open-source NFL community

The functionality of nflscraPy was designed to allow Python users to easily ingest boxscore and seasonal data from publicly available resources - in particular, Pro Football Reference

Hopefully, this package builds upon the availabilty of open-source resources for the football and data analytics community

-----------------------------------------------------

Installation

pip install nflscraPy

-----------------------------------------------------

Request Limits

To be respectful of Pro Football Reference's servers each function incorporates sleeps between 3.5 to 5.5 seconds after every request

Remove or reduce these intervals at your own risk – Pro Football Reference & Cloudflare will temporarily/permanently suspend access if you throttle their servers

-----------------------------------------------------

Table of Contents

-----------------------------------------------------

Seasons

import nflscraPy

season_gamelogs = nflscraPy._gamelogs(
    2022
)

Description:

Returns all Gamelogs & Boxscore Stats Links for every NFL season

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

status

The Status of the game: upcoming/postponed | closed

season

The Season's Year

week

The Week of the Game

week_day

Weekday of the Game

event_date

The Date of the Game

tm_nano

nflSlowPy Nano ID

tm_market

nflSlowPy Market

tm_name

nflSlowPy Name

tm_alias

nflSlowPy Alias

tm_alt_market

Pro Football Reference Market

tm_alt_alias

Pro Football Reference Alias

opp_nano

nflSlowPy Nano ID

opp_market

nflSlowPy Market

opp_name

nflSlowPy Name

opp_alias

nflSlowPy Alias

opp_alt_market

Pro Football Reference Market

opp_alt_alias

Pro Football Reference Alias

tm_location

Tm's Location: H | A | N

opp_location

Opp's Location: H | A | N

tm_score

Tm's Full Game Score

opp_score

Opp's Full Game Score

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Metadata

import nflscraPy

gamelog_metadata = nflscraPy._gamelog_metadata(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Returns Stadium Information | Game Weather Conditions | Vegas Spreads & Totals | Coin Toss Outcomes

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

season

The Season's Year

event_date

The Date of the Game

tm_nano

nflSlowPy Nano ID

tm_market

nflSlowPy Market

tm_name

nflSlowPy Name

tm_alias

nflSlowPy Alias

opp_nano

nflSlowPy Nano ID

opp_market

nflSlowPy Market

opp_name

nflSlowPy Name

opp_alias

nflSlowPy Alias

tm_spread

Consensus Closing Spread Per Pro Football Reference – a spread of 0 indicates the Game Was a Pk'em

opp_spread

Consensus Closing Spread Per Pro Football Reference – a spread of 0 indicates the Game Was a Pk'em

total

Consensus Closing Total Per Pro Football Reference

attendance

The Total Recorded Attendance For The Event

duration

The Duration of the Event in Minutes – a duration of 210 indicates the game lasted for 3:30 Hrs

roof_type

The Type of the Stadiums Roof (Where Applicable)

surface_type

The Type of the Stadiums Field Surface Type

won_toss

The Alias of the Team That Won the Opening Coin Toss

won_toss_decision

Whether the team that Won the Opening Coin Toss Deferred or Accepted the Opening Kickoff

History on Coin Toss Decisions From 2009 to Present

won_toss_overtime

The Alias of the Team That Won the Overtime Coin Toss

won_toss_overtime_decision

Whether the team that Won the Overtime Coin Toss Deferred or Accepted the Overtime Kickoff

History on Coin Toss Decisions From 2009 to Present. (Let's Be Honest, Every Team is Accepting This)

temperature

The Recorded Temperature of the Game in Fahrenheit

humidity_pct

The Recorded Humidity of the Game

wind_speed

The Recorded Wind Speed of the Game (Where Applicable - i.e., Indoor Fields Will Not Have Wind Speed)

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Statistics

import nflscraPy

gamelog_statistics = nflscraPy._gamelog_statistics(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Returns a Game's Basic Boxscore Statistics

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

Attributes:

season

The Season's Year

event_date

The Date of the Game

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

rush_att

Total Offensive Rushing Attempts

rush_yds

Total Offensive Rushing Yards

rush_tds

Total Offensive Rushing Touchdowns

pass_cmp

Total Offensive Passes Completed

pass_att

Total Offensive Passes Attempted

pass_cmp_pct

Ratio of Passes Completed to Attempts

pass_yds

Total Offensive Passing Yards

pass_tds

Total Offensive Passing Touchdowns

pass_int

Total Offensive Passing Interceptions

passer_rating

Team's Total Passer Rating

net_pass_yds

Total Offensive Passing Net Yards

total_yds

Overall Offensive Yards

times_sacked

Total Times That a Quarterback(s) Was Sacked For

yds_sacked_for

Total Yards That a Quarterback(s) Was Sacked For

fumbles

Total Fumbles – Includes Those That Were Recovered and Lost

fumbles_lost

Fumbles Lost and Recovered by Opponent's Defense

turnovers

Total Turnovers, Including Fumbles Lost and Interceptions

penalties

Total Number of Offensive and Defensive Penalties

penalty_yds

Total Number of Offensive and Defensive Penalty Yards

first_downs

Total Offensive First Downs

third_down_conv

Offensive Third Down Conversions

third_down_att

Offensive Third Down Attempts

third_down_conv_pct

Ratio of Third Down Conversions to Attempts

fourth_down_conv

Offensive Fourth Down Conversions

fourth_down_att

Offensive Fourth Down Attempts

fourth_down_conv_pct

Ratio of Fourth Down Conversions to Attempts

possesion_time

Total Offensive Possession Time in Seconds

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Expected Points

import nflscraPy

gamelog_expected_points = nflscraPy._gamelog_expected_points(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Expected Points utilizes historic play-by-play data to estimate the 'point value' associated with the start of each play, based on down, distance to go, and field position

For further information on this please see Pro Football Reference's Blog Post on the topic

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

Attributes

season

The Season's Year

event_date

The Date of the Game

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

exp_pts

Total Expected Points For the Game (Negative For Losing Team and Positive For Winning Team)

exp_pts_off

Total Offensive Expected Points

Formula = Rusing Offense + Passing Offense + Offensive Penalties

exp_pts_off_pass

Expected Points From Passing Offense Plays

exp_pts_off_rush

Expected Points From Rushing Offense Plays

exp_pts_off_turnover

Expected Points From Offense Turnovers

exp_pts_def

Total Defensive Expected Points

Formula = Rusing Defense + Passing Defense + Defensive Penalties

exp_pts_def_pass

Expected Points From Passing Defense Plays

exp_pts_def_rush

Expected Points From Rushing Defense Plays

exp_pts_def_turnover

Expected Points From Defense Turnovers

exp_pts_st

Total Sepecial Teams Expected Points

Formula = Kickoff + Kick Return + Punt + Punt Return + FGs + XPs

exp_pts_kickoff

Expected Points From Kickoffs

exp_pts_kick_return

Expected Points From Kick Returns

exp_pts_punt

Expected Points From Punt Plays

exp_pts_punt_return

Expected Points From Punt Return Plays

exp_pts_fg_xp

Expected Points From FGs and XPs

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Scoring

import nflscraPy

gamelog_roster = nflscraPy._gamelog_roster(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Returns the Quarter, Time, Description, of every Scoring Event During a Game

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

season

The Season's Year

event_date

The Date of the Game

tm_nano

nflSlowPy Nano ID

tm_market

nflSlowPy Market

tm_name

nflSlowPy Name

tm_alias

nflSlowPy Alias

opp_nano

nflSlowPy Nano ID

opp_market

nflSlowPy Market

opp_name

nflSlowPy Name

opp_alias

nflSlowPy Alias

quarter

The Quarter In Which the Scored Occured

time

The Time on the Clock in Seconds When the Score Occured – a clock of 120 indicates there was 2 minutes remaining in the quarter when the score occurred.

scoring_team

The nflSlowPy Alias Corresponding to the Scoring Team

tm_score

The Score of the Game's tm After the Scoring Event Occured

opp_score

The Score of the Game's opp After the Scoring Event Occured

description

The Description of the Scoring Event – Including the Player's invovled, the play type, and associated yardage of the scoring event.

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Roster

import nflscraPy

gamelog_roster = nflscraPy._gamelog_roster(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Returns the Roster of Active Players Across Both Teams For a Game

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

season

The Season's Year

event_date

The Date of the Game

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

player_href

The Player's Pro Football Reference HREF – Useful as a Unique Player ID and to Scrape Additional Player Statistics Not Yet Covered by nflSlowPy

player

Player's First and Last Name Per Pro Football Reference

position

A Player's Primary Position Per Pro Football Reference

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Snap Counts

import nflscraPy

gamelog_snap_counts = nflscraPy._gamelog_snap_counts(
    'https://www.pro-football-reference.com/boxscores/202212180jax.htm'
)

Description:

Returns the Offensive | Defensive | Special Teams Snap Counts and Percentages For a Game

History:

2012 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

season

The Season's Year

event_date

The Date of the Game

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

player_href

The Player's Pro Football Reference HREF – Useful as a Unique Player ID and to Scrape Additional Player Statistics Not Yet Covered by nflSlowPy

player

Player's First and Last Name Per Pro Football Reference

position

A Player's Primary Position Per Pro Football Reference

snap_count_offense

The Total Offensive Number of Snap Counts Taken by a Player

snap_count_offense_pct

The Percentage of Offensive Snap Counts Taken by a Player

Formula = Player Snap Counts / Total Offensive Plays

snap_count_defense

The Total Defensive Number of Snap Counts Taken by a Player

snap_count_defense_pct

The Percentage of Defensive Snap Counts Taken by a Player.

Formula = Player Snap Counts / Total Defensive Plays

snap_count_special_teams

The Total Special Teams Number of Snap Counts Taken by a Player

snap_count_special_teams_pct

The Percentage of Special Teams Snap Counts Taken by a Player.

Formula = Player Snap Counts / Total Special Teams Plays

boxscore_stats_link

The Gamelog's Primary Key

-----------------------------------------------------

Season Splits

import nflscraPy

szn_splits_for_tm = nflscraPy._season_splits(
    2022,
    'jax',
    'For',
)

szn_splits_against_tm = nflscraPy._season_splits(
    2022,
    'jax',
    'Against',
)

Description:

Returns either the Offensive or Defensive Season Split Stats.

Splits are broken down by Game Location, Quarter, Yards To Go, Game Siutuation, and More

History:

2000 to Present

Downloads:

To Be Respectful of Pro Football Reference's Servers Please Backfill with Historic Data Available in the Releases

Downloads

Attributes:

season

The Season's Year

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

splits_by

Scenario of the Split By: Location, Quarter, Down, Distance, Score Differential, Game Situation, and Play Type

splits_type

Individual Scenario of the Split

splits_side

Whether the Split Stats Reflect the Teams's Performance or Opponent's in any Given Scenario – For | Against

total_plays

Number of Plays For The Split

yds_to_go

Avg. Yards to Go

avg_yds_gained

Avg. Yards Gained

rush_att

Number of Rush Attempts

rush_yds

Number of Rush Yards

rush_yds_per_att

Number of Rush Yards Per Attempt

rush_tds

Number of Rushing Touchdowns

rush_first_downs

Number of Rushing First Downs

pass_cmp

Number of Passes Completed

pass_att

Number of Passes Attempts

pass_cmp_pct

Passing Completion Percentage

pass_yds

Number of Passing Yards

pass_yds_per_att

Number of Passing Yards Per Attempt

pass_adj_net_yds_per_att

Number of Net Adjusted Passing Yards Per Attempt

pass_int

Number of Passing Interceptions

pass_tds

Number of Passing Touchdowns

pass_first_downs

Number of Passing First Downs

passer_rating

Team Passer Rating

times_sacked

Number of Times Sacked

splits_stats_link

The Season Split's HREF

-----------------------------------------------------

FiveThirtyEight

import nflscraPy

five_thirty_eight = nflscraPy._five_thirty_eight()

Description:

FiveThirtyEight's traditional model uses Elo ratings (a measure of strength based on head-to-head results and quality of opponent) to calculate teams’ chances of winning their regular-season games and advancing to and through the playoffs. Our quarterback-adjusted Elo model incorporates news reports to project likely starters for every upcoming game and uses our quarterback Elo ratings to adjust win probabilities for those games. A team’s current quarterback adjustment is based on its likely starter in its next game and how much better or worse that QB is than the team’s top starter. Full methodology »

Full Methodology

History:

1970 to Present

Downloads:

To Be Respectful of fiveThirtyEight's Servers Please Backfill with Historic Data Available in the Releases

Downloads

Attributes:

season

The Season's Year

event_date

The Date of the Game

neutral

playoff

tm_nano

nflSlowPy Nano ID

tm_market

nflSlowPy Market

tm_name

nflSlowPy Name

tm_alias

nflSlowPy Alias

tm_alt_alias

FiveThirtyEight Alias

opp_nano

nflSlowPy Nano ID

opp_market

nflSlowPy Market

opp_name

nflSlowPy Name

opp_alias

nflSlowPy Alias

opp_alt_alias

FiveThirtyEight Alias - Preserved Historicall

tm_score

Tm's Full Game Score

opp_score

Opp's Full Game Score

tm_elo_pre

Tm's Pregame Elo Rating

opp_elo_pre

Opp's Pregame Elo Rating

tm_elo_win_prob

Tm's Pregame Win Probability

opp_elo_win_prob

Opp's Pregame Win Probability

tm_elo_post

Tm's Updated Postgame Elo Rating

opp_elo_post

Opp's Updated Pregame Elo Rating

tm_qb_elo_pre_game

Tm's Pregame Elo Rating w/ Quarterback Adjustment Incl.

opp_qb_elo_pre_game

Opp's Pregame Elo Rating w/ Quarterback Adjustment Incl.

tm_qb

Tm's Starting Quarterback

opp_qb

Opp's Starting Quarterback

tm_qb_elo_value_pre

Tm's Starting Quarterback Elo Pregame

opp_qb_elo_value_pre

Opp's Starting Quarterback Elo Pregame

tm_qb_elo_adj

Tm's Starting Quarterback Elo Adjustment

opp_qb_elo_adj

Opp's Starting Quarterback Elo Adjustment

tm_qb_elo_win_prob

Tm's Pregame Win Probability w/ Quarterback Adjustment Incl.

opp_qb_elo_win_prob

Opp's Pregame Win Probability w/ Quarterback Adjustment Incl.

tm_qb_game_value

Tm's QB Game Elo Value – Performance Based

opp_qb_game_value

Opp's QB Game Elo Value – Performance Based

tm_qb_post_game_value

Tm's Starting Quarterback Elo Postgame - Adjusted for Game Elo Performance

opp_qb_post_game_value

Opp's Starting Quarterback Elo Postgame - Adjusted for Game Elo Performance

tm_qb_elo_post_game

Tm's Updated Postgame Elo Rating w/ Quarterback Adjustment Incl.

opp_qb_elo_post_game

Opp's Updated Postgame Elo Rating w/ Quarterback Adjustment Incl.

quality

Quality is determined by the harmonic mean of the teams’ Elo ratings – Based on a 0-100 Scale

importance

Importance measures how much the result will alter playoff projections – Based on a 0-100 Scale

total_rating

The overall number is the average of the quality and importance values.

-----------------------------------------------------

Madden Ratings

Scraper Unavailable

Description:

Returns all available Madden Ratings from the Madden Database

History:

2021 to Present

Downloads:

To Be Respectful of Madden's Servers Please Backfill and Forward Fill with Historic/Present Data Available in the Releases

Downloads

status

The Status of the Madden Ratings

season

The Season's Year

week

The Week of the Game

iteration_date

The "as of" Publishing Date of the Madden Ratings

The anchor point of each player's Madden Rating

iteration

The "as of" Publishing Week type of the Madden Ratings

nano

Madden's Team ID

nano

nflSlowPy Nano ID

market

nflSlowPy Market

name

nflSlowPy Name

alias

nflSlowPy Alias

player_first_name

Player's First Name

player_last_name

Player's Last Name

player_full_name

Player's Full Name

player_asset_name

Player's Unique Madden Asset Name

player_jersey

Player's Jersey Number

player_position

Player's Primary Position

player_archetype

Player's Primary Position with Additional Specialized Attributes

player_running_style

Player's Running Style – Default indicates Average Stride

player_handedness

Player Handedness – Right or Left

player_height

Player Height in Inches

player_weight

Player Weight in Pounds

player_age

Player Age

player_years_pro

Number of Years Player was Pro

player_college

Player's College

player_signing_bonus

Player's Signing Bonus

player_total_salary

Player's Total Salary

player_impact_value

The Replacement Value of a Player Calculated Based on a Player's Overall Rating.

Used to determine the "value" of a player's replacement due to Injury/Illnes//Etc.

Formula from "Sharper: A Guide to Modern Sports Betting"

player_impact_value = (overall_rating - 66) / 8

overall_rating

Player's Overall Madden Rating

For Overall and All Ratings, the Higher the Rating the Better

injury_rating

Player's Injury Rating

toughness_rating

Player's Toughness Rating

strength_rating

Player's Strength Rating

stamina_rating

Player's Stamina Rating

agility_rating

Player's Agility Rating

speed_rating

Player's Speed Rating

jumping_rating

Player's Jumping Rating

play_action_rating

Player's Play Action Rating

awareness_rating

Player's Awareness Rating

break_sack_rating

Player's Break Sack Rating

release_rating

Player's Release Rating

throw_power_rating

Player's Throw Power Rating

throw_accuracy_short_rating

Player's Throw Accuracy Short Routes Rating

throw_accuracy_mid_rating

Player's Throw Accuracy Mid Routes Rating

throw_accuracy_deep_rating

Player's Throw Accuracy Deep Routes Rating

throw_on_the_run_rating

Player's Throw on the Run Rating

throw_under_pressure_rating

Player's Throw Under Pressure Rating

trucking_rating

Player's Trucking Rating

carrying_rating

Player's Carrying Rating

juke_move_rating

Player's Juke Move Rating

break_tackle_rating

Player's Break Tackle Rating

change_of_direction_rating

Player's Change of Direction Rating

block_shedding_rating

Player's Block Shedding Rating

power_moves_rating

Player's Power Moves Rating

stiff_arm_rating

Player's Stiff Arm Rating

acceleration_rating

Player's Acceleration Rating

spin_move_rating

Player's Spin Move Rating

finesse_moves_rating

Player's Finesse Moves Rating

ball_carrier_vision_rating

Player's Ball Carrier Vision Rating

catching_rating

Player's Catching Rating

catch_in_traffic_rating

Player's Catch in Traffic Rating

spectacular_catch_rating

Player's Spectacular Catch Rating

short_route_running_rating

Player's Short Route Running Rating

medium_route_running_rating

Player's Medium Route Running Rating

deep_route_running_rating

Player's Deep Route Running Rating

tackle_rating

Player's Tackle Rating

pursuit_rating

Player's Pursuit Rating

press_rating

Player's Press Rating

play_recognition_rating

Player's Play Recognition Rating

man_coverage_rating

Player's Main Coverage Rating

zone_coverage_rating

Player's Zone Coverage Rating

hit_power_rating

Player's Hit Power Rating

lead_block_rating

Player's Lead Block Rating

impact_blocking_rating

Player's Impact Blocking Rating

run_block_rating

Player's Run Block Rating

run_block_power_rating

Player's Run Block Power Rating

run_block_finesse_rating

Player's Run Block Finesse Rating

pass_block_rating

Player's Pass Block Rating

pass_block_power_rating

Player's Pass Block Power Rating

pass_block_finesse_rating

Player's Pass Block Finesse Rating

kick_return_rating

Player's Kick Return Rating

kick_power_rating

Player's Kick Power Rating

kick_accuracy_rating

Player's Kick Accuracy Rating

primary_key

Primary Key Associated with the Madden Database Entry

About

Datasets and Scraping Functions for NFL Data

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages