filter position players from pitching stats & filter pitchers from batting stats

[TobiasCortese]

I'd like to filter position players from my pitching stats and I'd also like to filter pitchers from my batting stats.
maybe there's a datasource I can join to which contains position info (maybe that's roster data?)? or maybe there's a function?

[TobiasCortese]

I'm thinking that appearances() from the lahman database is the best way to deduce this info. specifically, iterate over the games played at each position and select the position where the player played most of their games as that player's position

[BrayanMnz]

What is your desired output @TobiasCortese,
could you add an example ?

[TobiasCortese]

@BrayanMnz Thanks for following up! Desired output is a method to determine whether a player is actually a pitcher (as opposed to a position player who has pitched) or actually a position player (as opposed to a pitcher who has recorded some ABs). From a tangible perspective, maybe the desired output would be a list of all players in history with IDs and that pitcher/position information. and in that list, Babe Ruth and Shoei Ohtani would have 2 rows (or maybe 1 row with the value 'both') indicating that they are legitimately both.

here's the hack I've developed so far...

def identify_hitters_and_pitchers():
appearances_df = spark.createDataFrame( appearances() )

appearances_df = appearances_df
.withColumn( 'games_in_the_field', f.col('G_c') + f.col('G_1b') + f.col('G_2b') + f.col('G_3b') +
f.col('G_ss') + f.col('G_lf') + f.col('G_cf') + f.col('G_rf') + f.col('G_dh') )
.withColumnRenamed( 'G_p', 'games_pitched' )
.withColumnRenamed( 'playerID', 'bbrefID' )

positions_df = appearances_df.groupBy( ['bbrefID'] )
.agg( f.sum( 'games_pitched' ).alias( 'games_pitched' ),
f.sum( 'games_in_the_field' ).alias( 'games_in_the_field' ) )
.withColumn( 'position', f.when( f.col( 'games_pitched' ) >= f.col( 'games_in_the_field' ), 'pitcher' ).otherwise( 'hitter' ) )

return positions_df

[BrayanMnz]

Looking forward to see your Pull Request with these changes.
Looks promising.

[TobiasCortese]

So it looks like the info is available from get_splits()

player_info: Boolean. Optional. If set to True, the get_splits function will return both the split stats dataframe and a dictionary of player info that contains player position, handedness, height, weight, and team

but I'm getting an index out of range error when attempting to execute the sample code

[TobiasCortese]

Could someone submit a pull request to change the following code in split_stats.py per the below?
I'm unable to push changes from databricks to GitHub at the moment and I need to spend some time getting that worked out (if anyone has tips, I'd be grateful).

Old
about_info = soup.find_all( "div", {"itemtype": "https://schema.org/Person"})

New
about_info = soup.find_all( "div", id="info", class_=re.compile("players") )

[TobiasCortese]

figured it out...
#314

[BrayanMnz]

so, since you figured out how to solve that issue - is the splits() working now ? @TobiasCortese

[TobiasCortese]

Hey @BrayanMnz - I'm not sure I understand the question.

there was never any problem with this method...
get_splits('troutmi01')

but this method generates an error...
get_splits('troutmi01', player_info=True)

so i downloaded the code locally, made adjustments, got it working, tested it and then forked the repo, created a branch and pushed the branch to main (pull request 314, which is currently outstanding). So the code works on my machine but is still broken in production.

[BrayanMnz]

Hi Tobias, still broken on production because the PR has not been merged yet - it will be available on production once it get into the master branch and we made a release.

[TobiasCortese]

Yep. That process generally aligns with standard practices and is as expected, and I've been tracking the status of the PR since inception. Thanks for confirming.

[tjburch]

Thanks @TobiasCortese! See my comment in your PR (#314), this was covered in #318. Appreciate the sleuthing and finding this, sorry it got doubled up,