How to generate games.csv

[YakobovLior]

Hey @Nathanlauga,

First of all, nice job with this project, I'm a huge NBA fan.
I'm trying to figure out which script I should run in order to generate the file games.csv,
I tried to run the script files but couldn't figure out which one should create it.

Thanks,
Lior

[Nathanlauga]

Hey !
Thanks for the message. Sorry for the mess, it's not the best code that I've done.
But to run the new game generation you need to download the current data from https://www.kaggle.com/nathanlauga/nba-games and put it into the data/ directory at the root of the repository.

Then, you can run python scripts/get_new_games.py and you should see scraping doing the job to add the new information.

I hope it answers your question.

Have a nice day,
Nathan.

[YakobovLior]

Hey @Nathanlauga,
Thanks for the really quick answer, so from what you're saying, there is no code to generate the file games.csv from scratch? only updating it with the script file python scripts/get_new_games.py ?

I need only the games.csv file and I also need the to have the code which is able to generate it (of course that I won't run it for data starting 2004, but I still need to have it as proof for data generation).

Thanks again,
Lior

[Nathanlauga]

I generated the orignals csv 2 years ago I think so I don't think the originals scripts work, but I used get_games.py for games.csv generation and get_games_stats.py for game details.

But if you want to generate it from scratch you can use and adapt the scrap code from get_new_games.py

The NBA stats website change over time so this is why old scripts are not working

[YakobovLior]

I understand.
I'll try to make adjustments and make the script file get_new_games.csv to be independent, meaning as it can run and collect all games since 2004 like the current CSV file, with the same columns (of course I won't collect all data again, it but need to have the ability to at least run it for several rows).

Thank you for the help,
Lior

[YakobovLior]

Hey @Nathanlauga,
I have some other questions and would like for your help here.

How did you found about the API endpoint and the different requests available?

I couldn't find in their website any information for API usage, so I wonder how did you found out about the available requests, for exmaple:

• https://stats.nba.com/stats/commonteamyears?LeagueID=00
• https://stats.nba.com/stats/teamdetails?TeamID=

Also, from time to time seems that I'm unable to connect to the endpoint and fetch data.
I have updated my HTTP headers as described in this issue, and it does work sometimes, but most of the time I get this error:

Traceback (most recent call last):
  File "scripts\get_teams.py", line 77, in <module>
    main()
  File "scripts\get_teams.py", line 69, in main
    list_team_ids = get_team_ids_list()
  File "scripts\get_teams.py", line 19, in get_team_ids_list
    response = requests.get(url, headers=HEADERS)
  File "C:\Users\liorya\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\liorya\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\liorya\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\liorya\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\liorya\AppData\Local\Programs\Python\Python37\lib\site-packages\requests\adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError("(10054, 'WSAECONNRESET')"))

Have you ever faced this issue as well?

Thanks in advance,
Lior

[Nathanlauga]

Sorry for the delay of my answer, I don't recall having this error.

Did you try to use headers in the get_new_games.py file ?

[YakobovLior]

Hey @Nathanlauga,

Thanks for the response, I eventually managed to figure it out.

Lior