Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add snapshotting to allow comparison between points and time. #101

Closed
matsonj opened this issue Aug 17, 2023 · 4 comments
Closed

Add snapshotting to allow comparison between points and time. #101

matsonj opened this issue Aug 17, 2023 · 4 comments

Comments

@matsonj
Copy link
Owner

matsonj commented Aug 17, 2023

Since ELO rating is re-forecasted after every game, it would be interesting to see how predicted outcomes change over time.

The chart below from 538 is an example of the kind of visual that is possible if we have snapshotting enabled.

image

It should be noted that this is where persistence comes into play since current models are NOT persisted (by design).

@matsonj
Copy link
Owner Author

matsonj commented Sep 4, 2023

I think I can do this a slightly coarser grain - which is weekly for football sims and daily for nba. The basic way to do this will be hold some folders in s3, and then based on when the most recent actual is recorded, we can use write the new results into a folder.

something like s3://some_bucket/sim_version/year/week/model.parquet, or for the nfl model it would be currently this path: s3://some_bucket/nfl/2023/0/nfl_reg_season_summary.parquet

@matsonj
Copy link
Owner Author

matsonj commented Sep 4, 2023

Pipeline then basically has a final step of dropping new files into S3, then building a duckdb file that can hit the entire directory. A thin copy of the parquet files. Then evidence hits that duckdb file instead of the local file.

@matsonj
Copy link
Owner Author

matsonj commented Sep 5, 2023

related to this conversation here: #102

@matsonj
Copy link
Owner Author

matsonj commented Sep 8, 2023

Actually this is a dupe of #102 and will be solved as part of that.

@matsonj matsonj closed this as not planned Won't fix, can't repro, duplicate, stale Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant