Add --date parameter to load_bigquery_stats job - #5369
Conversation
dylanjew
left a comment
There was a problem hiding this comment.
nice! Mostly LGTM. Just one nit.
I think you can go ahead and test this on dev by running the cron job with butler. You may need to verify that the args are passed in correctly, I remember that being a bit annoying when trying to run the cron
vitaliset
left a comment
There was a problem hiding this comment.
LGTM.
I think you can go ahead and test this on dev by running the cron job with butler.
+1. Please provide some evidence that it worked on dev before merging! :)
| def test_execute(self): | ||
| """Tests executing of cron job.""" | ||
| load_bigquery_stats.main() | ||
| self._assert_load_calls('20160907') |
There was a problem hiding this comment.
nit: i would add a comment or use a variable to help make it clear that this is one day before the mock in the test
There was a problem hiding this comment.
Thanks for the feedback.
I have added a variable (expected_yesterday) and a comment to clarify the unit tests.
689334d to
79b492a
Compare


This PR updates
load_bigquery_stats.pyto accept an optional--dateflag (inYYYY-MM-DDformat), allowing cluster operators and developers to execute the BigQuery stats loading job against an arbitrary target date (useful for backfilling or re-processing past stats).Key Highlights
--dateFlag: Supports loading stats for a specified date (e.g.--date 2026-06-20)._utc_now().date() - datetime.timedelta(days=1)), maintaining standard daily cron execution behavior.aggregate_fuzzer_stats.py.test_execute_with_dateto verify that target partition tables ($YYYYMMDD) and GCS source URIs (/date/YYYYMMDD/) are correctly generated when a custom date is provided.Testing Instructions