A dynamic view of GCS dataset acquisition

[naomi-henderson]

I have been working on a quick-and-dirty way to make our dataset collection process more transparent. Here is an experimental google sheet which is updated automatically from the scripts (which are still running on local resources - ugh - four jobs in parallel).

The top sheet is a list of on-going processes, with two additional sheets to define the terms and random issues.

A new request came in this morning for 3 hourly data. This will take quite a long time, so I decided to share the page to keep the requestor in the loop (and help explain why it will take a while). I have paused our usual background jobs and will use this to see what problems are encountered in the 3 hourly datasets!

Suggestions welcome! I am trying to work within the google sheet updating quotas - a bit of a nuisance ...