Improve data source downloading

[sjpfenninger]

What can be improved?

Rather than having the curl command repeated throughout the rules, it would probably be better to wrap it in one place.

This helper could optionally print a simple log message like "Downloading from {url} to {output}" which would help understanding and debugging.

Could further also improve the specification of data sources to e.g. check md5 hash to add protection against the source file having changed, and to optionally specify what dimensions and variables are expected in downloaded netCDF files, such as:

gridded-temperature-data:
    url: https://zenodo.org/records/6557643/files/temperature.nc?download=1
    output_file: data/automatic/gridded-weather/temperature.nc
    md5: 5ee6d0152f174c56d42b0a144c13c822
    nc_dims: time, site
    nc_vars: temperature, lat, lon

Version

1.2.0.dev