toa_bidirectional_reflectance variable allow "float" or "int" NetCDF output instead of "double"

[Steve-Albers]

To help save disk space with Himawari visible full disk data in NetCDF, can the output have an option to be written as "float" or "int" variables instead of "float"? This could be a keyword passed into 'save_datasets'.

Here is an example scene writing

# Band 03

filenames = glob("/users/sa/data/sat/him8_aws/*_B03_*.bz2")
global_scene = Scene(reader="ahi_hsd", filenames=filenames)
print(global_scene.available_dataset_names())
global_scene.load(['B03'])
band_vis = global_scene["B03"]
#print(global_scene)
print(band_vis)

# https://satpy.readthedocs.io/en/stable/api/satpy.writers.html#module-satpy.writers.cf_writer
# global_scene.save_datasets(writer='simple_image')

global_scene.save_datasets(writer='cf',engine='netcdf4',exclude_attrs=['raw_metadata'],include_lonlats=False)

[Steve-Albers]

"short" may save even more space for the reflectance in percent

[djhoese]

I did not write the cf writer so I don't know if this will work, but you may be able cheat by providing an encoding kwarg to save_datasets. See the xarray documentation for more information on it: http://xarray.pydata.org/en/stable/generated/xarray.Dataset.to_netcdf.html

Otherwise, you could try:

encoding = {
    "B03": {
        "dtype": "uint16",
        "_FillValue": 65535,
        "scale_factor": # some scale factor float,
        "add_offset": # some add offset float,
    }
}

global_scene.save_datasets(writer='cf',engine='netcdf4',exclude_attrs=['raw_metadata'],include_lonlats=False, encoding=encoding)

[Steve-Albers]

Thanks - I'll try this. I was wondering how the kwarg syntax might look if it's all in one line. Good to know that 'encoding' can be set ahead of time in a variable.

In a related note, I wonder what would the syntax look like to utilize NetCDF4 compression within the kwarg?

[djhoese]

The writer in Satpy has a compression keyword argument here: https://satpy.readthedocs.io/en/stable/api/satpy.writers.html#satpy.writers.cf_writer.CFWriter.save_datasets (that can be passed to save_datasets from the Scene). Otherwise, the encoding kwarg from xarray talks about how compression can be specified (see previous link to xarray docs).

[Steve-Albers]

Follow up discussion can be found in a newer issue here: #1866