Interface glmtools to get gridded products from GLM LCFA files on-the-fly.

[gerritholl]

Feature Request

Is your feature request related to a problem? Please describe.

It is currently somewhat cumbersome to use GLM gridded products. Users need to download GLM LCFA data, run it through glmtools to generate gridded files, and read those with satpy. This makes any automated processes more work.

Describe the solution you'd like

It would be nice if satpy had an interface to glmtools such that the user could pass a GLM LCFA file (including file system support to get this directly from a remote interface, see #1321) and obtain gridded products from there. The gridded products would be calculated on-the-fly, lazily, with dask, only for the area requested. Ideally this would also be cached on disk.

Describe any changes to existing user workflow

User experience improves. No backward compatibility problems foreseen.

Additional context

Rumour has it NOAA are planning to release gridded GLM products (presumably based on the process glmtools uses) in the future, although I don't know what future that may be. If true, that would make this issue superfluous.

[djhoese]

There is a meeting next week that I am going to try to attend about the future of gridded GLM (potentially a CSPP package wrapping glmtools). Better Satpy support could be part of that.

The intended way of doing this based on the skeleton code/yaml in the existing glm reader is to allow for LCFA file patterns and run glmtools in the background if provided the LCFA files. The difficulty may be in your request to fully dask delay that operation. Glmtols would have to be deeply integrated in that behavior so that things like the resulting AreaDefinition could be known before hand. I'm also not sure if it is currently possible to request one specific variable from glmtools.

[ghiggi]

@gerritholl @djhoese are there some advances in the generation of GLM gridded products with satpy, or I need to use glmtools?

[gerritholl]

The feature request here is not to replicate the functionality of glmtools in satpy, but rather to make the user experience from a satpy perspective smoother. I have not thought it detail about this for a while, though.

[djhoese]

As Gerrit said, the plan/hope is to incorporate the calling of glmtools into satpy since it is a python package and is used by people in our community. There is a placeholder file handler in the glm_l2 reader where Eric Bruning the creator of glmtools or me or someone else hope to add the code to run glmtools on the fly from an LCFA file.

satpy/satpy/readers/glm_l2.py

Line 41 in f34d563

# class NC_GLM_L2_LCFA(BaseFileHandler): — add this with glmtools

That work is not done yet so you will need to use glmtools externally and provide the resulting files to satpy's glm_l2 reader.

[ghiggi]

Thanks for the info !