Add "--fields" option for glance info

[adesmet-ssec]

--fields is a comma seperated list of fields to be output. By default, it's "filename,variable", which is the old output. You can use any of:

• filename
• variable
• shape - data shape of the variable
• ATTRIBUTE_NAME - any attribute associated with the variable. If missing, will be an empty string.
• stats(STAT_NAME) - any statistic as seen in glance inspectStats. The names are exactly as shown there, so "stats(min)" and "stats(finite_count)" are options.

I'd regularly wanted "shape" and the attribute "long_name" for use while exploring new data. A need to create a report of all of our output data drove generalizing the problem and adding stats(STAT_NAME).

This includes and completely replaces pull request #12

[evas-ssec]

This seems reasonable. Could you give me a couple of simple examples of using the syntax you designed?

[adesmet-ssec]

filename,variable is identical to using glance info without any --fields option at all.

$ glance info FILE.nc
FILE.nc: Latitude
         Longitude
         SensorAzimuth
         SensorZenith
         SolarAzimuth
         SolarZenith
         SpaceMask
$ glance info --fields filename,variable FILE.nc
FILE.nc: Latitude
         Longitude
         SensorAzimuth
         SensorZenith
         SolarAzimuth
         SolarZenith
         SpaceMask

If you aren't using --parsable, it will insert filename at the front if not already present. This is a side effect of striving for maximum backward compatibility.

$ glance info --fields long_name,filename FILE.nc 
FILE.nc: Latitude FILE.nc
         Longitude FILE.nc
         Sensor azimuth angle FILE.nc
         Sensor zenith angle FILE.nc
         Solar azimuth angle FILE.nc
         Solar zenith angle FILE.nc
         Space Mask FILE.nc

I use this command when exploring new files. shape is calculated by glance. units and long_name attributes associated with each variable.

$ glance info --parsable --fields variable,shape,units,long_name FILE.nc 
Latitude        (5424, 5424)    degrees_north   Latitude
Longitude       (5424, 5424)    degrees_east    Longitude
SensorAzimuth   (5424, 5424)    degrees Sensor azimuth angle
SensorZenith    (5424, 5424)    degrees Sensor zenith angle
SolarAzimuth    (5424, 5424)    degrees Solar azimuth angle
SolarZenith     (5424, 5424)    degrees Solar zenith angle
SpaceMask       (5424, 5424)    1       Space Mask

For a recent report, I needed a pile of the calculated statistics. The names are the same names that appear in the statistics reports. Here's a simple example:

$ glance info --parsable --fields 'variable,stats(min),stats(max)' FILE.nc 
Latitude        -81.1488        81.1469
Longitude       -156.195        6.19885
SensorAzimuth   0.0190026       359.981
SensorZenith    0.0151067       90.0
SolarAzimuth    0.000601057     360.0
SolarZenith     22.2221 179.96
SpaceMask       0.0     1.0

Here's the actual command I ran:

glance info --parsable --fields "filename,variable,long_name,units,shape,"\
"stats(min),stats(max),stats(mean),stats(std_val),stats(finite_count),"\
"stats(finite_fraction),stats(missing_count),stats(missing_fraction)" \
*.[nN][Cc] > variable-report.tsv

[evas-ssec]

Please add at least a minimal descriptive comment for the gather_stats_for_variable and info_impl methods. Also, if you could name the info_impl method something slightly more descriptive that would be easier to read.

Please change your option entry in the option parser for “—fields” to give information about the fields a user can ask for. Right now it’s very hard for a user to figure out how to use this feature without reading the code comments.

[evas-ssec]

(It might actually be best to include description or examples of how to use --fields in the comment that is printed for the info command when you ask for "glance help info".)

[adesmet-ssec]

• Docstring comments added to functions.
• info_impl renamed to info_library_call, which may not be more descriptive, but matches the conventions used elsewhere in the file
• Added summary of things you can put in --fields to "glance --help"
• Added explanation and examples for --fields to "glance help info"

I believe this addresses all of your requests.

[adesmet-ssec]

Missed the "shape" field; added.

[adesmet-ssec]

Migrated to https://gitlab.ssec.wisc.edu/evas/UW-Glance/merge_requests/4