Review FLIR fullfield extractor and output

[max-zilla]

Review issue for the fullfield extractor and output using flir thermal IR inputs.

Completion criteria:

• Code review complete (includes extractor, any functions in terrautils, etc)
• Sample data output verified
• Sample metadata verified
• Changes for 2017 release implemented
• Issue(s) for changes postponed until 2018 created
• Extractor tagged/versioned
• Extractor deployed
• Quality statement complete
• Final output verified/reviewed

[max-zilla]

LINKS

fullfield output dataset example
https://terraref.ncsa.illinois.edu/clowder/datasets/59b0e2134f0ca12ea0ce678a

an output dataset includes thumbnail- and full-resolution GeoTIFFs.

Ignore that this linked dataset has thousands of files, this is a glitch in extractor upload code. The top two .tif files are just duplicated endlessly so just look at the first ones listed.

outputs on ROGER are in /ua-mac/Level_1/fullfield.

[dlebauer]

One of the full field IR stitched images looks like this:

This appears to show very large artifacts (at boundaries, the transitions are not related to the plants)
With values ranging from 1221 to 1574. If this is temperature (K), values should mostly be in the range 275-300

[dlebauer]

A lower priority issue is that these show up blank in the Clowder interface:

[ZongyangLi]

@max-zilla @dlebauer
In the flir extractor code here: https://github.com/terraref/extractors-multispectral/blob/master/flir2tif/Get_FLIR.py#L174
Just make a very simple update that from
if not calibP.calibrated: to if calibP.calibrated:

And then the extractor will output a degree centigrade in geotiff files. That's a misunderstanding of the parameter in metadata of "is calibrated".

If we are looking for an output of Kelvin temperature we should delete the Kelvin parameter 273.15.

[ZongyangLi]

@max-zilla
Please use the updated codes here: https://github.com/terraref/extractors-multispectral/blob/master/flir2tif/Get_FLIR.py

[dlebauer]

The variable to be inserted into BETYdb is surface_temperature with units of K. See https://terraref.ncsa.illinois.edu/bety/variables/6000000198 or https://terraref.ncsa.illinois.edu/bety/api/beta/variables?key=9999999999999999999999999999999999999999&name=surface_temperature

[dlebauer]

Questions about the extractor https://github.com/terraref/extractors-multispectral/blob/master/flir2tif/Get_FLIR.py

• there are many magic numbers that appear without comment,
  • examples: gmin, gmax, H, T, D, E, HEIGHT_MAGIC_NUMBER
• there are many functions that appear to belong in (and / or are redundant with) terrautils
  • examples: createVrt, createMapTiles, generate_googlemaps, parse_metadata, get_bounding_box, get_bounding_box_with_formula, create_geotiff
  • are these all currently used in the pipeline?
• Line 226 says "TODO Something wonky w/ uint8s --> ending up w/ lots of gaps in data (white pixels)". Can this be fixed now? If not, please add to the quality statement and create a new github issue to track this.