Rescaling TOA

[virginiayung]

@celoyd and I voiced on the rescaling factor and output dtype. We think that the default should be keeping post-TOA tifs as float (currently float32). This allows faster reading and minimum information lost when we work on post-TOA processes like atmospheric correction in the future. We can add a new cli option to output rescaled 16-bit.

Why are we rescaling?

TOA is the ratio of reflected radiation from the surface to incident radiation upon it. Its dimensionless nature lets it be expressed as a percentage and is measured on a scale from zero for no reflection of a perfectly black surface to 1 for perfect reflection of a white surface. We need to rescale TOA from this ratio back to 16-bit(or 8-bit). Note that we need to rescale post-TOA tifs to 55,000 instead of 2¹⁶ because USGS Landsat 8 products are delivered as 16-bit images but are actually scaled to 55,000 grey levels:

Here are the steps:

  toa = ((dn * Mr) + Ar) / sin(SE)
  scale_factor = 55000
  toa = (toa * scale_factor)
  if dst_dtype == 'uint16':
      toa = np.clip(toa, 1, np.iinfo(np.uint16).max).astype(np.uint16)

@perrygeo How about letting rio-color handle the 16 to 8-bit conversion?

cc @dnomadb @celoyd @perrygeo

[perrygeo]

Yes, rio color can do the scaling. In most cases this is how we'll want to do it: rio toa -> 16 bit -> rio color -> 8 bit. I think scaling to uint16 is a great idea from a data fideilty perspective and should be the default.

But there are also cases where we'll need 8bit for pragmatic reasons. Both of these are limitations of GDAL's implementation but limitations nonetheless.

• jpeg compression does not work with 16-bit bands
• alpha band interpretation is lost with 16-bit bands. GDAL treats band 4 like normal data rather than use it as a mask when warping.

In auto-landsat we have some intermediate steps that might be affected by the alpha band interpretation issue. So uint8 might be necessary.

I think we should have a dtype option that defaults to uint16 but also allows for uint8 or float32.

related note: when we implement this, make sure to expose the data rescaling function in the python API as well as in the worker/cli.

[virginiayung]

Completed rescaling: 3679f5d

dtype option that defaults to float32, but also allows for uint8 and uint16