New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rescaling TOA #10

Closed
virginiayung opened this Issue Jun 28, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@virginiayung
Contributor

virginiayung commented Jun 28, 2016

@celoyd and I voiced on the rescaling factor and output dtype. We think that the default should be keeping post-TOA tifs as float (currently float32). This allows faster reading and minimum information lost when we work on post-TOA processes like atmospheric correction in the future. We can add a new cli option to output rescaled 16-bit.

Why are we rescaling?

TOA is the ratio of reflected radiation from the surface to incident radiation upon it. Its dimensionless nature lets it be expressed as a percentage and is measured on a scale from zero for no reflection of a perfectly black surface to 1 for perfect reflection of a white surface. We need to rescale TOA from this ratio back to 16-bit(or 8-bit). Note that we need to rescale post-TOA tifs to 55,000 instead of 216 because USGS Landsat 8 products are delivered as 16-bit images but are actually scaled to 55,000 grey levels:

Here are the steps:
  toa = ((dn * Mr) + Ar) / sin(SE)
  scale_factor = 55000
  toa = (toa * scale_factor)
  if dst_dtype == 'uint16':
      toa = np.clip(toa, 1, np.iinfo(np.uint16).max).astype(np.uint16)

@perrygeo How about letting rio-color handle the 16 to 8-bit conversion?

rescaling_toa

cc @dnomadb @celoyd @perrygeo

@perrygeo

This comment has been minimized.

Contributor

perrygeo commented Jun 28, 2016

Yes, rio color can do the scaling. In most cases this is how we'll want to do it: rio toa -> 16 bit -> rio color -> 8 bit. I think scaling to uint16 is a great idea from a data fideilty perspective and should be the default.

But there are also cases where we'll need 8bit for pragmatic reasons. Both of these are limitations of GDAL's implementation but limitations nonetheless.

  • jpeg compression does not work with 16-bit bands
  • alpha band interpretation is lost with 16-bit bands. GDAL treats band 4 like normal data rather than use it as a mask when warping.

In auto-landsat we have some intermediate steps that might be affected by the alpha band interpretation issue. So uint8 might be necessary.

I think we should have a dtype option that defaults to uint16 but also allows for uint8 or float32.

related note: when we implement this, make sure to expose the data rescaling function in the python API as well as in the worker/cli.

@virginiayung

This comment has been minimized.

Contributor

virginiayung commented Jul 1, 2016

Completed rescaling: 3679f5d

dtype option that defaults to float32, but also allows for uint8 and uint16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment