Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RGBs should never have units, but some do #2066

Closed
gerritholl opened this issue Mar 21, 2022 · 5 comments · Fixed by #2068
Closed

RGBs should never have units, but some do #2066

gerritholl opened this issue Mar 21, 2022 · 5 comments · Fixed by #2068

Comments

@gerritholl
Copy link
Collaborator

gerritholl commented Mar 21, 2022

Describe the bug

When loading the snow_age RGB with the viirs_sdr reader, the resulting RGB has the attribute calibration set to reflectance and the attribute units set to %. This is incorrect.

It also leads to a failure in the ninjotiff writer, which unsuccessfully attempts a unit conversion, resulting in a NotImplementedError.

To Reproduce

This code checks the value for calibration and units for all loadable composites:

from satpy import Scene
from glob import glob
from satpy.utils import debug_on; debug_on()
filenames = glob("/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/*.h5")
sc = Scene(filenames=filenames, reader=["viirs_sdr"])
names = sc.available_composite_names()
sc.load(names)
ls = sc.resample("eurol")
for name in names:
    print(name, ls[name].attrs.get("calibration"), ls[name].attrs.get("units"))

Expected behavior

I expect that an RGB should never have calibration or units attributes set.

The DNB composites are a different issue; see #1985.

Actual results

The following composites unexpectedly have calibration and units attributes defined:

  • false_color
  • natural_color
  • ocean_color
  • snow_age
  • true_color
  • true_color_crefl

The DNB composites have those set as well, but that is more debatable, as the output is still mode L, so one could argue units are still valid. See #1985 for a separate issue on this.

The composite ssec_fog also has units set, but this is a single-channel output so could be correct (the values I get range from -80 to +15, with the metadata claiming calibration in brightness temperature with units of K, so this is also wrong, but in a different way than for the RGBs).

There may be other RGBs that unexpectedly have units; this lists just the ones loadable with VIIRS.

I cannot include the full output here (GitHub error: comment is too long), therefore an excerpt only.

output of MCVE
[DEBUG: 2022-03-21 18:03:25 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_sdr.yaml',)
[DEBUG: 2022-03-21 18:03:25 : satpy.readers.yaml_reader] Assigning to viirs_sdr: ['/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1415329_e1416556_b21597_c20220118142622635962_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM10_j01_d20220118_t1409498_e1411144_b21597_c20220118142629439978_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM08_j01_d20220118_t1415329_e1416556_b21597_c20220118142627590076_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM12_j01_d20220118_t1418226_e1419471_b21597_c20220118144233408674_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM11_j01_d20220118_t1416568_e1418214_b21597_c20220118144234413570_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM14_j01_d20220118_t1412414_e1414059_b21597_c20220118142611797034_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVDNB_j01_d20220118_t1408259_e1409486_b21597_c20220118142541129075_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM15_j01_d20220118_t1415329_e1416556_b21597_c20220118142628981472_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/IVCDB_j01_d20220118_t1412414_e1414059_b21597_c20220118142606308719_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI01_j01_d20220118_t1414071_e1415316_b21597_c20220118142622522970_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM09_j01_d20220118_t1408259_e1409486_b21597_c20220118143845056485_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM02_j01_d20220118_t1411156_e1412401_b21597_c20220118142621799572_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM03_j01_d20220118_t1412414_e1414059_b21597_c20220118143857734314_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM06_j01_d20220118_t1414071_e1415316_b21597_c20220118142628059985_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM11_j01_d20220118_t1414071_e1415316_b21597_c20220118142629226258_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM14_j01_d20220118_t1409498_e1411144_b21597_c20220118142630214980_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVDNB_j01_d20220118_t1418226_e1419471_b21597_c20220118142930329859_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1415329_e1416556_b21597_c20220118142501513194_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1415329_e1416556_b21597_c20220118143750684569_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI04_j01_d20220118_t1411156_e1412401_b21597_c20220118143912427305_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM04_j01_d20220118_t1418226_e1419471_b21597_c20220118142933269188_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM07_j01_d20220118_t1412414_e1414059_b21597_c20220118142613387204_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1416568_e1418214_b21597_c20220118142923991551_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI02_j01_d20220118_t1414071_e1415316_b21597_c20220118143842970748_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM06_j01_d20220118_t1412414_e1414059_b21597_c20220118142610237381_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM11_j01_d20220118_t1408259_e1409486_b21597_c20220118142549474743_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1408259_e1409486_b21597_c20220118143840908641_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM14_j01_d20220118_t1408259_e1409486_b21597_c20220118142550123499_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1416568_e1418214_b21597_c20220118144231563855_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1408259_e1409486_b21597_c20220118142549811826_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1409498_e1411144_b21597_c20220118142625483960_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM03_j01_d20220118_t1418226_e1419471_b21597_c20220118144231558405_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM03_j01_d20220118_t1414071_e1415316_b21597_c20220118143845676842_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM09_j01_d20220118_t1415329_e1416556_b21597_c20220118142627837088_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1418226_e1419471_b21597_c20220118144152385206_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1409498_e1411144_b21597_c20220118143906672879_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM01_j01_d20220118_t1415329_e1416556_b21597_c20220118142625208054_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI01_j01_d20220118_t1415329_e1416556_b21597_c20220118143908975041_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM06_j01_d20220118_t1416568_e1418214_b21597_c20220118144233612332_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM05_j01_d20220118_t1416568_e1418214_b21597_c20220118144233432126_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM14_j01_d20220118_t1415329_e1416556_b21597_c20220118143915473205_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI02_j01_d20220118_t1415329_e1416556_b21597_c20220118142622155625_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM02_j01_d20220118_t1412414_e1414059_b21597_c20220118143857521661_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM05_j01_d20220118_t1414071_e1415316_b21597_c20220118142627838424_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM01_j01_d20220118_t1415329_e1416556_b21597_c20220118143911627687_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM16_j01_d20220118_t1412414_e1414059_b21597_c20220118142612153651_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM09_j01_d20220118_t1414071_e1415316_b21597_c20220118143847270292_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM08_j01_d20220118_t1416568_e1418214_b21597_c20220118144233958498_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM11_j01_d20220118_t1409498_e1411144_b21597_c20220118143906378965_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM12_j01_d20220118_t1411156_e1412401_b21597_c20220118143916454227_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM05_j01_d20220118_t1414071_e1415316_b21597_c20220118143846252342_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/IVCDB_j01_d20220118_t1414071_e1415316_b21597_c20220118143841234577_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI01_j01_d20220118_t1416568_e1418214_b21597_c20220118144230853130_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI04_j01_d20220118_t1414071_e1415316_b21597_c20220118143844012328_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI05_j01_d20220118_t1411156_e1412401_b21597_c20220118142620887436_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM14_j01_d20220118_t1408259_e1409486_b21597_c20220118143846517368_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1411156_e1412401_b21597_c20220118143916565789_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1412414_e1414059_b21597_c20220118143855718433_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1408259_e1409486_b21597_c20220118143751232547_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/IVCDB_j01_d20220118_t1415329_e1416556_b21597_c20220118143907674949_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM01_j01_d20220118_t1418226_e1419471_b21597_c20220118144231243634_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1412414_e1414059_b21597_c20220118143735221365_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM12_j01_d20220118_t1416568_e1418214_b21597_c20220118144234510990_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM16_j01_d20220118_t1409498_e1411144_b21597_c20220118142630567664_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM08_j01_d20220118_t1411156_e1412401_b21597_c20220118143915809239_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI04_j01_d20220118_t1409498_e1411144_b21597_c20220118143903443205_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1409498_e1411144_b21597_c20220118142629972496_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/IVCDB_j01_d20220118_t1409498_e1411144_b21597_c20220118142622618319_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVDNB_j01_d20220118_t1412414_e1414059_b21597_c20220118142606137765_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM12_j01_d20220118_t1415329_e1416556_b21597_c20220118142628459561_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI02_j01_d20220118_t1409498_e1411144_b21597_c20220118143902366929_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GMTCO_j01_d20220118_t1409498_e1411144_b21597_c20220118142458195001_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVDNB_j01_d20220118_t1411156_e1412401_b21597_c20220118142617271546_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1414071_e1415316_b21597_c20220118142629545376_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM08_j01_d20220118_t1416568_e1418214_b21597_c20220118142927966813_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM07_j01_d20220118_t1409498_e1411144_b21597_c20220118143908979028_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM13_j01_d20220118_t1418226_e1419471_b21597_c20220118144233504245_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM15_j01_d20220118_t1409498_e1411144_b21597_c20220118142630388449_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI04_j01_d20220118_t1415329_e1416556_b21597_c20220118142623884588_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GITCO_j01_d20220118_t1409498_e1411144_b21597_c20220118142449366198_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM04_j01_d20220118_t1409498_e1411144_b21597_c20220118143905023420_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM10_j01_d20220118_t1415329_e1416556_b21597_c20220118143913514566_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/GITCO_j01_d20220118_t1414071_e1415316_b21597_c20220118142446916835_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI03_j01_d20220118_t1409498_e1411144_b21597_c20220118143902838280_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM04_j01_d20220118_t1412414_e1414059_b21597_c20220118143857947485_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVM16_j01_d20220118_t1416568_e1418214_b21597_c20220118142929410057_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596/SVI02_j01_d20220118_t1411156_e1412401_b21597_c20220118142618629055_cspp_dev.h5', '/media/nas/x21308/scratch/VIIRS/noaa20_20220118_1408_21596
...
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-d1685565d06a73f47dd8998b1c59bfc2
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='M10', wavelength=WavelengthRange(min=1.58, central=1.61, max=1.64, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-a19032323202df4080fdc9a4de85e819
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='M11', wavelength=WavelengthRange(min=2.225, central=2.25, max=2.275, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-8b399bcddf82ca119067afc796defc52
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='M12', wavelength=WavelengthRange(min=3.61, central=3.7, max=3.79, unit='µm'), resolution=742, calibration=<calibration.brightness_temperature>, modifiers=())
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-0d8b15dbebbd99f95789e9438b20406c
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='M15', wavelength=WavelengthRange(min=10.263, central=10.763, max=11.263, unit='µm'), resolution=742, calibration=<calibration.brightness_temperature>, modifiers=())
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-7ed74f5d7d017ca38b69d13f976a1129
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='adaptive_dnb', resolution=743)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling _apply_factors-213648d8c43763e0c868876788286243
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='ash', resolution=742)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling where-f649e83ab08109d29dbce22d1d7a8446
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='cloudtop', resolution=371)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling where-863c803e42e2eb2e649dce33d255124f
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='cloudtop_daytime', resolution=742)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling where-a5b71b3b77a332fd0ecc1c302de1c6c1
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='dust', resolution=742)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:57 : satpy.resample] Resampling where-f649e83ab08109d29dbce22d1d7a8446
[DEBUG: 2022-03-21 18:04:57 : satpy.scene] Resampling DataID(name='dynamic_dnb', resolution=743)
[INFO: 2022-03-21 18:04:57 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling _apply_factors-213648d8c43763e0c868876788286243
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='fire_temperature', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-9fb94a7096052b6b3eea45ce8d65a1d7
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='fire_temperature_39refl', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-ec93185694e8454a787aa9d091013bb4
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='fire_temperature_awips', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-656be685b097431ab0eb535ecfd9423f
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='fire_temperature_eumetsat', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-656be685b097431ab0eb535ecfd9423f
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='fog', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-f649e83ab08109d29dbce22d1d7a8446
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='green_snow', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-8cf43eea868e39b03f03d08b08a29728
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='histogram_dnb', resolution=743)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling _apply_factors-213648d8c43763e0c868876788286243
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='hncc_dnb', wavelength=WavelengthRange(min=0.5, central=0.7, max=0.9, unit='µm'), resolution=743, calibration=<calibration.radiance>, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling _apply_factors-213648d8c43763e0c868876788286243
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='hr_cloudtop_daytime', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-47c762abc5ee629aa3ad98a653fd7af9
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='hr_overview', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-82c34dede33c8bc63cfeee5ad0ca8a36
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='ir108_3d', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling concatenate-8b5b7bc43a2698019e6febf61a78ef3c
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='ir_cloud_day', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling concatenate-22b4c49f0dbb59feff80a27ad3166c47
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='natural_color_raw', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-f5836d10445949bbc4d4b4e097f1b0b3
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='natural_color_sun', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-f5836d10445949bbc4d4b4e097f1b0b3
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='natural_color_sun_lowres', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-ee9c5de10e3f2608fb606eec579bfb0e
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='night_fog', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-dd0a24b94d20e5450b55dbc8746a1a75
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='overview', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-eba6966f19027aaf99539a852545a5f3
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='satellite_azimuth_angle', resolution=742, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling array-85dc64b1bfd3bb0b07397cf876197804
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='satellite_zenith_angle', resolution=742, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling array-7fe059a065aa20e40c7863cd3aeffbb1
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='snow', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-dcf97211f792e95687604a1ef613d6ef
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='snow_age', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-318236ed44b8766f50ec7e4b0602778c
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='snow_hires', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-dcf97211f792e95687604a1ef613d6ef
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='snow_lowres', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-f1aa870f7cb9398e19b4d381c8445360
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='solar_azimuth_angle', resolution=742, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling array-83dd0d1f4f782951a1de73b3e589262b
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='solar_zenith_angle', resolution=371, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling array-31b2a446d129ead0c15759f84958c291
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='solar_zenith_angle', resolution=742, modifiers=())
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling array-f4c1fff3e47b1389622b22fa298e7541
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='ssec_fog', resolution=371)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling None
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='true_color_lowres', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-c5778f439e0ba85d143502abc8c34c9b
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='true_color_lowres_crefl', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-429a73610a7552a450809bef6337f01d
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='true_color_lowres_land', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-15203cb257b0f9854d35e6e3394ca896
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='true_color_lowres_marine_tropical', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-59e79f1fee35fc3e1414395c8c9ae0a3
[DEBUG: 2022-03-21 18:04:58 : satpy.scene] Resampling DataID(name='true_color_raw', resolution=742)
[INFO: 2022-03-21 18:04:58 : satpy.scene] Not reducing data before resampling.
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2022-03-21 18:04:58 : satpy.resample] Resampling where-14fee1032098a62713e1cc0153cb852d
[DEBUG: 2022-03-21 18:04:58 : satpy.modifiers.geometry] Applying sun zen correction
[DEBUG: 2022-03-21 18:04:58 : satpy.modifiers.geometry] Computing sun zenith angles.
[DEBUG: 2022-03-21 18:04:58 : satpy.modifiers.geometry] Apply the standard sun-zenith correction [1/cos(sunz)]
[DEBUG: 2022-03-21 18:04:58 : satpy.composites] Sharpening image with high resolution red band
[DEBUG: 2022-03-21 18:04:58 : satpy.composites] Sharpening image with high resolution red band
[DEBUG: 2022-03-21 18:04:59 : satpy.composites] Sharpening image with high resolution red band
[DEBUG: 2022-03-21 18:04:59 : satpy.composites] Sharpening image with high resolution blue band
[DEBUG: 2022-03-21 18:04:59 : satpy.writers] Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f7ff43efe20>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0, 'max_stretch': 120}}, {'name': 'gamma', 'method': <function gamma at 0x7f7ff4f3d7e0>, 'kwargs': {'gamma': 1.8}}]
[DEBUG: 2022-03-21 18:04:59 : trollimage.xrimage] Applying stretch crude with parameters {'min_stretch': 0, 'max_stretch': 120}
[DEBUG: 2022-03-21 18:04:59 : trollimage.xrimage] Applying gamma 1.8
[DEBUG: 2022-03-21 18:04:59 : satpy.writers] Enhancement configuration options: [{'name': 'stretch', 'method': <function stretch at 0x7f7ff43efe20>, 'kwargs': {'stretch': 'crude', 'min_stretch': [-4, 0, 243], 'max_stretch': [2, 6, 293]}}, {'name': 'gamma', 'method': <function gamma at 0x7f7ff4f3d7e0>, 'kwargs': {'gamma': [1, 2, 1]}}]
[DEBUG: 2022-03-21 18:04:59 : trollimage.xrimage] Applying stretch crude with parameters {'min_stretch': [-4, 0, 243], 'max_stretch': [2, 6, 293]}
[DEBUG: 2022-03-21 18:04:59 : trollimage.xrimage] Applying gamma <xarray.DataArray (bands: 3)>
array([1, 2, 1])
Coordinates:
  * bands    (bands) <U1 'R' 'G' 'B'
[DEBUG: 2022-03-21 18:04:59 : satpy.composites] Sharpening image with high resolution blue band
[INFO: 2022-03-21 18:04:59 : satpy.modifiers.atmosphere] Removing Rayleigh scattering with atmosphere 'us-standard' and aerosol type 'rayleigh_only' for 'I01'
[INFO: 2022-03-21 18:04:59 : pyspectral.rayleigh] Atmosphere chosen: us-standard
[DEBUG: 2022-03-21 18:04:59 : pyspectral.rayleigh] LUT filename: /home/gholl/.local/share/pyspectral/rayleigh_only/rayleigh_lut_us-standard.h5
[DEBUG: 2022-03-21 18:04:59 : pyspectral.rsr_reader] Filename: /home/gholl/.local/share/pyspectral/rsr_viirs_NOAA-20.h5
[DEBUG: 2022-03-21 18:04:59 : pyspectral.rsr_reader] Filename: /home/gholl/.local/share/pyspectral/rsr_viirs_NOAA-20.h5
[DEBUG: 2022-03-21 18:05:00 : pyspectral.rayleigh] Band name: I1  Effective wavelength: 0.639021um
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='DNB', wavelength=WavelengthRange(min=0.5, central=0.7, max=0.9, unit='µm'), resolution=743, calibration=<calibration.radiance>, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=371, calibration=<calibration.reflectance>, modifiers=('sunz_corrected_iband',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=371, calibration=<calibration.reflectance>, modifiers=('sunz_corrected_iband', 'rayleigh_corrected_crefl_iband'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=371, calibration=<calibration.reflectance>, modifiers=('sunz_corrected_iband', 'rayleigh_corrected_iband'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I03', wavelength=WavelengthRange(min=1.58, central=1.61, max=1.64, unit='µm'), resolution=371, calibration=<calibration.reflectance>, modifiers=('sunz_corrected_iband',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I04', wavelength=WavelengthRange(min=3.58, central=3.74, max=3.9, unit='µm'), resolution=371, calibration=<calibration.brightness_temperature>, modifiers=('nir_reflectance',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I05', wavelength=WavelengthRange(min=10.5, central=11.45, max=12.3, unit='µm'), resolution=371, calibration=<calibration.brightness_temperature>, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M03', wavelength=WavelengthRange(min=0.478, central=0.488, max=0.498, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected_crefl'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M04', wavelength=WavelengthRange(min=0.545, central=0.555, max=0.565, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected_crefl'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M05', wavelength=WavelengthRange(min=0.662, central=0.672, max=0.682, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected', 'rayleigh_corrected_crefl'))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M07', wavelength=WavelengthRange(min=0.846, central=0.865, max=0.885, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M09', wavelength=WavelengthRange(min=1.371, central=1.378, max=1.386, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M10', wavelength=WavelengthRange(min=1.58, central=1.61, max=1.64, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M11', wavelength=WavelengthRange(min=2.225, central=2.25, max=2.275, unit='µm'), resolution=742, calibration=<calibration.reflectance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M12', wavelength=WavelengthRange(min=3.61, central=3.7, max=3.79, unit='µm'), resolution=742, calibration=<calibration.brightness_temperature>, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='M15', wavelength=WavelengthRange(min=10.263, central=10.763, max=11.263, unit='µm'), resolution=742, calibration=<calibration.brightness_temperature>, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='satellite_azimuth_angle', resolution=742, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='satellite_zenith_angle', resolution=742, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='solar_azimuth_angle', resolution=742, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='solar_zenith_angle', resolution=371, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='solar_zenith_angle', resolution=742, modifiers=())
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='DNB', wavelength=WavelengthRange(min=0.5, central=0.7, max=0.9, unit='µm'), resolution=743, calibration=<calibration.radiance>, modifiers=('sunz_corrected',))
[DEBUG: 2022-03-21 18:05:00 : satpy.scene] Unloading dataset: DataID(name='I01', wavelength=WavelengthRange(min=0.6, central=0.64, max=0.68, unit='µm'), resolution=371, calibration=<calibration.reflectance>, modifiers=('sunz_corrected_iband', 'rayleigh_corrected'))
adaptive_dnb radiance 1
ash None None
cloud_phase None None
cloud_phase_distinction None None
cloud_phase_distinction_raw None None
cloud_phase_raw None None
cloud_type None None
cloud_type_raw None None
cloudtop None None
cloudtop_daytime None None
day_microphysics None None
dust None None
dynamic_dnb radiance 1
false_color reflectance %
fire_temperature None None
fire_temperature_39refl None None
fire_temperature_awips None None
fire_temperature_eumetsat None None
fog None None
green_snow None None
histogram_dnb radiance 1
hncc_dnb radiance W m-2 sr-1
hr_cloudtop_daytime None None
hr_overview None None
ir108_3d None None
ir_cloud_day None None
natural_color reflectance %
natural_color_raw None None
natural_color_sun None None
natural_color_sun_lowres None None
natural_enh None None
natural_with_night_fog None None
night_fog None None
night_microphysics None None
night_overview None None
ocean_color reflectance %
overview None None
overview_raw None None
snow None None
snow_age reflectance %
snow_hires None None
snow_lowres None None
ssec_fog brightness_temperature K
true_color reflectance %
true_color_crefl reflectance %
true_color_lowres None None
true_color_lowres_crefl None None
true_color_lowres_land None None
true_color_lowres_marine_tropical None None
true_color_raw None None
Text output of actual results or error messages including full tracebacks if applicable.

Environment Info:

  • Satpy Version: v0.35.0-7-gf34d5630
@gerritholl
Copy link
Collaborator Author

I thought I had discussed this before, but couldn't find a corresponding issue, so maybe it was only in slack. I apologise if this is a duplicate for an issue I couldn't find right now.

@djhoese
Copy link
Member

djhoese commented Mar 21, 2022

So I'm a little torn on this, but only barely. The units and calibration metadata are retained because the input bands all have the same value so the combine_metadata utility function considers it useful or at least transferable to the new composite. I agree that it seems odd to have units for an RGB other than 1 (if we had to put some units on it).

The calibration is an interesting problem though. I think technically you can request a composite and specify a different calibration and Satpy will allow that to continue on through processing as if it was the "higher" level of calibration. I think @simonrp84 tried to do something like that before, but I'm not sure how that went. So the question is, is it wrong for that RGB to have a calibration="radiance" if the inputs were radiances and not reflectances? Even "worse" is that calibration is part of the DataID (or at least it can be). So you could theoretically load DataQuery(name="some_rgb", calibration="reflectance") and DataQuery(name="some_rgb", calibration="radiance") and they would be two different composites (based on inputs with those calibrations).

I guess the "safest" answer is to say neither should be present (or maybe units="1") and any user wanting to do fancy loading of composites should have to make a custom definition to load the specific calibrations they want. We should/could be more explicit in the satpy builtin composites to and add calibration: reflectance and calibration: brightness_temperature so what I talked about isn't possible.

@gerritholl
Copy link
Collaborator Author

I don't feel strongly about calibration in RGBs, but units — including units: 1 — is wrong for anything that is not a quantity. The units attribute is used downstream by the ninjotiff writer, which (rightly, IMHO) assumes that if values have units associated with them, they can be converted to different units. In the case of RGBs, this fails. As for calibration, even if we would derive different RGB composites from differently calibrated inputs, then units would have to be an array with three values, in case we have one composite derived from brightness temperature + reflectance and another purely from radiances.

@gerritholl
Copy link
Collaborator Author

I have left a comment on #1985 with some thoughts on why non-quantities should not have units associated with them. Those thoughts apply to RGBs as well.

@djhoese
Copy link
Member

djhoese commented Mar 22, 2022

assumes that if values have units associated with them, they can be converted to different units. In the case of RGBs, this fails.

Just to clarify, the main failure is that it is trying to do this with a 3D image (RGB/A) and not a 2D image, right?

As for calibration, even if we would derive different RGB composites from differently calibrated inputs, then units would have to be an array with three values, in case we have one composite derived from brightness temperature + reflectance and another purely from radiances.

This is a good point. As it is right now any resulting metadata in a composite can be thought of as existing because all inputs had that piece of metadata with that exact value. But that doesn't mean it is accurate to the RGB and it causes complications when input metadata was not all the same. So either the metadata handling should be updated to produce multiple inputs or it should be removed. Due to the complications of the multiple values for low-level parameters like calibration I think for now we need to remove these metadata fields.

One thought, what if a user wanted to force the units of an RGB or something else? Right now where you have the pop in #2068, the composite can never have the units attribute. What if I set it in the YAML though?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants