geolocation problem with MODIS LAADS data

[hproe]

Describe the bug
MODIS hdf data from LAADS in satpy 0.24 has problems with orbital parameters when asking for Rayleigh correction, eg in true_color composite

Actual results
KeyError Traceback (most recent call last)
~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/utils.py in get_satpos(dataset)
294 try:
--> 295 orb_params = dataset.attrs['orbital_parameters']
296

KeyError: 'orbital_parameters'

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last)
in
18 scn=Scene(filenames=file,reader='modis_l1b')
19 cmp='true_color'
---> 20 scn.load([cmp])
21 scn0=scn.resample('northarabiansea',resampler='bilinear')
22 scn0.show(cmp,{'coast_dir':'/home/hp/DATA/maps/gshhg-shp-2.3.7','color':'orange','resolution':'l'})

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/scene.py in load(self, wishlist, calibration, resolution, polarization, level, generate, unload, **kwargs)
987 self._read(**kwargs)
988 if generate:
--> 989 keepables = self._generate_composites()
990 else:
991 # don't lose datasets we loaded to try to generate composites

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/scene.py in _generate_composites(self, nodes)
900 required_nodes = self._wishlist - set(self._datasets.keys())
901 nodes = set(self._dependency_tree.trunk(nodes=required_nodes)) - set(self._datasets.keys())
--> 902 return self._read_composites(nodes)
903
904 def _remove_failed_datasets(self, keepables):

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/scene.py in _read_composites(self, compositor_nodes)
876 keepables = set()
877 for item in compositor_nodes:
--> 878 self._generate_composite(item, keepables)
879 return keepables
880

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/scene.py in _generate_composite(self, comp_node, keepables)
850
851 try:
--> 852 composite = compositor(prereq_datasets,
853 optional_datasets=optional_datasets,
854 **self.attrs)

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/modifiers/atmosphere.py in call(self, projectables, optional_datasets, **info)
65 if not optional_datasets or len(optional_datasets) != 4:
66 vis, red = self.match_data_arrays(projectables)
---> 67 sata, satz, suna, sunz = self.get_angles(vis)
68 red.data = da.rechunk(red.data, vis.data.chunks)
69 else:

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/modifiers/atmosphere.py in get_angles(self, vis)
47 sunz = sun_zenith_angle(vis.attrs['start_time'], lons, lats)
48
---> 49 sat_lon, sat_lat, sat_alt = get_satpos(vis)
50 sata, satel = get_observer_look(
51 sat_lon,

~/anaconda3/envs/satpy/lib/python3.8/site-packages/satpy/utils.py in get_satpos(dataset)
323 except KeyError:
324 # Legacy
--> 325 lon = dataset.attrs['satellite_longitude']
326 lat = dataset.attrs['satellite_latitude']
327 alt = dataset.attrs['satellite_altitude']

KeyError: 'satellite_longitude'

Screenshots
If applicable, add screenshots to help explain your problem.

Environment Info:

• OS: Linux
• Satpy Version: 0.24
• Reader: modis_l1b

[djhoese]

As mentioned on slack, this should only happen if you haven't provided the geolocation files for MODIS. See this line of code:

satpy/satpy/modifiers/atmosphere.py

Line 65 in a701b9a

if not optional_datasets or len(optional_datasets) != 4:

This line basically says "if we weren't given the currently optional solar_zenith_angle DataArray and other angle DataArrays (total of 4) then try to create them from orbital parameters". These angle datasets should come from the geolocation files for MODIS. Can you provide the code you ran (including the filenames) to produce this error?

Overall this is still a bug since the rayleigh correction for MODIS should be configured for these angle datasets to NOT be optional.

[hproe]

Please find below the link to my code and the data file used. As already said the data are in a hdf5 file retrieved from LAADS DAAC. I see that true-color RGB (and similar composites in contained modis.yaml) require orbital parameters, which are not found in my data from LAADS DAAC. I wonder what data (and where from) has been used to test  the RGB schemes given in modis.yaml. cheers, HP MODIS.7z <https://we.tl/t-RQclU5It7M>(150 MB)WeTransferhttps://we.tl/t-RQclU5It7M <https://we.tl/t-RQclU5It7M> Mozilla Thunderbird <https://www.thunderbird.net> makes it easy to share large files over email.

…

On 28/12/2020 16:49, David Hoese wrote: As mentioned on slack, this should only happen if you haven't provided the geolocation files for MODIS. See this line of code: https://github.com/pytroll/satpy/blob/a701b9ad915e30b1dff6aaa23d254a6d705c9a1b/satpy/modifiers/atmosphere.py#L65 <https://github.com/pytroll/satpy/blob/a701b9ad915e30b1dff6aaa23d254a6d705c9a1b/satpy/modifiers/atmosphere.py#L65> This line basically says "if we weren't given the currently optional solar_zenith_angle DataArray and other angle DataArrays (total of 4) then try to create them from orbital parameters". These angle datasets should come from the geolocation files for MODIS. Can you provide the code you ran (including the filenames) to produce this error? Overall this is still a bug since the rayleigh correction for MODIS should be configured for these angle datasets to /NOT/ be optional. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1494 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AITAXJL4QE6B4IQO4FMC6YLSXCSIBANCNFSM4VLYPOBA>.

[djhoese]

I downloaded your file. Next time I think you could just paste the code here on github. Your provided data file(s) only include the MYD021KM file. There are also MYD03 which contain the geolocation information. Without these files the rayleigh correction cannot be done accurately. You must use both files (MYD/MOD02 and MYD/MOD03) when creating the Scene object so that Satpy can find all the necessary information.

[hproe]

Aha, I have not been aware of the MYD/MOD03 files. Got it working now. thanks a lot, HP

…

On 29/12/2020 15:17, David Hoese wrote: I downloaded your file. Next time I think you could just paste the code here on github. Your provided data file(s) only include the MYD021KM file. There are also MYD03 which contain the geolocation information. Without these files the rayleigh correction cannot be done accurately. You must use both files (MYD/MOD02 and MYD/MOD03) when creating the Scene object so that Satpy can find all the necessary information. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1494 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AITAXJJX5KGFQJCW2NVAIYLSXHQHFANCNFSM4VLYPOBA>.