scene.available_composite_names() return a composite even the dependency is not fullfilled

[yufeizhu600]

Describe the bug
It seems the scene.available_composite_names() function has bug to check the dependencies of the ABI airmass composite, it returns airmass as one of the available composite even one of the dependency is missing.

To Reproduce
The ABI airmass composite depend on 4 channels data files, however even one of the file is missing, the scene.available_composite_names() still return airmass in the list as as a available composite.

To reproduce the issue, one can copy a set of 16 GOES-16 FullDisk passes to the test input directory then manually delete one of the dependency of airmass composite.

%cp /goes-16-grb/*s20191922000356* ./test/
%rm ./test/OR_ABI-L1b-RadF-M6C08_G16_s20191922000356_e20191922010064_c20191922010119.nc

#!/usr/bin/env python
import os
import sys
import glob

from satpy.scene import Scene
from satpy.utils import debug_on
debug_on()

input_dir = './test/'

dt_start = 's20191922000356'
fns = glob.iglob(input_dir + '/*' + dt_start + '*.nc')
scn = Scene(reader="abi_l1b", filenames=fns)
   
print scn.available_composite_names()
scn.load(['airmass'])

Expected behavior
Not expecting airmass in the print out list and loading 'airmass' composite should failed.

Actual results
'airmass' was in the available composite list, loading 'airmass' failed as expected.

['airmass', 'ash', 'cloudtop', 'convection', 'day_microphysics', 'dust', 'fog', 'green', 'green_crefl', 'green_raw', 'green_snow', 'ir108_3d', 'ir_cloud_day', 'natural', 'natural_color', 'natural_color_raw', 'natural_color_sun', 'night_fog', 'night_ir', 'night_microphysics', 'overview', 'overview_raw', 'pseudo_color_nomod', 'true_color', 'true_color_crefl', 'true_color_raw']

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Reading in get_dataset C12.

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Calibrating to brightness temperatures

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Reading in get_dataset C14.

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Calibrating to brightness temperatures

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Reading in get_dataset C10.

[DEBUG: 2019-07-11 20:30:26 : satpy.readers.abi_l1b] Calibrating to brightness temperatures

[WARNING: 2019-07-11 20:30:27 : satpy.readers.yaml_reader] Required file type 'c08' not found or loaded for 'C08'

[DEBUG: 2019-07-11 20:30:27 : satpy.scene] Missing prerequisite for 'DatasetID(name='airmass', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)': 'DatasetID(name='C08', wavelength=(5.77, 6.185, 6.6), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())'

[WARNING: 2019-07-11 20:30:27 : satpy.scene] The following datasets were not created and may require resampling to be generated: DatasetID(name='airmass', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)

[DEBUG: 2019-07-11 20:30:27 : satpy.scene] Unloading dataset: DatasetID(name='C12', wavelength=(9.42, 9.61, 9.8), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())

[DEBUG: 2019-07-11 20:30:27 : satpy.scene] Unloading dataset: DatasetID(name='C14', wavelength=(10.8, 11.2, 11.6), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())

[DEBUG: 2019-07-11 20:30:27 : satpy.scene] Unloading dataset: DatasetID(name='C10', wavelength=(7.24, 7.34, 7.44), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())

Environment Info:

• OS: Linux Ubuntu 16.04
• Satpy Version: 0.15.1

Additional context
Add any other context about the problem here.

[djhoese]

Can you double check that this is still an issue @yufeizhu600 with current master? I forgot about this and this is an important issue.

[yufeizhu600]

I upgraded to satpy==0.16.2.dev340+gd09edbc and done the same test as above. I still got ['airmass'] in the output of available composite names.

[DEBUG: 2019-09-30 20:42:56 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/local/home/goesdev1/project/goesr/abi_l2_reader/config/'
[DEBUG: 2019-09-30 20:42:56 : satpy.readers] Reading ['/mnt/proc-dev-goesr1/data/goesdev1/miniconda2/envs/py27_satpy-0.1x_test/lib/python2.7/site-packages/satpy/etc/readers/abi_l1b.yaml']
[DEBUG: 2019-09-30 20:42:56 : satpy.readers.yaml_reader] Assigning to abi_l1b: ['./test/OR_ABI-L1b-RadF-M6C10_G16_s20192732020261_e20192732029581_c20192732030030.nc', './test/OR_ABI-L1b-RadF-M6C11_G16_s20192732020261_e20192732029569_c20192732030039.nc', './test/OR_ABI-L1b-RadF-M6C14_G16_s20192732020261_e20192732029569_c20192732030070.nc', './test/OR_ABI-L1b-RadF-M6C16_G16_s20192732020261_e20192732029582_c20192732030059.nc', './test/OR_ABI-L1b-RadF-M6C09_G16_s20192732020261_e20192732029575_c20192732030055.nc', './test/OR_ABI-L1b-RadF-M6C15_G16_s20192732020261_e20192732029575_c20192732030064.nc', './test/OR_ABI-L1b-RadF-M6C12_G16_s20192732020261_e20192732029576_c20192732030035.nc', './test/OR_ABI-L1b-RadF-M6C13_G16_s20192732020261_e20192732029581_c20192732030045.nc', './test/OR_ABI-L1b-RadF-M6C01_G16_s20192732020261_e20192732029569_c20192732030033.nc', './test/OR_ABI-L1b-RadF-M6C02_G16_s20192732020261_e20192732029569_c20192732030017.nc', './test/OR_ABI-L1b-RadF-M6C03_G16_s20192732020261_e20192732029569_c20192732030040.nc', './test/OR_ABI-L1b-RadF-M6C04_G16_s20192732020261_e20192732029569_c20192732030000.nc', './test/OR_ABI-L1b-RadF-M6C05_G16_s20192732020261_e20192732029569_c20192732030037.nc', './test/OR_ABI-L1b-RadF-M6C06_G16_s20192732020261_e20192732029575_c20192732030047.nc', './test/OR_ABI-L1b-RadF-M6C07_G16_s20192732020261_e20192732029581_c20192732030029.nc']
[DEBUG: 2019-09-30 20:42:57 : satpy.composites] Looking for composites config file abi.yaml
[DEBUG: 2019-09-30 20:42:57 : satpy.composites] Looking for composites config file visir.yaml
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='satellite_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_azimuth_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None): Unknown dataset DatasetID(name='solar_zenith_angle', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional satellite_azimuth_angle: Unknown dataset satellite_azimuth_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional satellite_zenith_angle: Unknown dataset satellite_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_azimuth_angle: Unknown dataset solar_azimuth_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional satellite_azimuth_angle: Unknown dataset satellite_azimuth_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional satellite_zenith_angle: Unknown dataset satellite_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_azimuth_angle: Unknown dataset solar_azimuth_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
[DEBUG: 2019-09-30 20:42:57 : satpy.node] Skipping optional solar_zenith_angle: Unknown dataset solar_zenith_angle
['airmass', 'ash', 'cimss_green', 'cimss_green_sunz', 'cimss_green_sunz_rayleigh', 'cimss_true_color', 'cimss_true_color_sunz', 'cimss_true_color_sunz_rayleigh', 'cloudtop', 'convection', 'day_microphysics', 'dust', 'fog', 'green', 'green_crefl', 'green_raw', 'green_snow', 'ir108_3d', 'ir_cloud_day', 'natural_color', 'natural_color_raw', 'natural_color_sun', 'night_background', 'night_background_hires', 'night_fog', 'night_microphysics', 'overview', 'overview_raw', 'true_color', 'true_color_crefl', 'true_color_raw']
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Reading in get_dataset C12.
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Calibrating to brightness temperatures
[DEBUG: 2019-09-30 20:42:58 : satpy.resample] Could not add 'crs' coordinate with pyproj<2.0
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Reading in get_dataset C10.
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Calibrating to brightness temperatures
[DEBUG: 2019-09-30 20:42:58 : satpy.resample] Could not add 'crs' coordinate with pyproj<2.0
[WARNING: 2019-09-30 20:42:58 : satpy.readers.yaml_reader] Required file type 'c08' not found or loaded for 'C08'
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Reading in get_dataset C13.
[DEBUG: 2019-09-30 20:42:58 : satpy.readers.abi_l1b] Calibrating to brightness temperatures
[DEBUG: 2019-09-30 20:42:58 : satpy.resample] Could not add 'crs' coordinate with pyproj<2.0
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Missing prerequisite for 'DatasetID(name='_airmass_dep_0', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)': 'DatasetID(name='C08', wavelength=(5.77, 6.185, 6.6), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())'
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Missing prerequisite for 'DatasetID(name='airmass', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)': 'DatasetID(name='_airmass_dep_0', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)'
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Missing prerequisite for 'DatasetID(name='_airmass_dep_0', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)': 'DatasetID(name='C08', wavelength=(5.77, 6.185, 6.6), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())'
[WARNING: 2019-09-30 20:42:58 : satpy.scene] The following datasets were not created and may require resampling to be generated: DatasetID(name='airmass', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Unloading dataset: DatasetID(name='C12', wavelength=(9.42, 9.61, 9.8), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Unloading dataset: DatasetID(name='_airmass_dep_1', wavelength=None, resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Unloading dataset: DatasetID(name='C10', wavelength=(7.24, 7.34, 7.44), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())
[DEBUG: 2019-09-30 20:42:58 : satpy.scene] Unloading dataset: DatasetID(name='C13', wavelength=(10.1, 10.35, 10.6), resolution=2000, polarization=None, calibration='brightness_temperature', level=None, modifiers=())