Metadata filtering #58
Conversation
| @@ -381,7 +381,9 @@ def __init__(self, | |||
| start_time=None, | |||
| end_time=None, | |||
| area=None, | |||
| filter_filenames=True, **kwargs): | |||
There was a problem hiding this comment.
I am not 100% sure, but I believe the **kwargs is needed for any additional keyword arguments that may have been specified for one reader, but are not useful for another. The two cases where this would show up are 1. a Scene with more than one reader (not a thing right now I think) and 2. instantiating readers in order to match files names and passing all user provided keyword arguments.
satpy/readers/yaml_reader.py
Outdated
| @@ -543,8 +560,8 @@ def new_filehandlers_for_filetype(self, filetype_info, filenames): | |||
| filehandler_iter = self.new_filehandler_instances(filetype_info, | |||
| filename_iter) | |||
| return [fhd | |||
| for fhd in self.filter_fh_by_area(self.filter_fh_by_time( | |||
| filehandler_iter))] | |||
| for fhd in self.filter_fh_by_area(self.filter_fh_by_time(self.filter_fh_by_mda( | |||
There was a problem hiding this comment.
Since these are generators maybe it would look better to instantiate them separately, above the list comprehension. Then use the final one in the list comprehension...or shouldn't this just be list(generator)?
There was a problem hiding this comment.
so like
filtered = self.filter_fh_by_area(self.filter_fh_by_time(filehandler_iter))
return filtered?
We could probably return the iterator btw
There was a problem hiding this comment.
I mean:
filtered_time = self.filter_fh_by_time(filehandler_iter)
filtered_area = self.filter_fh_by_area(filtered_time)
return list(filtered_area)
Or something similar.
satpy/readers/yaml_reader.py
Outdated
| yield filehandler | ||
| continue | ||
| for key, val in self.metadata.items(): | ||
| if key in filehandler.mda and val != filehandler.mda[key]: |
There was a problem hiding this comment.
Is mda a common shorthand for metadata? Should we be more specific and consistent and call it metadata?
There was a problem hiding this comment.
Also, maybe the base file handler should create the metadata dictionary and assign the filename info to it?
See conversation in PR #58
3 similar comments
This adds the possibility to use metadata to filter data to load, eg:
glbl = Scene(service='0° Service', sensor="seviri", reader="hrit_msg", start_time=tslot, end_time=tslot + dt.timedelta(minutes=5), base_dir=thedir)which would filter on
serviceif it is defined in the filehandler metadata