Add arbitrary filename suffix to ABI L1B reader #1380
Conversation
djhoese
added
enhancement
|
I'm open to suggestions. Maybe this should be called |
Codecov Report
@@ Coverage Diff @@
## master #1380 +/- ##
=========================================
Coverage ? 90.56%
=========================================
Files ? 228
Lines ? 33421
Branches ? 0
=========================================
Hits ? 30269
Misses ? 3152
Partials ? 0
Continue to review full report at Codecov.
|
|
Why replace an existing standard filename pattern and not add it at the end of the pattern list as we usually do? |
This reverts commit ee653fc
|
Hm I could call the filename field "testing_identifier" or something to give it more meaning than "suffix" or "filename_suffix". |
|
👉 View analysis in DeepCode’s Dashboard | Configure the bot |
| @@ -81,8 +81,9 @@ def get_dataset(self, key, info): | |||
| # although we could compute these, we'd have to update in calibration | |||
| res.attrs.pop('valid_range', None) | |||
| # add in information from the filename that may be useful to the user | |||
| for attr in ('observation_type', 'scene_abbr', 'scan_mode', 'platform_shortname'): | |||
| res.attrs[attr] = self.filename_info[attr] | |||
| for attr in ('observation_type', 'scene_abbr', 'scan_mode', 'platform_shortname', 'suffix'): | |||
There was a problem hiding this comment.
I know this is not new, but I don't really like having to add to this everytime there is a new item popping up in the file patterns. Could these items to be passed to the user be defined in the yaml file instead?
There was a problem hiding this comment.
I wouldn't be against it, but I'm not sure what you mean. Do you mean the list of what should be added to the attrs should be in the YAML? Or do you mean that the actual values for these are defined in the YAML?
If the former, would that be on each file type? Or maybe in the main reader section?
There was a problem hiding this comment.
I was thinking the former, in the file type definition, but maybe the reader section is more appropriate
There was a problem hiding this comment.
@mraspaud I was going to try to implement this, but realized the problem with defining this list of attrs in the reader (the more logical place) makes the code difficult because the file handler's don't have access to the reader kwargs...oh unless I modify the ABI Base file handler and have it expect this kwarg. 🤔 Will look tomorrow, but let me know what you think.
There was a problem hiding this comment.
I think having them in the filetype info would be ok, because you might want to have different attributes for different filetypes.
There was a problem hiding this comment.
Yeah but then I have 16 repeat blocks of YAML plus the parameters are typically per "style" of file pattern and not necessarily file type. I'd just like to avoid this "clean up". It is beginning to not be clean at all. Let me know how strongly you care.
There was a problem hiding this comment.
I care about how clean the solution is more than where this info is :) so it's up to you!
There was a problem hiding this comment.
How about this: If this list of attributes changes again then I promise I will find a clean solution for this where the values are put somewhere more configure-y (YAML). Honestly, we could probably just copy all the filename parameters if they aren't already defined earlier in get_dataset. I'm going to merge this.
There was a problem hiding this comment.
ok, it's up to you. I'll take you on that promise though :D
There was a problem hiding this comment.
The abi l1b reader (at least the get_dataset method) is gross enough as it is. It needs to be refactored but this isn't the PR or the time for it. I know that technically breaks the "law of refactoring" but I didn't get enough sleep last night to care 😉
This addresses a user request in SIFT (ssec/sift#299) where @jgerth has some ABI L1b files that are generated by his own processing and are similar but not exactly the same as the original ABI L1b files that fed the processing. This PR allows him to easily name these files in an identifiable way while also providing some metadata to do that identification.
flake8 satpy