New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix combination of raw metadata (again) #1670
Conversation
Instead of dropping raw metadata, enable combine_metadata to recursively compare nested dictionaries.
Codecov Report
@@ Coverage Diff @@
## main #1670 +/- ##
==========================================
+ Coverage 92.66% 92.67% +0.01%
==========================================
Files 258 258
Lines 37988 38071 +83
==========================================
+ Hits 35200 35283 +83
Misses 2788 2788
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
While I'm OK with this, it is only partially because I don't use SEVIRI data on a regular basis. What kind of performance issues can we expect with something like this? If they are significant, maybe dict comparison should be turned off by default and have a keyword argument to enable it? Then the SEVIRI file handlers could override the base |
For SEVIRI HRIT it increases the runtime of |
Co-authored-by: David Hoese <david.hoese@ssec.wisc.edu>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good from what I understand.
In my recent attempt to fix the combination of raw metadata I updated
satpy.dataset.metadata.combine_metadata
to drop theraw_metadata
attribute. This works fine with single-file datasets (e.g. SEVIRI Native), but for multi-file datasets (e.g. SEVIRI HRIT) the attribute is dropped even if only a single dataset has been loaded. This is of course not desired.This PR enables
combine_metadata
to compare (nested) dictionaries, so that the removal of theraw_metadata
attribute is not necessary anymore. In the end it wasn't as complicated as I thought.