Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scene metadata overwriting composite metadata and handling sets in filename generation #1550

Closed
djhoese opened this issue Feb 18, 2021 · 2 comments · Fixed by #1551
Closed

Comments

@djhoese
Copy link
Member

djhoese commented Feb 18, 2021

Describe the bug
I thought I had a bug report for this already but can't find it now. So here we go.

When creating a composite that is a SingleBandCompositor, the Scene's metadata is applied after the prerequisite metadata. This means that it always gets overwritten. The biggest issue for this is that the sensor metadata may not actually make sense. If you load multiple readers in a Scene then sensor likely has two values {'abi', 'glm'} even if my SingleBandCompositor only uses one of them. Almost as bad is that even if I have one sensor (ex. 'abi') the sensor from the Scene is always a set so {'abi'}.

This is a big problem for me when I'm trying to write filenames with the sensor in the filename. See code below.

To Reproduce

from satpy import Scene; from glob import glob

scn = Scene(reader='seviri_l1b_hrit', filenames=glob('/data/satellite/seviri/hrit/H-000-MSG3__-MSG3________-*'))
scn.load(['colorized_ir_clouds', 'IR_108'])
print(scn['IR_108'].attrs['sensor'])  # 'seviri'
print(scn['colorized_ir_clouds'].attrs['sensor'])  # {'seviri'}

scn.save_datasets(filename="{name}_{sensor}.tif")  # colorized_ir_clouds_{'\''seviri'\''}.tif
scn.save_datasets(filename="{name}_{sensor!l}.tif")  # the `!l` makes it do lowercase on the sensor name and this fails

The last line produces:

AttributeError: 'set' object has no attribute 'lower'

Expected behavior

  1. DataArray metadata doesn't get overwritten by Scene metadata
  2. The get_filename operations of the writer can handle the sensor as a set. Even if not the prettiest, it should handle it in some way so it doesn't error out (it needs a string representation).

Actual results

See above. Set __repr__ in a filename or exceptions. Or in the abi/glm case an incorrect sensor attribute.

@djhoese
Copy link
Member Author

djhoese commented Feb 18, 2021

Proposed Solution

  1. Modify the SingleBandCompositor so it only updates metadata that isn't already present. Current code:

            new_attrs.update({key: val
                           for (key, val) in attrs.items()
                           if val is not None})
    

    Should have a if key not in new_attrs or some cleaner way of doing htis.

  2. Update get_filename to handle special keys like sensor. This is not very generic but if we consider 'sensor' as a special/standard Satpy key then maybe it is ok. '-'.join(sorted(attrs['sensor']))?

@djhoese
Copy link
Member Author

djhoese commented Feb 18, 2021

One downside to number one above is that attrs is supposed to be metadata that the user is providing to set on the composite, but I'm not sure this is necessary in most cases. I honestly don't know why the Scene is passing its metadata to the compositor at all. I think the original idea was to handle resolutions in some way, but Scene.attrs never contains a resolution value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant