Add more flexible masking

[pnuu]

This PR adds a more flexible way to apply masking to composites.

• Closes Add more versatile options for masking datasets #1168
• Tests added
• Tests passed
• Passes flake8 satpy
• Fully documented

[pnuu]

It seems that my initial idea of adding a modifier isn't feasible, as the modifier can't be used to add alpha channel. Thus I went with yet another compositor.

The first test image looks like this:

Here brightness temperature values above 270 K are fully transparent, and values from 260 to 270 are with 50 % transparency and the values are inverted before saving.

Corresponding composite definition:

  ir_test:
    compositor: !!python/name:satpy.composites.MaskCompositor
    conditions:
    # operator, limit/threshold/value, transparency
    # the below is equivalent to
    #    alpha_val = 1. - transparency / 100.
    #    data = data.where(np.greater_equal(mask, 260), alpha_val, 1.0)`
    - [greater_equal, 260, 50]
    - [greater_equal, 270, 100]
    prerequisites:
    - name: IR_108
    - name: IR_108
    standard_name: ir_test

and enhancement:

  ir_test:
    standard_name: ir_test
    operations:
    - name: invert
      method: *stretchfun
      kwargs:
        stretch: crude
        # Note: also the alpha-channel limits need to be defined!
        min_stretch: [300, 0]
        max_stretch: [215, 1]

[pnuu]

I'm not sure what causes the weird pattern on the right of the image. It's not visible when I open the original GeoTIFF image in GIMP, but it is visible when viewed with qiv.

[coveralls]

Coverage decreased (-0.07%) to 89.534% when pulling 6ba1aa6 on pnuu:feature-mask-compositor into 65cd83f on pytroll:master.

[coveralls]

Coverage increased (+0.2%) to 89.766% when pulling 0853863 on pnuu:feature-mask-compositor into 65cd83f on pytroll:master.

[codecov]

Codecov Report

Merging #1175 into master will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1175      +/-   ##
==========================================
+ Coverage   89.60%   89.67%   +0.07%     
==========================================
  Files         200      200              
  Lines       29487    29738     +251     
==========================================
+ Hits        26422    26669     +247     
- Misses       3065     3069       +4     

Impacted Files	Coverage Δ
satpy/composites/__init__.py	79.53% <100.00%> (+0.76%)	⬆️
satpy/tests/test_composites.py	100.00% <100.00%> (ø)
satpy/readers/avhrr_l1b_gaclac.py	93.33% <0.00%> (-1.34%)	⬇️
satpy/readers/hrit_jma.py	97.94% <0.00%> (-1.22%)	⬇️
satpy/readers/yaml_reader.py	95.30% <0.00%> (-0.13%)	⬇️
satpy/writers/cf_writer.py	93.77% <0.00%> (-0.03%)	⬇️
satpy/writers/geotiff.py	90.74% <0.00%> (ø)
satpy/readers/netcdf_utils.py	100.00% <0.00%> (ø)
satpy/readers/seviri_l1b_native.py	71.96% <0.00%> (ø)
satpy/tests/reader_tests/test_ahi_hrit.py	100.00% <0.00%> (ø)
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65cd83f...0853863. Read the comment docs.

[pnuu]

As a side note: if I save directly to PNG or omit the tiled=True kwarg to the lcl.save_datasets() the image is fine.

[pnuu]

Would it be more clear to have

    - method: greater_equal
      value: 260
      transparency: 50

instead of

    - [greater_equal, 260, 50]

?
Or maybe there are better names for the "dictionary" version above, or another completely different way to give these information. Suggestions are wellcome! I'm going to adapt for the method/value/transparency approach in the meanwhile, it seems much more clear.

[pnuu]

The composite configuration is now like this:

  ir_test:
    compositor: !!python/name:satpy.composites.MaskingCompositor
    conditions:
    # the below makes everything above 270 K completely transparent, and values between 250 ... 270 K 50 % transparent
    - method: greater_equal
      value: 260
      transparency: 50
    - method: greater_equal
      value: 270
      transparency: 100
    prerequisites:
    # this is the data that will be shown
    - name: IR_108
    # this is used for masking with conditions shown above
    - name: IR_108
    standard_name: ir_test

[pnuu]

In the end I'll be merging this to the existing MaskingCompositor which was created to handle specifically masking with categorical cloud type data from NWC SAF PPS/GEO.

[pnuu]

Now also the existing composites that were using MaskingCompositor are converted to use the new syntax, and the functionalities are merged in to one compositor.

[pnuu]

I added one new composite that uses this feature: hrv_severe_storms_masked. It also uses a new feature, where np.isnan can be used to create a mask. Otherwise the data outside HRV channel would get a non-transparent background in the alpha channel.

[pnuu]

This is a crop of the new hrv_severe_storms_masked composite around central Africa. The composite is inted to be used overlayd on a static background, so it doesn't look very good on the white background.

[pnuu]

I can't make Codebeat any happier. The MaskingCompositor.__init__() needs all the arguments, and the other complaints are to files that I didn't touch.

[mraspaud]

LGTM, just one nitpick

[mraspaud]

So in principle this means any numpy method can be called. Just for security reasons, would you mind adding a check that method is not outside what we expect ?

[pnuu]

I added a list of accepted methods, and an unit test for trying to use something else. I hope I listed all the currently relevant methods.