Add guide and configurable outlier removal to low-pass filtering

[oberonia78]

This PR updates the ionosphere low-pass filtering workflow to make gap filling more robust in noisy scenes and to make the smoothing/outlier behavior configurable.

The main issue in the previous implementation was that interpolation could propagate noisy patterns from valid pixels into invalid regions before the final low-pass filtering step. This PR introduces a smoother guide image for filling, optional robust outlier rejection before filling, and configurable parameters for guide construction and outlier detection.

This PR does not affect on the areas over not-masked areas. It improves only areas where pixels are masked and their neighbor pixels.

Main changes
Update low_pass_filter to improve stability in noisy regions by introducing:

NaN-aware guide smoothing (nan_aware_gaussian)
Optional robust outlier removal before filling
Configurable parameters for guide and outlier detection

Changes

• Add parameters:

  • guide_filter_method
  • guide_median_size
  • outlier_threshold
  • outlier_min_scale

• Build guide image using:

  • median + Gaussian (default)
  • Gaussian only
  • or no smoothing

• Apply MAD-based outlier detection (optional)

• Skip outlier computation when disabled

• Preserve original valid pixels after filling

Previous:

Current:

ionosphere difference (old - new):

[hfattahi]

Thanks @oberonia78 . We will review this PR in more details shortly, but here is my main comment so far.

[hfattahi]

This reminds me of a similar attempt we had a decade ago to avoid the more complicated iterative filtering;) At that time I got mixed results with very good results and very bad results. So I gave up at that time for the reasons I explain below. But I hope this algorithm is improved and will lead to consistent improvements in the filtered data.

I'm a bit concerned about this approximation. I call it approximation because of the following:
Let's say our noisy data is F and we have a Mask M with 0, 1 values. Then the masked data is :
Fm = F.M
Where Fm is the masked data. When we filter the masked data with a filter kernel G we have :

G*Fm = G * (F.M)
Where * is convolution.

However the problem is the filtering artifacts at the edges of the masked regions. We would like to have filtered data without those edge artifacts. One could rewrite the equation above:

G*(F.M) ~ GF . GM

And we are interested in the original filtered data without the ringing introduced by the filtered mask. So we could normalize or deconvolve (even though people may distinguish between the two, but to me it is decnvolution!):

GF ~ G(F.M)/G*M

So as you see it is an approximation and can only be exact if F locally (within the smoothing kernel) is constant. Variation of F within the smoothing window starts violating the equation and the more high frequency in the window, diverges the equation from exact.

[hfattahi]

We discussed this PR offline. While the theoretical details above is correct, but the intention here is not to replace this approach with independent gap filling algorithm in place. The existing workflow interpolates unfiltered data to fill gaps based on masks and the current PR changes that by first filtering the ionospheric phase over valid pixels only and then fill the gaps using filtered data. The guided filter above is used only in the initial filtering of the valid pixels. So this is fine with me.

[hfattahi]

LGTM! The results from this PR has over all improved the filtering of ionospheric phase. The added functionality is configurable and this addition allows to be tested at scale.

[hfattahi]

We discussed this PR offline. While the theoretical details above is correct, but the intention here is not to replace this approach with independent gap filling algorithm in place. The existing workflow interpolates unfiltered data to fill gaps based on masks and the current PR changes that by first filtering the ionospheric phase over valid pixels only and then fill the gaps using filtered data. The guided filter above is used only in the initial filtering of the valid pixels. So this is fine with me.

[xhuang-jpl]

Thank @oberonia78 , it looks good to me. Few more questions, and then I will approve it.

[xhuang-jpl]

Thank you for addressing my concerns. LGTM!