tone equalizer: eigf does not tolerate adding contrast to the mask

[Mark-64]

During my tests for developing #10027, I stumbled upon what I believe is an issue of the eigf default TE algorithm, and in particular of its surface blur. I will illustrate here how to reproduce with an example.

Start with an edit and roughly adjust exposure, filmic etc.

The pic is a bit hazy, so say I want to deepen shadows with TE. I dial some negative exp compensation to roughly center the mask

histogram looks quite compressed, I need to add some contrast to set apart shadows from the rest of the image.
I dial in some contrast, making sure I get no orange warnings in the grey bar

but now look how in the mask there no surface blur anymore in shadows and most of them went pitch black.
If I now try to lower shadows:

I get artifacts

because there is no longer detail preservation.

Note that the guided filter algorithm does not show that problem, I can actually apply way more contrast to the mask without adverse effects

[Mark-64]

@aurelienpierre
I believe this has a lot to do with how constrast is applied.
Current linear contrast is prone to this issue, while a plain log contrast behave much better for the shadows (and a bit less so for highlights).
See also my comments in my old PR #10014, now closed.

[aurelienpierre]

You simply need to reduce the contrast here…

[aurelienpierre]

Note that the guided filter algorithm does not show that problem, I can actually apply way more contrast to the mask without adverse effects

Of course, because the vanilla GF is not exposure-invariant and therefore blurs the shadows more heavily than the highlights.

[Mark-64]

In eigf.h, the function

void eigf_blending_no_mask(float *const restrict image,
                  const float *const restrict av, const size_t Ndim,
                  const dt_iop_guided_filter_blending_t filter,
                  const float feathering)
{

#ifdef _OPENMP
#pragma omp parallel for simd default(none) \
  dt_omp_firstprivate(image, av, Ndim, feathering, filter) \
  schedule(simd:static) aligned(image, av:64)
#endif

  for(size_t k = 0; k < Ndim; k++)
  {
    const float avg_g = av[k * 2];
    const float var_g = av[k * 2 + 1];
    const float norm_g = fmaxf(avg_g * image[k], 1E-6);
    const float normalized_var_guide = var_g / norm_g;
    const float a = normalized_var_guide / (normalized_var_guide + feathering);
    const float b = avg_g - a * avg_g;
    if(filter == DT_GF_BLENDING_LINEAR)
    {
      image[k] = fmaxf(image[k] * a + b, MIN_FLOAT);
    }
    else
    {
      // filter == DT_GF_BLENDING_GEOMEAN
      image[k] *= fmaxf(image[k] * a + b, MIN_FLOAT);
      image[k] = sqrtf(image[k]);
    }
  }
}

IMO norm_g can go down too low for deep shadows that are created by the linear contrast when pushed a bit.
As a consequence normalized_var_guide can easily be 200+, a becomes very close to 1 irrespective of feathering, b becomes very close to 0. Therefore
image[k] = fmaxf(image[k] * a + b, MIN_FLOAT);

becomes equal to itself, there is no blending and no blur at all. So vanilla gf blurs shadows too much, but eigf too little.
I found that replacing 1E-6 with 1E-3 as a minimum value keeps some small blur for deep shadows and performs much better when contrast is increased. see the previous example where I increased contrast even more, the histogram spans overs almost the full range and yet no artifacts in shadows.

@aurelienpierre if you are interested I can provide a test I have done with synthetic images that demonstrates current eigf applies no blur at all in deep shadows and the proposed modification improves a lot

[Mark-64]

By the way in the documentation comments at the top of eigf.h, it says

* we use a ratio:
 * variance / (pixel_value)^2
 * we tried also the following ratios:
 * - variance / average^2
 * - variance / (pixel_value * average)
 * we kept variance / (pixel_value)^2 as it seemed to behave a bit better than
 * the other (dividing by average^2 smoothed too much dark details surrounded
 * by bright pixels).

but the code implements variance / (pixel_value * average) instead.

[aurelienpierre]

I don't understand the goal here.

Nobody said that any filter was immune to bad contrast values. Instead, the contrast correction is meant to make the mask more manageable. So the contrast has to be set such that the filter behaves, it's not for the filter to accommodate insane contrast settings. It's a purely technical control that is only meant to push the filters in their sweet spot. If instead it makes them fail, then it's ill-set. It's not artistic, anyway it has no direct aesthetical effect on the picture at the end.