Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tone equalizer: better mask autotune (for testing) #10014

Closed
wants to merge 4 commits into from

Conversation

Mark-64
Copy link
Contributor

@Mark-64 Mark-64 commented Sep 16, 2021

This PR is not ready for merging as it will break old edits and presets.

It is meant to test and discuss improved algorithm for mask auto-tune pickers in tone equalizer.
If those changer are accepted, I will transform it in a real PR.

Today one issue in tone equalizer is the effectiveness of the pickers for auto exposure and contrast correction, also reflected in discussions in DT forums. Moreover, there is cross-talk between the two sliders, as exposure correction changes contrast and vice-versa.
Here, we introduce two changes:

1) auto-exposure correction
This is the easiest, as it will not break edits.
The auto-exposure correction is based on mask histogram deciles. However, those are calculated on an histogram restricted in the [-8, 0] EV range. Luminance values outside that range are capped and counted in the first or last bin of the histogram.
This is especially an issue for the highlights part, as TE comes in the unbound part of the pipeline and after exposure correction most of the times the luminance exceeds 0 EV. This can lead to an inaccurate estimation of g->histogram_last_decile and underestimating the amount of exposure correction.
The solution implemented here is to calculate first an extended histogram in the [-8, +8] EV range (with double number of bins), use that for deciles estimation and then remap the extended histogram in the normal one.
It is not computationally expensive, just another small array of integers.
I couldn't find a method to avoid the two steps with temp extended histogram and calculate the deciles directly in the OpenMP parallel loop. Maybe someone more expert than me can find a better implementation.
Also I changed deciles in "ventiles" as with deciles we get potentially 20% of pixels out of reach of the TE correction and this is way too much, as most often than not the whole purpose TE is used is to fix those extreme values.
Overall the new auto correction works really well and in most cases is the only adjustment needed for the mask.

2) contrast correction and auto-contrast
this is trickier, as it will break old edits and it requires to step up TE version.
the issue here is the use of a linear contrast function

static float linear_contrast(const float pixel, const float fulcrum, const float contrast)
{
  // Increase the slope of the value around a fulcrum value
  return fmaxf((pixel - fulcrum) * contrast + fulcrum, MIN_FLOAT);
}

the use of subtraction with non log encoded data quickly drives to negative numbers as contrast correction increases, which then are capped to MIN_FLOAT. The result is the histogram change in asymmetrical way, the shadows part extend much more and the luminance mask goes black for the shadows.
I replaced that function with a log contrast like others used in DT (for example in color balance rbg)

static float log_contrast(const float pixel, const float fulcrum, const float contrast)
{
  // Increase the slope of the value around a fulcrum value
  return fmaxf(pixel * powf(pixel / fulcrum, contrast), MIN_FLOAT);
}

The auto correction math is updated as a consequence.
I am aware of the problems with non-linear correction and color, but here it is just a monochromatic mask and nothing bad seems to happen.
Overall the contrast slider and the auto picker behave really well IMO, it is super easy to nail a mask that covers all correction nodes and TE becomes a joy to use.

@aurelienpierre please test the PR if you like and give me feedback.

@Mark-64
Copy link
Contributor Author

Mark-64 commented Sep 17, 2021

Ok, here are some tests on real images.
The test is done by roughly adjusting exposure for midtones, turning on TE and clicking in sequence auto exposure and auto contrast, only one click each.
For each image see below histogram and mask

Image1

1

current master

1i
1m

this PR

1i2
1m2

Image 2

2

current master

2i
2m

this PR

2i2
2m2

Image 3

3

current master

3i
3m

this PR

3i2
3m2

@aurelienpierre
Copy link
Member

The contrast algo seems to fight against the edge-detection from the guided filter : just see how your examples don't behave like a surface blur anymore, especially in highlights. That can prove tricky to actually achieve details preservation. The linear contrast behaves well regarding the patch-wise variance, which is used by the guided filter to detect edges. But if you rewrite the relationship between neighbouring pixels in a non-linear way, then you rewrite the variance in a random way, and then the edge detection becomes erratic.

@Mark-64
Copy link
Contributor Author

Mark-64 commented Sep 17, 2021

I see your point, reverted to linear contrast.
With linear contrast, exposure boost and contrast boost crosstalk, and optimizing one parameter at a time with the two pickers does not really work.
An idea could be to adjust both exposure and contrast together, which means one picker only.
In the last commit, I left the exposure picker as it is to adjust only exposure, while in the contrast picker I put formulas to adjust simultaneously contrast and exposure, targeting 90% of the histogram in [-7, -1] EV range.
@aurelienpierre can you please tell me what you think? If you like the idea I can remove one of the two pickers.

@Mark-64
Copy link
Contributor Author

Mark-64 commented Sep 18, 2021

By the way, coming back to linear vs log contrast, I wanted to compare the two algos in the same conditions for an image, so I adjusted mask exposure and contrast so that the histogram spans more or less in the same range. See below:

Linear contrast
Screenshot from 2021-09-18 07-44-38

Screenshot from 2021-09-18 07-45-27

Log contrast
Screenshot from 2021-09-18 07-51-51

Screenshot from 2021-09-18 07-52-20

To my (not well trained) eye the difference between the two can be explained by the different distribution of the contrast across the tonal range.
By nature, linear contrast will be concentrated in highlights, while log contrast will be more effective in shadows.
This is nothing new or strange, but I don't see how log contrast would undermine edge detection in guided filter.
@aurelienpierre where am I wrong?

(maybe we could have both options?)

@Mark-64
Copy link
Contributor Author

Mark-64 commented Sep 18, 2021

with the last commit I splitted the auto-adjust formulas in the two pickers, they work pretty decently also separated

@Mark-64
Copy link
Contributor Author

Mark-64 commented Sep 19, 2021

Never mind, I found a decent solution for auto exposure and constrast, leaving TE algorithm as is.
Discussion linear vs log contrast will be for another day.
I will rebase and merge this with #9998, to clean up
closing here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants