Highlights reconstruction : introduce new method #10711
Conversation
aurelienpierre
added
feature: enhancement
|
This image shared by @TurboGit on #10716 (which was problematic for @jenshannoschwalm ) seems also problematic here: https://drive.google.com/file/d/1XFKbgAkbLfckY-JZKrn4mP9MsxLTc59c/view?usp=sharing |
|
This image has a too large clipped region, it's beyond reconstructibility. Also, it falls under "poor photographying skills" since clipped highlights should only be primary light sources (where you can't avoid it), not 25% of the image. |
|
I still think the algorithm should fallback nicely in such case, inpainting everything in grey for instance. For such case where big clouds are clipped, we don't really need details, just to avoid a too strong color cast. Also, I found a strange behavior on this image:
|
It's not possible. "Grey" is possible only after a full chromatic adaptation by enforcing R = G = B. We are here before demosaicing, we have no idea if the illuminant is right or not, we know no color at that point, only sparse gradients. Degrading to grey is the responsibility of filmic reconstruction that happens after proper color profiling and adaptation, meaning it has a sane and sanitized color framework to make assumptions upon colors.
Latest commit should deal with that. Thanks for your tests ! |
Another small issue, when zooming into the image in the darkroom, image is surrounded with a row and column of red or blue pixels Also, with new commits, the results on the puffin image are worse than before, the orange parts become green whatever the feathering values. |
It's normal, I don't demosaic the first/last rows and columns yet.
What is the last commit that works here ? |
|
6cc7c8a (remove WB corrections) is the first bad commit |
|
As we are at it, why the highlight recovery module default to "clip"? Can't we use a safe default like "reconstruct in LCh"? |
|
Hi @aurelienpierre I liked to try this out, but I get compilation errors on Windows MSYS2: Any idea for a fix? I'm building from an empty directory, as is normal. |
It's really not safe.
Fixed by 99f9958, I forgot to commit one file where that constant is defined. |
Can be worst than "clip" :) |
Clip is the only white-balance-agnostic method. All the others rely on the quality of the WB… which is not guaranteed. |
aurelienpierre
force-pushed
the
highlights-reconstruction-guided-laplacian
branch
from
99f9958 to
b21cc1f
Compare
|
@aurelienpierre don't know if #10716 has a chance to get into dt codebase too but if we bumb the module version it might be good to leave room in modules parameters for lets say two ints and floats each. Wouldn't hurt but would make testing for other folks much easier. Or maybe do a seperate pr just handling the module version to prepare for such a scenario? |
|
@jenshannoschwalm actually @rawfiner raised a good point : why are we taking the trouble of recovering HL before demosaicing ? We need as many branches as CFA patterns and a test demosaicing that will be ditched. So I wonder if this wouldn't be better in a new module after demosaicing. |
|
For me this seems to be good too. Would also
Would offer to implement the modules framework ... |
|
Great, I'll take it. If you reserve me 3 floats in addition of the clipping threshold in the params structure, then I can rebase my work on yours without legacy import. |
|
To sum up what I think between being before or after demosaic for highlight reconstruction:
I think it is worth trying to have the module after demosaic for both of your approaches:
|
|
Notice that I already postfilter the clipping mask with a 5x5 box blur because the reconstruction creates overshoots otherwise. |
|
On second thought , processing per channel will be 4-fold, correct? |
what do you mean ? |
|
At least for the segmentation I do, in Bayer space the size for width / height are half size compared to after demosaicing. |
|
Indeed if you did segmentation using red or blue channel, you get 4x more pixels after demosaic, so it can result in a slowdown |
|
You can always downsample though. |
|
No strange behavior on AMD although it is a bit slow. |
|
I know it's slow, but I don't see any workaround… |
|
Note: I have changed the sequence a bit, for the sake of run time. The number of wavelets scales is limited to 8, meaning a max radius of 256 px. But since the process is now iterative, we need to double the padding size to avoid straight artifacts at tile's edges. So the max padding is 512 px. The result is very large blown areas, with radii larger than 512 px will not be corrected at all. It's either this or insane runtimes. |
|
"The result is very large blown areas, with radii larger than 512 px will not be corrected at all. It's either this or insane runtimes." Just a thought, could potentially have both, with a user drop down to select blown areas: small or large, where small allows for many iterations, but large allows only for one. |
|
Build fails on win10: introduced with dec805c |
aurelienpierre
force-pushed
the
highlights-reconstruction-guided-laplacian
branch
from
5b1f2b8 to
3c2c685
Compare
|
I solved conflicts, rebased and fixed the anisotropic diffusion kernel similarly to what @flannelhead did for diffuse.c. @TurboGit this is ready to go ! |
|
I've not looked at the code yet, testing it I see that I have some magenta color cast on tow test images. I'm using Filmic and recontruct is threshold is negative -0.8IL. First image, module highlight reconstruction set to
Second image:
Is that a wrong use on my side? Also I see that the the X-Trans (RAF) are not supported, I suppose this is expected? |
|
If I change the white rel exposure in Filmic it somehow solves the issue. To me this method seems to be very sensitive to while being too strong, again not sure it is expected. Until the doc is there it is hard for me to guess all that :) |
|
With Filmic v6 it is even harder to get a decent output with this algorithm even when tweaking the white:
|
|
With Filmic default setting in v5: in v6: |
|
@TurboGit It's important to understand that this algo doesn't care about colors. There is no desaturation or no explicit color propagation like the other methods do. What is done is:
This means that the algo will have an hard time with large areas, for which you need to increase the diameter of reconstruction and the number of iterations, at the expense of runtime. Also, since filmic v6 does not desaturate highlights mandatorily, obviously magenta casts are more visible. |
Ok, will test again with that in mind. Thanks for the explanation! |
|
I have hard time to find a picture where it makes a difference with the other algorithm. I generally find that "color reconstruction" works best without the color cast and that Laplacian gives almost the same result but with noise added with the image is high iso. @aurelienpierre : Can you share a picture where this algorithm gives results that cannot be achieved with the others? |
|
There is a missing update somewhere. To reproduce:
|
Messing with the LF layers causes non-smooth edge artifacts around areas to reconstruct. The cost of using only HF layers is more iterations and/or larger reconstruction radii are needed. The additional benefit is now the wavelets decomposition can be rewritten from top to bottom, meaning only 3 buffers (LF, HF, synthesis) need to be stored, instead of all HF scales. This will prevent tiling in most situations.
|
@TurboGit GUI bug fixed, it was an easy one. Next commit changes slightly the reconstruction logic to fix an artifact I uncovered. It actually removes some operation. It will allow to rewrite the wavelets decomposition to be sequential instead of parallel, meaning only 3 buffers (LF, HF, synthesis) will be needed instead of having to store all HF buffers. That will prevent tiling and the huge perf penalty that goes with it. |
elstoc
added
documentation-complete
aurelienpierre
deleted the
highlights-reconstruction-guided-laplacian
branch
We merge the spirit of guided filters, wavelets multiscale processing and isotropic diffusion to introduce a new highlights reconstruction method that guides the laplacian of the R, G, B channels with the laplacian of the norm. This goes in the usual RAW highlights module.
What it does:
What it does not:
Where it fails:
Notes:
It works better in conjunction with filmic reconstruction, which uses a similar method but with different priorities (smoothness first).
Limitations:
Demo : https://discuss.pixls.us/t/guiding-laplacians-to-restore-clipped-highlights/28493