HDR Tone Mapping Is Still Causing Clipping in 2021

[sovon]

MacOS Big Sur 11.3.1 (20E241) - Apple Silicon Mac Mini M1 - LG 4K HDR 27UK850

mpv 0.33.1 Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
 built on Mon Apr  5 10:16:25 PDT 2021
FFmpeg library versions:
   libavutil       56.51.100 (runtime 56.70.100)
   libavcodec      58.91.100 (runtime 58.134.100)
   libavformat     58.45.100 (runtime 58.76.100)
   libswscale      5.7.100 (runtime 5.9.100)
   libavfilter     7.85.100 (runtime 7.110.100)
   libswresample   3.7.100 (runtime 3.9.100)
FFmpeg version: 4.4

I've talked about this issue in the past. Not going to waste everyone's time by doing this same thing again. Just wanted to let the community know about a recent discovery of mine. I purchased a license of Infuse today after I saw what that app can do with HDR content. I want to support this mpv open source community and so I'm posting these discoveries of mine after hours of testing in case they can be implemented in MPV as well.

It's a well known fact that displaying HDR content on a SDR monitor is never a straightforward conversion. I have played around A LOT with the 6 algorithms - hable, clim, gamma, linear, mobius and reinhard. And for the past 5 years my mpv config has consistently looked something like this:

#----------------------------------------------------------- VIDEO
icc-profile-auto
hwdec=videotoolbox
tone-mapping=reinhard
tone-mapping-param=0.65

#----------------------------------------------------------- PLAYER
no-osd-bar
volume=100
volume-max=150
window-scale=0.5
script-opts=osc-scalefullscreen=0.7

#----------------------------------------------------------- SUBTITLE
sub-scale=0.5
sub-color="#dadada"


#----------------------------------------------------------- EXTERNAL
--cache=yes
--demuxer-max-bytes=50M


#----------------------------------------------------------- SCREENSHOT
screenshot-directory=/Users/perfectidiot/Desktop

These settings have changed here and there over time but never have I ever seen ANY player that handled this scene from Avengers:IW this good like Infuse does here. That out of the way, here are the comparisons between Infuse and MPV. Take a look:

MPV: Obvious clipping aside you'll notice that the overall image is too orange-ish/yellow-ish

Infuse: No clipping anywhere that I could find and more impressively the image is very natural looking. I've matched it with OTT stream of disneyplus and the stream looked almost identical to this, so I think whatever Infuse is doing here, whatever magic algorithm/tweak they have concocted, it's doing the job right. Please compare all these images side by side to see the orangish hue.

MPV

Infuse

MPV

Infuse

MPV: On dark scenes, you'll notice that the image from mpv looks a little less sharp, bit washed out and less contrasty compared to Infuse.

Infuse

Instead of Infuse, I'll continue to use MPV as my primary player cause it's lightweight, way faster and customisable. If anyone can help me achieve the same level of HDR results as my Infuse app, I'd be very grateful.

[qyot27]

c9f6c45?

[haasn]

You might be interested in https://code.videolan.org/videolan/libplacebo/-/merge_requests/155

Or rather, if you can help test it, it'd be a great way to fast-track those changes into libplacebo (and then mpv, via #8799)

[sovon]

c9f6c45?

Apologies but I'm a code illiterate. I'm a designer and figuring out how to write the config files and how to change between various options in MPV were about the most difficult thing I had to do in years. I wouldn't know how to read what you've linked, let alone figuring out what to do with it.

[sovon]

You might be interested in https://code.videolan.org/videolan/libplacebo/-/merge_requests/155

Or rather, if you can help test it, it'd be a great way to fast-track those changes into libplacebo (and then mpv, via #8799)

I can put in the effort and spend time testing whatever is needed but I don't think I understand the technicality all that much. Is there something easier that I can follow?

[Argon-]

c9f6c45?

Apologies but I'm a code illiterate. I'm a designer and figuring out how to write the config files and how to change between various options in MPV were about the most difficult thing I had to do in years. I wouldn't know how to read what you've linked, let alone figuring out what to do with it.

He's (presumably) asking whether your version of mpv contains this piece of code. The precise version of your mpv binary would allow to deduce this.

[sovon]

He's (presumably) asking whether your version of mpv contains this piece of code. The precise version of your mpv binary would allow to deduce this.

Oh. I just added my MPV build information on the top of the opening post.

[Argon-]

According to this info your mpv contains the mentioned commit and you could try with tone-mapping=bt.2390 if you haven't already.

[sovon]

I didn't before but I just did now and the results are dim, literally.

The Dimmest

icc-profile-auto
hwdec=videotoolbox
tone-mapping=bt.2390
tone-mapping-param=0.65

Medium Dim

hwdec=videotoolbox
tone-mapping=bt.2390
tone-mapping-param=0.65

Infuse

[NSQY]

https://mpv.io/manual/master/#options-gamut-clipping might be the problem in this case, I have found that along with bt.2390/hdr-compute-peak chroma can get verrry clipped.

[haasn]

@NSQY oh yes indeed, I just discovered that same bug in libplacebo but haven't fixed it for vo_gpu yet, 1 sec

[haasn]

Fixed the issue I mentioned but on second though I'm not convinced it's actually relevant for you, since it only triggered with HDR outputs (not SDR outputs).

[Robot-DaneelOlivaw]

FWIW, tone-mapping-desaturate and tone-mapping-desaturate-exponent also control desaturation for overly bright colors.

[sovon]

FWIW, tone-mapping-desaturate and tone-mapping-desaturate-exponent also control desaturation for overly bright colors.

This is from two years ago but may still be relevant. May be.
Please see this: #6405 (comment)

[NSQY]

Fixed the issue I mentioned but on second though I'm not convinced it's actually relevant for you, since it only triggered with HDR outputs (not SDR outputs).

I may have been conflating problems.

With sources that have a strong chrominance component, gamut clipping can get a little bizarre https://slow.pics/c/8Mvmb77e
This occurs with all algorithms, so it's not just a BT.2390 problem.

Edit: It is, however, much worse with hdr-compute-peak + BT.2390 - but not only with just gamut-clipping https://slow.pics/c/aKSEbw6n

[Doofussy2]

Try disabling compute peak. hdr-compute-peak=no

Oh, you're on apple. I don't think that's supported.

[Doofussy2]

I wonder if lowering the --target-peak might help, here? The default is 203, and the display may not be that high?

Try --target-peak=190

[sovon]

I wonder if lowering the --target-peak might help, here? The default is 203, and the display may not be that high?

Try --target-peak=190

The higher the number, the dimmer the image.

hwdec=videotoolbox
tone-mapping=reinhard
tone-mapping-param=0.65
target-peak=300

target-peak=190

target-peak=130

[Doofussy2]

The higher the number, the dimmer the image.

Yes, that's why I suggested lowering it. I should have been clear in intending you to try the default and not reinhard. bt.2390 is the default and the default peak is 203. So try bt.2390 with a lower target peak. And remember that bt.2390 isn't tunable so it doesn't have a mapping parameter.

So all you need to do to test, is use one line: target-peak=190 or whatever number you wish to test. This is just to see if you attain the brightness that you are looking for. You could also test it with gamut-clipping=no.

[sovon]

hwdec=videotoolbox
tone-mapping=bt.2390
target-peak=150
gamut-clipping=no

In my opening post I mentioned 2 issues - one was clipping and the other was orange-ish hue. WIth the above setting, the orange-ish hue is almost gone while target-peak=150 is set to achieve approximate brightness. The clipping however is still present.

MPV - image from the opening post w/ orange-ish hue (reinhard + tone-mapping param 0.65)

MPV -bt.2390 - orange-ish hue gone but clipping is there

Infuse

[Doofussy2]

Now you'll have to test haasn's new commit addressing the gamut clipping.

[haasn]

@sovon What you might actually want is to add this

tone-mapping-desaturate=1.0
tone-mapping-desaturate-exponent=0.0

[sovon]

mpv 0.33.1 Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
 built on Mon Apr  5 10:16:25 PDT 2021
FFmpeg library versions:
   libavutil       56.51.100 (runtime 56.70.100)
   libavcodec      58.91.100 (runtime 58.134.100)
   libavformat     58.45.100 (runtime 58.76.100)
   libswscale      5.7.100 (runtime 5.9.100)
   libavfilter     7.85.100 (runtime 7.110.100)
   libswresample   3.7.100 (runtime 3.9.100)
FFmpeg version: 4.4

hwdec=videotoolbox
tone-mapping=bt.2390
target-peak=150
gamut-clipping=no
tone-mapping-desaturate=1.0
tone-mapping-desaturate-exponent=0.0

Like #6405 (comment) tone-mapping-desaturate is blowing the highlights.
MPV: blue-purple becomes white-ish purple and highlight data is lost.

hwdec=videotoolbox
tone-mapping=bt.2390
target-peak=150
gamut-clipping=no
###tone-mapping-desaturate=1.0
tone-mapping-desaturate-exponent=0.0

MPV: Only with tone-mapping-desaturate-exponent=0.0 the results look just a bit better, less highlight data is lost and the shift in the purple isn't as drastic as the one above.

Infuse: As a reference.

These are all of course done in my existing mpv version that comes from brew upgrade and not that commit @Doofussy2 had mentioned. I am not sure how to do that.

[Doofussy2]

So here's something out of left field. With:

hdr-compute-peak=no
brightness=1
tone-mapping-desaturate-exponent=0.7
gamma=2

I get this:

That's pretty damn close to the Infuse shot, except for the increased flaring in the center.

I have noticed there is darkening in all things HDR. Whether I tone-map or passthrough, the image is always slightly darker than it should be. For all content, I have a general correction of brightness=1 and gamma=2. I think there must be a gamma variation.

@haasn it looks to me like that what you're working on with the blackpoint and not setting a gamma curve, would correct this?

[sovon]

I may be wrong but in my personal experience I have NEVER been able to get a satisfied result by tweaking brightness or contrast. It may work on a movie I'm watching at that moment but it will definitely ruin other films for me later on. With brightness increased by even just 1 unit, I can clearly tell where the letterbox starts, and I dark scenes never come out as good. Same thing happens for contrast as well, +/- whatever that is.

[Doofussy2]

Don't adjust the contrast.

[sovon]

I don't do either. I leave brightness, contrast, gamma & saturation untouched and advise others to do the same. To me these settings may help in unique circumstances such as where the display panel itself isn't as good or where the ICC color profile isn't great or has a limited LAB plot (enhancing the look and feel by bumping up the saturation a bit etc. is only temporary) But that is VERY rare and I never meddle with superficial tweaks to keep it as close as to the filmmaker's vision as possible.

In order to get a reference to the "filmmaker's vision", I take screenshots from bluray.com and compare my image to those for that exact frame. Sadly these OTT platforms like netflix or apple TV doesn't allow me to take screenshots so I have to compare windows but I've found that the effort is worth it.

For brightness and contrast, I found out the issue the hard way. I was watching Mandalorian; when Mando was in that spider cave, MPV looked so dim compared to the safari version of disneyplus even though reinhard, param-0.65 was there. I modified the brightness contrast and gamma of MPV to match the safari window and it was great for the rest of the season but a week later I was watching Riddick and omg I've never seen a washed out pitch black scene like that before. Because I have seen the film in theatre and remember it was actually pitch black, at first I thought my monitor broke but then I saw the mpv config and figured out why it was like that.

[Doofussy2]

Yes, that's where blackpoint/gamma/trc comes in. Which is the PR that haasn linked to and why I made reference to it.

https://code.videolan.org/videolan/libplacebo/-/merge_requests/155

Unfortunately, my build environment doesn't allow me to apply patches, or I would be testing the work with libplacebo. Hopefully haasn will complete it in the not too distant future. I'm very much looking forward to putting it through it's paces.

[Robot-DaneelOlivaw]

I think it's hard to judge whether a tone-mapped screenshot is dim or not without knowing each others' screen brightness. If you care about brightness precision, you might want to measure peak brightness of your screen and set target-peak accordingly.

[Doofussy2]

I think it's hard to judge whether a tone-mapped screenshot is dim or not without knowing each others' screen brightness.

All you have to do is put both images next to each other on the same display. Which ostensibly, posting them on here, does.

What I found was, when I juxtaposed the video using passthrough to my Samsung Q80R with how mpv tone-maps with the default settings, was that it was very similar (clipping not withstanding). The 'darkness' was comparative, which is what I would want. Why obtain HDR media if you just want to strip it and remake it the same as the SDR mastering? Just use the SDR version, and you'll have exactly what you want. I want the deep color and the expanded light range. If you don't want HDR, don't use HDR. So once the blackpoint is correctly applied with haasn's PR, I think we'll be golden.

[sovon]

I think it's hard to judge whether a tone-mapped screenshot is dim or not without knowing each others' screen brightness.

You are right. Independently across different devices that comparison would definitely be weird. But if you open two images of the exact same frame and toggle between them with your keyboard maybe, the differences would be drastically pronounced. From there it's only a matter of choice - which one you prefer to the other. As a reference we can compare frames from established forums or from sources that we know are sure to produce accurate results (such as bluray.com screenshot/video quality review section or maybe from a web OTT service such as Apple TV or DIsneyPlus or HBO Max etc.). So you can compare your rendered image with theirs and easily find out what is different in your media consumption device.

Why obtain HDR media if you just want to strip it and remake it the same as the SDR mastering? Just use the SDR version, and you'll have exactly what you want.

Well, I for one would gladly rent an SDR master for sure because I don't have a HDR TV. I, like many others, do all my media consumption on my desktop computer which although has a 4K HDR monitor, that hdr functionality remains mostly unused. Implementation of HDR in macOS or Windows is abysmal to say the least and no one can work on hdr mode because the gamut gets destructively compressed (at least for designers who export sRGB for web). That would be madness. So many people would just use the SDR mode of their monitor.

Also they don't make 4K UHD blurays SDR mastered. Every single UHD bluray I ever saw is HDR mastered. And I personally am lazy to rip it myself and store the 90GB data on my storage, so from time to time, I might obtain a small 4K encode from a popular release group in the internet, which 99.9% of the time is an HDR HEVC encode. So it's not very practical or easy to get SDR version of a 4K HDR mastered release and as such people need to convert HDR to SDR on their own.

But I should to make the point again that I didn't raise this issue here in github for my personal use. Infuse is doing wonderfully although it costs a little too much but I am happy with what I have. My intention was to show it to the community, so that the same quality / features can be more publicly and openly available. An ideal media player should be able to handle the weirdness of the media file, irrespective of source. An end user who might not be aware of all these nitty gritties, should ideally still be able to consume a piece of content in its full glory (or as close to the filmmaker's vision as possible).

[Doofussy2]

An ideal media player should be able to handle the weirdness of the media file, irrespective of source. An end user who might not be aware of all these nitty gritties, should ideally still be able to consume a piece of content in its full glory (or as close to the filmmaker's vision as possible).

Aside from the clipping, mpv does tone-map the way I believe it should. It should closely mimic HDR, as best it can, and not try to reproduce SDR. That was my point.

And I have a lot of 4K SDR media. It's very common.

[hooke007]

vf=format=gamma=gamma2.4

Do not do this if you watch HDR videos and use icc profile at the same time. Just to ruin the color.

target-trc=gamma2.4

This option will be overrided by icc.

[amariami]

This option will be overridden by ICC.

Thanks, I gotcha

I read a reference from here
BT.1886
Compact ICC Profiles from GitHub saucecontrol
and
User:Colin/BrowserTest

[amariami]

Set the 3D-LUT size --icc-3dlut-size=<r>x<g>x<b> could solve the problem?
#--icc-profile=auto
#--icc-profile=C:\cru-1.4.2\MPV-32-bits\sRGB IEC 61966-2.4.icm
--icc-profile=C:\cru-1.4.2\MPV-32-bits\sRGB IEC 61966-2.1.icm
--icc-intent=0 #[0 perceptual, 1 (default) relative colorimetric, 2 saturation, 3 absolute colorimetric]
--icc-3dlut-size=343x343x343 #[2 to 512[
--scaler-lut-size=7 #[4 to 10]

#--icc-3dlut-size=1000x1000x1000 #impossible exceed number
#--scaler-lut-size=10

#--icc-3dlut-size=729x729x729 #impossible exceed number
#--scaler-lut-size=9

#--icc-3dlut-size=512x512x512
#--scaler-lut-size=8

#--icc-3dlut-size=343x343x343
#--scaler-lut-size=7

#--icc-3dlut-size=216x216x216
#--scaler-lut-size=6

#--icc-3dlut-size=125x125x125
#--scaler-lut-size=5

#--icc-3dlut-size=64x64x64
#--scaler-lut-size=4

I see reference in here;

1. 3D LUTs: https://affinityspotlight.com/article/1d-vs-3d-luts/#3d-luts
   For a 64x64x64 3D LUT, we would calculate 64^3, which equals 262,144 values. That’s still not enough to cover the range of 10-bit footage, though—to get that number, we would calculate 1024^3, which equals 1,073,741,824. Quite a difference and the resulting file size of a LUT that contained this many values would be around 4GB. This is prohibitive for both storage and performance considerations since caching and reading a LUT file this large would be difficult: therefore, a 3D LUT would never realistically contain this many values. LUT designed for 8-bit precision that contained every possible value would still need 16,777,216 values (calculated from 256^3), resulting in a file size of around 65MB.

2. Three-Dimensional LUTs: https://developer.nvidia.com/gpugems/gpugems2/part-iii-high-quality-rendering/chapter-24-using-lookup-tables-accelerate-color
   Whereas a 1D LUT requires only 4 elements to sample 4 locations per axis, the corresponding 3D LUT requires 4^3 = 64 elements. Beware of this added dimensionality; 3D LUTs grow very quickly as a function of their linear sampling rate.
   Limitations of 3D LUTs: The color transform must be reasonably continuous as sparsely sampled data sets are ill-suited to represent discontinuous transformations, If smoothly interpolating over the sampled transform grid yields unacceptable results, lookup tables are not the appropriate acceleration technique.

[sovon]

Is there anything else I can test out? (any new development to this perhaps)

[haasn]

@sovon You could help test out libplacebo master, which got a complete overhaul of the HDR code very recently.

Those changes will make their way into mpv itself with #8799.

[sovon]

@sovon You could help test out libplacebo master, which got a complete overhaul of the HDR code very recently.

How do I test it with mpv?

[Doofussy2]

We need a kind person to make a test build for us 😁 Sadly, until it's merged, I can't build with it.

[sovon]

With my previous configuration, every time I played a SDR film in mpv, I used to get a warning, something like - "icc profile... too high... sanity.. etc". With the current tone-mapping=bt.2390 I don't see that anymore.
Anyways, just chiming in. Infuse is high maintenance af, not suitable at all for click and play. For my personal use too, I hope mpv gets this big new update soon you guys have been talking about.

[sovon]

We need a kind person to make a test build for us 😁 Sadly, until it's merged, I can't build with it.

Did something happen yet?

[Dudemanguy]

This is a pre --vo=gpu-next issue and so many of the variables have changed since then that I will close this issue (left plenty of other HDR issues open still so feel free to complain in there).