Luminance is incorrect for HDR videos taken on iPhone

[low-batt]

System and IINA version:

• macOS 12?
• IINA 1.3.0

Expected behavior:
When playing HDR video taken on iPhone in IINA on a MacBook 16 2021 (M1 Max) the picture matches what is seen with Quicktime.

Actual behavior:
The luminance is inappropriate, resulting in a catastrophic loss of details.

Quicktime:

IINA:

Steps to reproduce:
Take a video with an iPhone 13 Pro and play the video in IINA.

The issue is more noticeable in indoor clips and/or at higher brightness levels.

This issue was reported by @jesec in discussions about PR #3701, the commit that added HDR playback support in 1.3.0. This was also reported in the IINA Telegram Group.

• MPV does not have this problem.

How often does this happen?
Everytime.

[low-batt]

This is my understanding of the situation. As this is outside of my area of expertise, if someone replies with a correction, believe them, not me.

The expectation is that this behavior is due to the use of the Apple OpenGL renderer which is not being updated by Apple. Solving this requires transitioning to Metal.

IINA uses a library from mpv for video and audio support. This uses the OpenGL graphics API provided by macOS. The industry is transitioning from OpenGL to Vulkan, OpenGL's successor.

Instead of transitioning to Vulkan, Apple has transitioned to Metal. From the macOS Mojave 10.14 Release Notes:

Open GL and Open CL

Deprecations

The APIs in the OpenGL and OpenCL frameworks are deprecated and remain present for compatibility purposes. Transition to Metal if your app is using OpenGL or OpenCL.

Apple has deprecated OpenGL in favor of their proprietary Metal API. This is a problem for software that is not dedicated to Apple hardware. To stay current multi-platform software must now support both Vulkan and Metal. That requires additional code that then needs to be maintained. Defects need to be fixed for both Vulkan and Metal.

MoltenVK attempts to solve this by providing a mapping from Vulkan to Metal. The intent is that software developers can code to the Vulkan API and the application will be able to run on Apple hardware and take advantage of Metal.

Although IINA is Apple only, mpv is multi-platform. This means mpv has to transition to Vulkan and use MoltenVK for Vulkan support on macOS and this support will need to be available to applications using libmpv.

I believe this mpv issue mpv-player/mpv#5571 tracks the work required for Vulkan on macOS. Changes for that issue are in the mpv PR mpv-player/mpv#7482. A more recent experiment with getting this to work can be found in libplacebo issue haasn/libplacebo#111

Once this mpv work has been completed then IINA will need to be updated to use Vulkan/Metal on Macs that support it.

[ottob]

maybe the code in this branch could be used? https://github.com/rcombs/mpv/tree/mac_vulkan

[galad87]

iPhone HDR videos are HLG + Dolby Vision 8.4 + ambient viewing environment metadata (defined in ITU-T H.274), so to play back them "properly" you'll have to honour the ambient viewing metadata too.