Dolby Vision - run as single layer

[jartas]

Hi, after fix 188 I try run Dolby Vision movie, there is problem with colors. I read about this problem and I read that Infuse use fake DV and run only HDR layer 10bit. Could you run only single HDR layer for DV video ?? Thank you

[kingslay]

Can you provide me this demo video. i try

[jartas]

Hi, i download sample from dolby, there is 4K Dolby Vision, problem is that color is pink and incorrectly detects dolby vision.

https://iscc.site/DVProf5_UHD_Dolby_Atmos.mp4

[kingslay]

This video is missing some parameters for video color. So there is something wrong with the displayed color. I also have this problem with vlc, but there is no problem with Apple's quicktime player. I'm sorry that I haven't found a solution for this problem. I will continue to try

[kingslay]

Duplicate of #150

[Alanko5]

@kingslay

Please look at that:

Dolby Vision is implemented in FFmpeg+mpv (IPTPQc2 colorspace plus MMR reshaping, output to PQ surface works, reference with LG C9, that is reference v4 Dolby Vision implementation from Dolby), no support for (F)EL or dynamic metadata.

Do you think this is the solution for us?
https://trac.ffmpeg.org/ticket/7624

HDR displays:
mpv.com --target-trc=pq --target-prim=bt.2020 --vo=gpu-next --gpu-context=d3d11 file.mp4/mkv/ts

SDR gamma 2.2 display:
mpv.com --target-trc=gamma2.2 --target-prim=bt.709 --vo=gpu-next --gpu-context=d3d11 file.mp4/mkv/ts

avsforum :

Since 2 days ago profile 5 and other even HLG profile 8 MMR decode to PQ in d3d11 context in mpv. Reference with LG C9. Congrats to me, haasn and others involved, like quitvoid for patches to ffmpeg.

mpv.com --target-trc=pq --target-prim=bt.2020 --vo=gpu-next --gpu-context=d3d11 file.mp4/mkv/ts

IPTPQc2

[Alanko5]

https://code.videolan.org/videolan/libplacebo/-/issues/197

https://code.videolan.org/videolan/vlc/-/issues/26584

[Alanko5]

@kingslay check this:
https://github.com/quietvoid/vs-nlq

Here the mixing of BL and EL layer is done.
The result is a 12-bit DV image.

[Alanko5]

In your FFMPEG build not have -> dovi_rpu.c
libavcodec/dovi_rpu.c

Inital commit for DolbiVision in FFMPEG
FFmpeg/FFmpeg@fe04033

lavc: Implement Dolby Vision RPU parsing
Based on a mixture of guesswork, partial documentation in patents, and
reverse engineering of real-world samples. Confirmed working for all the
samples I've thrown at it.

Contains some annoying machinery to persist these values in between
frames, which is needed in theory even though I've never actually seen a
sample that relies on it in practice. May or may not work.

Since the distinction matters greatly for parsing the color matrix
values, this includes a small helper function to guess the right profile
from the RPU itself in case the user has forgotten to forward the dovi
configuration record to the decoder. (Which in practice, only ffmpeg.c
and ffplay do..)

Notable omissions / deviations:

• CRC32 verification. This is based on the MPEG2 CRC32 type, which is
  similar to IEEE CRC32 but apparently different in subtle enough ways
  that I could not get it to pass verification no matter what parameters
  I fed to av_crc. It's possible the code needs some changes.
• Linear interpolation support. Nothing documents this (beyond its
  existence) and no samples use it, so impossible to implement.
• All of the extension metadata blocks, but these contain values that
  seem largely congruent with ST2094, HDR10, or other existing forms of
  side data, so I will defer parsing/attaching them to a future commit.
• The patent describes a mechanism for predicting coefficients from
  previous RPUs, but the bit for the flag whether to use the
  prediction deltas or signal entirely new coefficients does not seem to
  be present in actual RPUs, so we ignore this subsystem entirely.
• In the patent's spec, the NLQ subsystem also loops over
  num_nlq_pivots, but even in the patent the number is hard-coded to one
  iteration rather than signalled. So we only store one set of coefs.

[kingslay]

i had upgrade ffmpeg to 5.1. it had dovi_rpu

[Alanko5]

I downloaded the latest version from git and it's not there.

dae3e8d#diff-e4e468a875016df080685ba4caa2a240bbe532939c226b5b349e88f17e41f179

[kingslay]

you can run
nm -m KSPlayer/Sources/Libavcodec.xcframework/ios-arm64_arm64e/Libavcodec.framework/Libavcodec
and search dovi_rpu

[Alanko5]

[kingslay]

commit 5da0776

[kingslay]

not dovi_rpu.h . is dovi_rpu.o

[kingslay]

/KSPlayer/Sources/Libavcodec.xcframework/ios-arm64_arm64e/Libavcodec.framework/Libavcodec:dovi_rpu.o:
(undefined) external _av_buffer_allocz
(undefined) external _av_buffer_create
(undefined) external _av_buffer_replace
(undefined) external _av_buffer_unref
(undefined) external _av_dovi_metadata_alloc
(undefined) external _av_frame_new_side_data_from_buf
(undefined) external _av_free
(undefined) external _av_log
(undefined) external _avpriv_request_sample
---------------- (LTO,CODE) external _ff_dovi_attach_side_data
---------------- (LTO,CODE) external _ff_dovi_ctx_flush
---------------- (LTO,CODE) external _ff_dovi_ctx_replace
---------------- (LTO,CODE) external _ff_dovi_ctx_unref
---------------- (LTO,CODE) external _ff_dovi_rpu_parse
---------------- (LTO,CODE) external _ff_dovi_update_cfg
(undefined) external _ff_golomb_vlc_len
(undefined) external _ff_ue_golomb_vlc_code
---------------- (LTO,CODE) non-external _get_se_coef

[Alanko5]

Then I don't understand why it doesn't work, DV detection and correct matrix setting.

[kingslay]

If DV's detection is correct, will Dolby Vision be displayed on TV without problems? @Alanko5

[Alanko5]

yes, we know it will happen, but it's the first step...

can you advise how to do it?

[Alanko5]

If DV's detection is correct, will Dolby Vision be displayed on TV without problems? @Alanko5

No, if DV dual layer or DV profile 5, the colors are wrong.
If DV profile 8 single layer, colors is OK.

HDR colors are correct.

[Alanko5]

If DV profile 8 single layer, colors is OK.

Not if it has rehhsaping, polinomial or if it has MMR.

do you have any ideas?

[Alanko5]

https://developer.apple.com/av-foundation/Incorporating-HDR-video-with-Dolby-Vision-into-your-apps.pdf

Reading HDR metadata, for renderer/reshaper.

if let sideData = avframe.pointee.side_data.pointee?.pointee {
                        if sideData.type == AV_FRAME_DATA_DOVI_RPU_BUFFER {
                            let rpuBuff = sideData.data.withMemoryRebound(to: [UInt8].self, capacity: 1) { $0 }
                            print("")
                        }
                        if sideData.type == AV_FRAME_DATA_DOVI_METADATA { // AVDOVIMetadata
                            let doviMeta = sideData.data.withMemoryRebound(to: AVDOVIMetadata.self, capacity: 1) { $0 }
                            let header = av_dovi_get_header(doviMeta)
                            let mapping = av_dovi_get_mapping(doviMeta)
                            let color = av_dovi_get_color(doviMeta)
                            print("")
                        }
                        if sideData.type == AV_FRAME_DATA_DYNAMIC_HDR_PLUS { // AVDynamicHDRPlus
                            let hdrPlus = sideData.data.withMemoryRebound(to: AVDynamicHDRPlus.self, capacity: 1) { $0 }.pointee
                            print("")
                        }
                        if sideData.type == AV_FRAME_DATA_DYNAMIC_HDR_VIVID { // AVDynamicHDRVivid
                            let hdrVivid = sideData.data.withMemoryRebound(to: AVDynamicHDRVivid.self, capacity: 1) { $0 }.pointee
                            print("")
                        }
                        
 }

[Alanko5]

VLC implent DOVI

https://code.videolan.org/videolan/libplacebo/-/merge_requests/207/diffs

https://code.videolan.org/videolan/libplacebo/-/merge_requests/208/diffs

https://code.videolan.org/videolan/libplacebo/-/issues/113

[kingslay]

我现在接入了libplacebo。能否提供一个测试url。看下还会不会有问题

[cdguy]

我现在接入了libplacebo。能否提供一个测试url。看下还会不会有问题

I will try to find a video link for you @kingslay

[cdguy]

@kingslay I am downloading movies with different DV profiles. I want to be sure before I post something. I will wait until Monday and let you know about the results

[cdguy]

Hello @kingslay , with the help of @JeremTM93 , he tried quite a few movies and series on AppleTV and we found a correlation when the colours are greenish and purplish.

It's always happening when the Dolby Vision is “profile 5” (more precisely dvhe.05.06 BL+RPU) with bt.1886 gamma encoded in bt.601-525 regardless of the level (limited or full). // you can refer to this post too #348 //

When you open the links (provided via e-mail) via Infuse, the app creates a “fake DV output” and the colours looks normal. The TV info says that the videos are DV.

Please only make changes to “DV Profile 5” because other profiles work normally :)

[cdguy]

Ksplayer

Infuse (the TV says the content is DV, but we all know it is a fake DV)

[cdguy]

when you are done with all updates, can you please update the testflight app; @JeremTM93 is the best person that can run all DV profile tests and let you know if something is wrong

[karelrooted]

@kingslay is performance a issue on Apple TV 4K 2022 ? on Apple TV 4K 2017 , is basically unusable

I also test it on mpv
with vo=videotoolbox-copy & vf=libplacebo, is basically unusable too
with vo=gpu-next , with default profile , there is a lot of frame drop , after change to fast profile, the video play fines though

[cdguy]

@kingslay is performance a issue on Apple TV 4K 2022 ? on Apple TV 4K 2017 , is basically unusable

I also test it on mpv with vo=videotoolbox-copy & vf=libplacebo, is basically unusable too with vo=gpu-next , with default profile , there is a lot of frame drop , after change to fast profile, the video play fines though

I was about to write something similar:

on ios device the colour output is very good however almost every 7-8 seconds I have almost 100 frame drops

[kingslay]

@kingslay is performance a issue on Apple TV 4K 2022 ? on Apple TV 4K 2017 , is basically unusable

I also test it on mpv with vo=videotoolbox-copy & vf=libplacebo, is basically unusable too with vo=gpu-next , with default profile , there is a lot of frame drop , after change to fast profile, the video play fines though

@karelrooted 你的意思是，你对mpv 配置了vo=gpu-next & --profile=sw-fast 之后。视频播放就很流畅了，是不是。

[karelrooted]

@karelrooted 你的意思是，你对mpv 配置了vo=gpu-next & --profile=sw-fast 之后。视频播放就很流畅了，是不是。

yes，with --profile=fast,video play smoothly on Apple TV 4K 2017

If performance is a issue, may be we need to render the decoded frame from videotoolbox directly with libplacebo render(like mpv's gpunext) (ffplay's latest git also has libplacebo render, I did't check if it has videotoolbox and libplacebo interlop)

[kingslay]

目前我用的画面渲染有两个 AVSampleBufferDisplayLayer和CAMetalLayer。所以要解决这个性能问题的话，那有两个技术方案：

1. 增加libplacebo 渲染器。libplacebo渲染器应该是在渲染的时候顺便做了颜色的转化，这样才避免了gpu和cup的内存切换。
2. 只针对profile 5这个格式，把libplacebo=color_primaries=9:color_trc=14的 颜色转化，用metal来实现一下。继续使用CAMetalLayer来进行渲染。

[karelrooted]

目前我用的画面渲染有两个 AVSampleBufferDisplayLayer和CAMetalLayer。所以要解决这个性能问题的话，那有两个技术方案：

1. 增加libplacebo 渲染器。libplacebo渲染器应该是在渲染的时候顺便做了颜色的转化，这样才避免了gpu和cup的内存切换。

2. 只针对profile 5这个格式，把libplacebo=color_primaries=9:color_trc=14的 颜色转化，用metal来实现一下。继续使用CAMetalLayer来进行渲染。

or we can wait for ffmpeg to fix videotoolbox and libplacebo interlop， if "-hwaccel=videotoolbox -hwupload=derive_device=vulkan" works，then performance issue is gone

the libplacebo part that make prpfile 5 color correct is vf=libplacebo=apply_dolbyvision=1， which is enable by default, so just use vf=libplacebo is enough for our purpose

did you test it on Apple TV 4K 2022? does it work on new more powerful device?

[cdguy]

目前我用的画面渲染有两个 AVSampleBufferDisplayLayer和CAMetalLayer。所以要解决这个性能问题的话，那有两个技术方案：

1. 增加libplacebo 渲染器。libplacebo渲染器应该是在渲染的时候顺便做了颜色的转化，这样才避免了gpu和cup的内存切换。
2. 只针对profile 5这个格式，把libplacebo=color_primaries=9:color_trc=14的 颜色转化，用metal来实现一下。继续使用CAMetalLayer来进行渲染。

or we can wait for ffmpeg to fix videotoolbox and libplacebo interlop， if "-hwaccel=videotoolbox -hwupload=derive_device=vulkan" works，then performance issue is gone

the libplacebo part that make prpfile 5 color correct is vf=libplacebo=apply_dolbyvision=1， which is enable by default, so just use vf=libplacebo is enough for our purpose

did you test it on Apple TV 4K 2022? does it work on new more powerful device?

@kingslay asking just in case; have you tried the code suggested above? ("-hwaccel=videotoolbox -hwupload=derive_device=vulkan")

[kingslay]

有的，但是这样是无法运行的。可能要等待ffmpeg支持了才可以

[kingslay]

目前ffmpeg 6.1.1还是不支持把vulkan当作解码器。我看ffmpeg的最新代码，是有开始支持用vulkan来进行渲染了。可以在等下一个版本

[FaiChou]

FFmpeg 7.0 现在支持了吗？

[kingslay]

FFmpeg 7.0的vulkan解码，要求vulkan要是动态库，不能是静态库。并且是naked的动态库。但是苹果要求是要用xcframework格式的动态库，两个有冲突。有个方式是要修改ffmpeg里面的动态库加载。我还在尝试

[anrtiger]

Hello, to have an update to FfMpeg 7, you have to pay $1000, for the first, you will have exclusivity for two months, then, anyone wishing to have this update to 7.0 in their application, must pay $500.

[kingslay]

Hello, to have an update to FfMpeg 7, you have to pay $1000, for the first, you will have exclusivity for two months, then, anyone wishing to have this update to 7.0 in their application, must pay $500.

不用的。在lgpl分支目前已经免费提供了FF mpeg7.0了

[FantasyKingdom]

我看杜比官网显示ios设备原生支持P5，不知道是不是真的
https://professionalsupport.dolby.com/s/article/Apple-iOS-Overview?language=en_US

[kingslay]

苹果可能是支持的。但是苹果没有开放对应的api的话，那我们还是无法正确显示p5.

[FantasyKingdom]

@kingslay k神，请教一个问题，ksplayer使用displayLayer播放杜比P8是不偏色的。请问最终显示的效果是兼容成HDR播放的还是完整P8规格播放的？

[kingslay]

我对这个也不是很清楚。但是我感觉应该是兼容成HDR来播放的。#727 这边有一个问题，提到没有根据动态信息进行调整亮度，并且播放器没有点亮Dolby Vision。

[FantasyKingdom]

FFmpeg 7.0的vulkan解码，要求vulkan要是动态库，不能是静态库。并且是naked的动态库。但是苹果要求是要用xcframework格式的动态库，两个有冲突。有个方式是要修改ffmpeg里面的动态库加载。我还在尝试

@kingslay K神，进展怎么样了，期待中

[cdguy]

FFmpeg 7.0的vulkan解码，要求vulkan要是动态库，不能是静态库。并且是naked的动态库。但是苹果要求是要用xcframework格式的动态库，两个有冲突。有个方式是要修改ffmpeg里面的动态库加载。我还在尝试

@kingslay K神，进展怎么样了，期待中

most of the DV content plays very well under MPV with the Swift wrapper that can be found in github

[kingslay]

FFmpeg 7.0的vulkan解码，要求vulkan要是动态库，不能是静态库。并且是naked的动态库。但是苹果要求是要用xcframework格式的动态库，两个有冲突。有个方式是要修改ffmpeg里面的动态库加载。我还在尝试

@kingslay K神，进展怎么样了，期待中

目前通过ffmpeg的代码可以把这个库改成是静态链接的方式。但是解码用vulkan的话，渲染也要用vulkan。这个改动就比较大了。还没改好。

[caobug]

要自己加filter libplacebo=color_primaries=9:color_trc=14

我已添加过滤但最终会崩溃在 av_buffersink_get_frame_flags 这一行：KSPlayer_vide (7): hit program assert

var url: URL? {
    didSet {
        if let url {
            let options = KSOptions()
            options.videoFilters.append("libplacebo=color_primaries=9:color_trc=14")
            options.hardwareDecode = false
            let res0 = KSPlayerResourceDefinition(url: url, definition: "高清", options: options)
            let asset = KSPlayerResource(name: url.lastPathComponent, definitions: [res0], cover: URL(string: "https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/Big_buck_bunny_poster_big.jpg/848px-Big_buck_bunny_poster_big.jpg"))
            playerView.set(resource: asset)
        } else {
            playerView.resetPlayer()
        }
    }
}

资源：https://kodi.wiki/view/Samples ：Dolby Vision Mystery Box (Profile 5)

绕过验证：In Xcode, in the "Product" Menu, go to "Scheme", then "Edit Scheme...". In the left panel select "Run (Debug)" and in Diagnostic tab uncheck "API Validation" under Metal settings

[kingslay]

debug 会crash，在release就不会crash了。

[caobug]

debug 会crash，在release就不会crash了。

关掉 metal -> api validation，在 debug 下就不会 crash 了

[kingslay]

现在我的app已经解决这个问题了。你可以用最新的testflight包试下。