BD-Folder gets corrupted when converting Dolby Vision m2ts or mkv with merged single layer BL+EL+RPU

[HoffmannTom]

Hello,
I have issues with creating a BD-Folder with tsMuxer.
I already have a suitable m2ts-File with 4k/UHD/HEVC Video and a TrueHD Audio with AC3-Core + Atmos.
The m2ts file plays well (e.g. in VLC) and I wanted to convert it into a BluRay-Folder so I can play it with a standalone player.
After converting the m2ts-file into a BD-Folder, neither the folder, nor the m2ts file will play anymore (PowerDVD crashes, VLC doesn't show video).
The m2ts-file has still the same size, but it won't play anymore. Replacing the m2ts-file within the STREAM-folder with the original file, doesn't work either. The BD-folder is not playable.

I tried the last nightly build from 2022-04-25-02-56-30 in order to have the latest fixes.

How can I track down the issue? Any help is appreciated.

I stripped the movie down to a 140 MB file and can reproduce the issue. Stripping it down further (cutting), corrupts the audio file unfortunately.

Thanks in advance!

[jcdr428]

@HoffmannTom please give the link for the 140MB BDMV faulty folder.
Has there been any old tsMuxer release working with this sample ?

[HoffmannTom]

Hello @jcdr428 ,
the m2ts file was truncated with tsMuxeR_2.6.12 before. The resulting m2ts was playing fine.
But the following converted BD-folder was broken too.

The conversion to BD-folder was done with the recent nightly build, mentioned above.
Here is the link: https://temp.sh/PcLMV/BDMV.7z (3 days valid)
PW: tsMuxeR

If I can assist somehow, just let me know.

Thanks in advance for taking a look at it!

[HoffmannTom]

Just a short update:
I tried all processing steps (truncating, extract THD, add AC-3 core, mux to m2ts, create BD) again.
Same result with 2.6.12 and the mentioned nighly build 2022-04-25-02-56-30.
m2ts file works, created BD-folder was not playing and m2ts-file inside broken too.

[jcdr428]

@HoffmannTom can you please also provide the original m2ts. I need to compare both hevc streams.

[HoffmannTom]

Hello @jcdr428
of course. Everything which helps. Here is the source-m2ts: https://temp.sh/KjmPd/00000_source.7z (3 days valid)
Same PW as first file.

MediaInfo shows a difference in video HDR format:
Src:
Dolby Vision, Version 1.0, dvhe.07.06, BL+EL+RPU, Blu-ray compatible / SMPTE ST 2094 App 4, Version 1, HDR10+ Profile B compatible

After putting into BD-folder:
SMPTE ST 2094 App 4, Version 1, HDR10+ Profile B compatible

So dolby vision turned to HDR10+. BL+EL+RPU seems to be substreams of Dolby vision. Maybe this is causing the faulty video(?)

Greetings, Thomas

[HoffmannTom]

Short update:
I extracted the BL stream of Dolby vision and created a blu-ray folder with txMuxer. That worked well and produced a playable folder.
I also tried with another Dolby vision video with a different DV profile (Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU).
This showed the same problem when converting to blu-ray folder (video corrupted).

Thus the issue seems to be related to Dolby vision (at least with profiles 7 + 8) and using "blu-ray-folder" mode.
As muxing works perfectly when using m2ts-mode, I hope the issue solve.

If I can support somehow, just let me know.
Thanks!

[HoffmannTom]

Hello @jcdr428
I can see that the DOVI Header in the generated m2ts file is missing in blu-ray mode.
It is skipped on purpose, as far as I can read the source code.
In hevcStreamReader.cpp there is a method getTSDescriptor(..) containing 2x the condition:
if (!blurayMode && m_hdr->isDVRPU) ...

So the DOVI-Header is not written on purpose when creating a blu-ray disc. I don't know why(?)
The BL+EL+RPU streams are probably still within the m2ts-file because the size didn't shrink.

I copied the DOVI-Header from the source file to the m2ts-file within the STREAM-folder (via HEX-Editor).
Afterwards, the m2ts-file was playable again but the blu-ray folder was not playing e.g. in PowerDVD.
PowerDVD crashed when opening the blu-ray folder. Maybe the meta-files contain some kind of checksum
or need some additional information about dolby vision(?)

I hope this helps to narrow down the issue.

Greetings,
Thomas

[jcdr428]

@HoffmannTom the DOVI header is not written in Blu-raymode because the DV mode is detected from the mpls/clpi: same as original UHD Blu-rays.

I downloaded both versions, I'll have a look when I have time.

[HoffmannTom]

Thanks for the information! 😊
It is quite hard to find readable documentation about all the involved formats.

[HoffmannTom]

BDInfo shows a hidden video track in the playlist. Maybe it's a hint in the right direction.
The Playlist only contains Audio and Subtitles. The video-stream might be missing in the playlist (?)

[jcdr428]

@HoffmannTom a DV Blu-ray m2ts is composed of two hevc streams, one which contains the non-DV (BL) BaseLine Layer, the second which contains the DV (EL) Enhanced Layer and the RPU.

The two were fusioned in your original into a single hevc stream containing all BL+EL+RPU, via "hacking" tools e.g. Makemkv, so that they can be read on android players having a single DV chip. You cannot create a Blu-Ray m2t from this original, you have to first re-separate the BL and EL+RPU into two hevc streams.

[HoffmannTom]

Hello @jcdr428
thank you very much for looking up the cause!
I already separated the three layers in three files. How can I proceed with these two video files?
Can I just add the two streams in tsMuxer and it will create a DV/HDR blu-ray out of these two streams?
Is the RPU-file with the meta-information obsolete?
Thank you in advance!

[jcdr428]

@HoffmannTom normally yes, if you recreated the two original BL and EL+RPU streams, you can successfully mux them with tsMuxer.
The double to single to double layer process shouldn't have changed anything in the RPU.

[HoffmannTom]

Hello @jcdr428
the muxed file is playing on my Sony-Player. Unfortunately, it only shows 8 Bit Color-Depth on TV.
Blu-Ray Player and AV-Receiver both support DV, TV supports HDR10.
Playing with Kodi+Shield shows 10 Bit. So currently I am not sure if there is something wrong with my setup / settings.
If you have any idea, how I can check, whether the Blu-Ray folder is properly set to DV, just drop a line.
The only tools I found are from the manufacturers and more affordable for millionairs :)

The only thing I have left otherwise would be a little enhancement:
txMuser shows a warning if adding a TrueHD audio without AC-3 core. This helps a lot.
If it's possible, it would be great to have a similar warning, if merged DV-streams are detected which are not compatible to BluRay, as far as I got to know.

Thank you very much for your support and efford!
Thomas

[jcdr428]

@HoffmannTom I can add the warning, this should be easy to implement.

[HoffmannTom]

@jcdr428
Great :)
The same problem happens for BL+RPU streams.
The above case was BL+EL+RPU.

[jcdr428]

@HoffmannTom I've pushed the fix with the warning, please check if ok.

[HoffmannTom]

Hello @jcdr428
I tested with different containers / videos:

1. mkv: Dolby Vision, Version 1.0, dvhe.07.06, BL+EL+RPU
   Warning! MLP codec is not standard for BD disks, the disk will not play in a Blu-ray player.

2. mkv: Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU
   Warning! Dolby Vision Double Layer Single Tracks are not standard for BD disks, the disk will not play in a Blu-ray player.

3. m2ts: Dolby Vision, Version 1.0, dvhe.07.06, BL+EL+RPU
   Warning! Dolby Vision Double Layer Single Tracks are not standard for BD disks, the disk will not play in a Blu-ray player.

4. mkv: V_MPEG4/ISO/AVC
   no warning

Looks good to me!

I would have one last question:
I also have a video with "Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU"
I can remove the RPU and everything works great with tsMuxer.
BL+RPU now shows a warning in tsMuxer.
Do you know any way how to preserve the RPU data but still be BluRay compliant?
As the video doesn't have an EL stream, I don't know if the RPU data can be used / muxed in some way to preserve DV.

Thanks for your quick help and adjustments!
Thomas

[jcdr428]

The only Blu-Ray compliant DV is double track 07.06 with BL in one track and EL+RPU in the other. Any other profile might not be read by the DV chip.

[HoffmannTom]

Thanks for your support and adjustments!

[quietvoid]

The only Blu-Ray compliant DV is double track 07.06 with BL in one track and EL+RPU in the other. Any other profile might not be read by the DV chip.

Apparently not anymore: https://forum.makemkv.com/forum/viewtopic.php?p=122075#p122075

[HoffmannTom]

Being possible doesn't necessarily mean it's compliant ;)
Let's see if there is more information in the future about that. Thanks for the link!

[jcdr428]

Anybody can share a sample ? It seems a fake to me: which distributor in its right mind would keep the x265 encoding parameters in the stream ?

[quietvoid]

Anybody can share a sample ? It seems a fake to me: which distributor in its right mind would keep the x265 encoding parameters in the stream ?

Here's the first 5s trimmed using tsmuxer (video only): https://0x0.st/omsn.7z
I'm not sure how else the original folder can be trimmed down.

Some authoring companies do use x265 and keep parameters, for example Fidelity in Motion did The Elephant Man for Studio Canal, and that still has the params.

[jcdr428]

@quietvoid could you please post a sample ( say 10MB) of the original m2ts ? Just cut the m2ts with an hex editor eg HxD -select all after 00A0 0000, hit delete and save under new name.

Edit: scratch that, I could obtain a sample of the StudioCanal Elephant Man. And indeed the encoding parameters are in the stream:

General
ID                                       : 0 (0x0)
Complete name                            : BDMV\STREAM\00020.m2ts
Format                                   : BDAV
Format/Info                              : Blu-ray Video
File size                                : 73.1 GiB
Overall bit rate mode                    : Variable
Maximum Overall bit rate                 : 109 Mb/s

Video #1
ID                                       : 4113 (0x1011)
Menu ID                                  : 1 (0x1)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : 36
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 24.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Writing library                          : x265 3.2+20-a4e060a44839:[Windows][GCC 6.3.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=6 / numa-pools=16,16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=28500 / level-idc=51 / high-tier=1 / uhd-bd=1 / ref=5 / no-allow-non-conformance / repeat-headers / annexb / aud / hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=1 / keyint=24 / gop-lookahead=0 / bframes=3 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=24 / lookahead-slices=0 / scenecut=10 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / amp / max-tu-size=32 / tu-inter-depth=3 / tu-intra-depth=3 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / tskip / nr-intra=0 / nr-inter=0 / constrained-intra / no-strong-intra-smoothing / max-merge=5 / limit-refs=0 / no-limit-modes / me=3 / subme=4 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / weightb / no-analyze-src-pics / deblock=-2:-2 / no-sao / no-sao-non-deblock / rd=6 / selective-sao=0 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=5.00 / psy-rdoq=50.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=78000 / qcomp=0.60 / qpstep=4 / stats-write=1 / stats-read=0 / slow-firstpass / vbv-maxrate=82000 / vbv-bufsize=99000 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / no-cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=2 / chromaloc-bottom=2 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.01 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qp
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2

Video #2
ID                                       : 4117 (0x1015)
Menu ID                                  : 1 (0x1)
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : 36
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 24.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Writing library                          : x265 3.2+20-a4e060a44839:[Windows][GCC 6.3.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=5 / numa-pools=16,16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=28500 / level-idc=51 / high-tier=1 / uhd-bd=1 / ref=4 / no-allow-non-conformance / repeat-headers / annexb / aud / hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=1 / keyint=24 / gop-lookahead=0 / bframes=3 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=24 / lookahead-slices=4 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=cbr / bitrate=500 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=500 / vbv-bufsize=550 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=2 / chromaloc-bottom=2 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.01 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qp
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2

This is DV profile 07.

[jcdr428]

@quietvoid are these the original .mpls/.clpi in your link, or are these the ones from tsMuxer ?

[quietvoid]

@quietvoid are these the original .mpls/.clpi in your link, or are these the ones from tsMuxer ?

They are from tsMuxer.

[jcdr428]

Can I please have the original Lamb BDMV (without the m2ts files) ? I'll see whether the DV flags are set.

[quietvoid]

Sure, there you go: https://0x0.st/omox.7z
The main playlist should be 00003.mpls

[jcdr428]

So the mpls definitely does not show any DV layer in the STN / the SubPath / the ExtensionData, as if it were a standard HDR10.
There are indeed some RPU nals in the hevc stream, but IMO the standalone Blu-ray players should read the BD-ROM as a standard HDR10.