Color space/primaries/transfer not parsed correctly for HDR video

[leonardyan]

I'm converting an HDR clip with BT.2020 color space and PQ(HDR10) transfer. The source file's HDR-related ffprobe JSON output says:

PS E:\> ffprobe -hide_banner -loglevel warning -select_streams v -print_format json -show_frames -read_intervals "%+#1" -show_entries "frame=color_space,color_primaries,color_transfer,side_data_list,pix_fmt" -i E:\kyoto.original.webm
{
    "frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084"
        }
    ]
}

For the conversion options, I chose 20000kbps 2-pass, modified output filename, and remained everything else as default, including "Remove HDR" as No.
Pass 1 command line, generated by FastFlix:
"C:\Users\25305\AppData\Roaming\FFmpeg\bin\ffmpeg.exe" -y -i "E:/kyoto.original.webm" -max_muxing_queue_size 1024 -map 0:0 -c:v libx265 -pix_fmt yuv420p10le -x265-params ":pass=1:no-slow-firstpass=1" -passlogfile "C:\Users\25305\AppData\Roaming\FastFlix\temp_7t8l2imn\pass_log_file_482a2118f87cd48affc0.log" -b:v 20000k -preset medium -an -sn -dn -f mp4 NUL
Pass 2 command line:
"C:\Users\25305\AppData\Roaming\FFmpeg\bin\ffmpeg.exe" -y -i "E:/kyoto.original.webm" -max_muxing_queue_size 1024 -map 0:0 -c:v libx265 -pix_fmt yuv420p10le -x265-params ":pass=2" -passlogfile "C:\Users\25305\AppData\Roaming\FastFlix\temp_7t8l2imn\pass_log_file_482a2118f87cd48affc0.log" -b:v 20000k -preset medium -map_metadata -1 -map_chapters 0 -map 0:1 -metadata:s:1 title="" -metadata:s:1 handler="" -metadata:s:1 language=eng -c:1 copy "E:/kyoto.2passtest.mp4"
Seems no HDR-related data, other than pixel format, appeared in the commands.
After conversion I played the output file on an HDR-enabled device. The color was incorrect, looked pale. The output file's HDR-related ffprobe JSON output says:

PS E:\> ffprobe -hide_banner -loglevel warning -select_streams v -print_format json -show_frames -read_intervals "%+#1" -show_entries "frame=color_space,color_primaries,color_transfer,side_data_list,pix_fmt" -i E:\kyoto.2passtest.mp4
{
    "frames": [
        {
            "pix_fmt": "yuv420p10le",
            "side_data_list": [
                {
                    "side_data_type": "H.26[45] User Data Unregistered SEI message"
                }
            ]
        }
    ]
}

Color space/primaries/transfer data from the original clip seem to be lost. This problem also appeared in CRF encoding.
Full ffprobe output of both these 2 clips are included in attachments.
input_file_full_ffprobe.txt
output_file_full_ffprobe.txt
Finally I solved this problem with command line ffmpeg:
ffmpeg -i E:/kyoto.original.webm -map 0 -c:v libx265 -x265-params hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:max-cll=0,0 -crf 20 -preset veryfast -pix_fmt yuv420p10le E:/kyoto.crf20-veryfast.mp4
I removed the master-display data because ffmpeg spits out error when I tried to manually specify it.
The output file played well, the color was correct(maybe, I'm not able to tell if there's any subtle color difference between source and output, but absolutly better than the previous output), and in the output file's ffprobe all HDR-related entries were back(including master-display).

[leonardyan]

I can upload the source and output video clip if you need them.

[leonardyan]

FastFlix v3.3.1, ffmpeg and libraries version can be seen in attachments

[cdgriffith]

Interesting, does it not have hdr10 metadata? (I think the issue is I don't currently specify the colorprim and so on if it doesn't have that, even if it is bt2020, so that needs fixed)

Could you give the output of this command:

ffprobe -select_streams v -print_format json -show_frames -read_intervals "%+#1" -show_entries "frame=color_space,color_primaries,color_transfer,side_data_list,pix_fmt" -i E:\kyoto.original.webm

[leonardyan]

Output for source file:

ffprobe version N-99707-g2b702015d8 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9.3-win32 (GCC) 20200320
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-zlib --enable-libxml2 --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-opencl --enable-libvmaf --disable-vulkan --enable-libvorbis --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --enable-ffnvcodec --enable-cuda-llvm --disable-libglslang --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-libs=-lgomp
  libavutil      56. 60.100 / 56. 60.100
  libavcodec     58.111.101 / 58.111.101
  libavformat    58. 62.100 / 58. 62.100
  libavdevice    58. 11.102 / 58. 11.102
  libavfilter     7. 88.100 /  7. 88.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
{
Input #0, matroska,webm, from 'E:\kyoto.original.webm':
  Metadata:
    ENCODER         : Lavf58.62.100
  Duration: 00:02:00.01, start: -0.007000, bitrate: 23905 kb/s
    Stream #0:0(eng): Video: vp9 (Profile 2), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 00:02:00.007000000
    Side data:
      Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2650,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.005000, max_luminance=1000.000000
    Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:02:00.001000000
    "frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084"
        }
    ]
}

[cdgriffith]

Thank you, and I did just check that even without HDR10 metadata, it should still pass that info in. So I am unsure the issue.

If you could share the clip (even a few seconds of it should be enough to test with ffmpeg -t 10 -i E:\kyoto.original.webm -c copy out.webm ) would be grateful. Can use a site like https://www.filedropper.com/ and if don't want to put link here can email it to me at chris@cdgriffith.com

[cdgriffith]

I haven't done any specific updates in 3.4.0 that would change this behavior, but it's worth a try quick try to see if they appear in the command window.

[leonardyan]

Writing these in additional x265 parameters also fixes color problem
hdr=1:hdr10=1:hdr-opt=1:hdr10-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc

[leonardyan]

Input clip 20 secs: http://www.filedropper.com/kyotosample

[leonardyan]

Also a question: If a newer version of FastFlix requires ffmpeg newer than which already exists on one computer, will the user be prompted to upgrade ffmpeg(on Windows)?

[leonardyan]

Using 3.4.0, force HDR10 signaling&HDR10 optimizations&Repeat headers checked, Still no color-space-related data in the commands. Pass 1:
"C:\Users\25305\AppData\Roaming\FFmpeg\bin\ffmpeg.exe" -y -i "E:/kyoto.sample.webm" -max_muxing_queue_size 1024 -map 0:0 -c:v libx265 -pix_fmt yuv420p10le -x265-params "aq-mode=2:repeat-headers=1:hdr10_opt=1:hdr10=0:pass=1:no-slow-firstpass=1" -passlogfile "C:\Users\25305\AppData\Roaming\FastFlix\temp_f7gjudv2\pass_log_file_aad047fdc6c6cbde57b8.log" -b:v 19500k -preset medium -an -sn -dn -f mp4 NUL
Pass 2:
"C:\Users\25305\AppData\Roaming\FFmpeg\bin\ffmpeg.exe" -y -i "E:/kyoto.sample.webm" -max_muxing_queue_size 1024 -map 0:0 -c:v libx265 -pix_fmt yuv420p10le -x265-params "aq-mode=2:repeat-headers=1:hdr10_opt=1:hdr10=0:pass=2" -passlogfile "C:\Users\25305\AppData\Roaming\FastFlix\temp_f7gjudv2\pass_log_file_4e50c82969f182b551fe.log" -b:v 19500k -preset medium -map_metadata -1 -map_chapters 0 -map 0:1 -metadata:s:1 title="" -metadata:s:1 handler="" -metadata:s:1 language=eng -c:1 copy "E:/kyoto.sample.test1.mp4"
Now I'm waiting for encoding to finish.

[leonardyan]

Still no color space data in ffprobe, and no correct color.

[cdgriffith]

Thank you so much for the sample video! It seems webm format stores the information differently than mkv so had to pull it from a different location. Adding a fix for it now.

And to answer the question abot FFmpeg, currently it does not have a way to auto-update. As the default FFmpeg is a nightly build, it would change every day. I do want to add a button to "download newest version" in the GUI and have done work on that front, just want a good way to have a progress bar or something so the user doesn't think it's frozen during the long download.

[leonardyan]

Waiting for final 3.4.1 release & many thanks to your hardworking.

[cdgriffith]

Should be good to go with 3.4.1 let me know if you run into more issues!

Thanks for the help!

[leonardyan]

Cool! Now everything seems OK. But why these master-display data in the generated commands look so weird and don't match with the source file? Pass 1:
"C:\Users\25305\AppData\Roaming\FFmpeg\bin\ffmpeg.exe" -y -i "E:/kyoto.sample.webm" -max_muxing_queue_size 1024 -map 0:0 -c:v libx265 -pix_fmt yuv420p10le -x265-params "aq-mode=2:repeat-headers=1:hdr10_opt=0:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(13249,69)B(3,3)R(17,8)WP(3127,329)L(1000,1):hdr10=1:pass=1:no-slow-firstpass=1" -passlogfile "C:\Users\25305\AppData\Roaming\FastFlix\temp_zia6qmb5\pass_log_file_60689a43e778349ada27.log" -b:v 19500k -preset medium -an -sn -dn -f mp4 NUL
Pass 2 uses the same master-display. Luckily this doesn't affect the final result, its side data are identical with the source.

[cdgriffith]

Good catch, the issue is all videos I have come across usually have the full ratios.

For example:

                {
                    "side_data_type": "Mastering display metadata",
                    "red_x": "35400/50000",
                    "red_y": "14600/50000",
                    "green_x": "8500/50000",
                    "green_y": "39850/50000",
                    "blue_x": "6550/50000",
                    "blue_y": "2300/50000",
                    "white_point_x": "15635/50000",
                    "white_point_y": "16450/50000",
                    "min_luminance": "50/10000",
                    "max_luminance": "40000000/10000"
                },

The video you provided seems to reduce them:

'red_x': '17/25', 'red_y': '8/25', 'green_x': '13249/50000', 'green_y': '69/100', 'blue_x': '3/20', 'blue_y': '3/50',

So I need to do mathz to get them back to the proper ratio before putting them in.

[leonardyan]

The video you provided seems to reduce them:

'red_x': '17/25', 'red_y': '8/25', 'green_x': '13249/50000', 'green_y': '69/100', 'blue_x': '3/20', 'blue_y': '3/50',

Is this another problem related to webm or vp9?

[cdgriffith]

I think it's just different containers / encoders are going to do things slightly differently. And I won't be able to account for them until I see them so if you have any others with issues after this fix, keep sending 'em!

[cdgriffith]

Re-fixed in 3.4.2