Incorrect colorspace (video looks green/purple-ish)

[shouya]

System and software version:

• macOS 10.12.4
• mpv 0.24.0

Actual behavior:

I'm playing this video (attached) with IINA, and its color doesn't look correct. Then I tried to play this video with plain mpv, same issue. I think it might be a bug with mpv's colorspace detection.

The video renders in an incorrect color with mpv. Here's a screenshot showing the phenomenon. At this scene the sky should be blue but it looks green in mpv, and the ground should look yellow but it shows purple-ish.

Expected behavior:

Here's what is should look. (This screenshot was taken with QuickTime)

The color is correct with VLC and QuickTime, but is incorrect with IINA and mpv.

• MPV log:

[   0.023][v][cplayer] mpv 0.24.0 (C) 2000-2017 mpv/MPlayer/mplayer2 projects
[   0.023][v][cplayer]  built on Mon Feb 20 19:54:01 HKT 2017
[   0.023][v][cplayer] ffmpeg library versions:
[   0.023][v][cplayer]    libavutil       55.34.101
[   0.023][v][cplayer]    libavcodec      57.64.101
[   0.023][v][cplayer]    libavformat     57.56.101
[   0.023][v][cplayer]    libswscale      4.2.100
[   0.023][v][cplayer]    libavfilter     6.65.100
[   0.023][v][cplayer]    libswresample   2.3.100
[   0.023][v][cplayer] ffmpeg version: 3.2.4
[   0.023][v][cplayer] 
[   0.023][v][cplayer] Configuration: waf configure --prefix=/usr/local/Cellar/mpv/0.24.0 --enable-zsh-comp --enable-libmpv-shared --enable-html-build --confdir=/usr/local/etc/mpv --datadir=/usr/local/Cellar/mpv/0.24.0/share/mpv --mandir=/usr/local/Cellar/mpv/0.24.0/share/man --docdir=/usr/local/Cellar/mpv/0.24.0/share/doc/mpv --zshdir=/usr/local/Cellar/mpv/0.24.0/share/zsh/site-functions
[   0.023][v][cplayer] List of enabled features: any-gl apple-remote asm atomics bsd-fstatfs build-date c11-tls cocoa coreaudio cplayer debug-build dlopen encoding fchmod gcc-tls gl gl-cocoa glob gnuc iconv is_ffmpeg jpeg lcms2 libass libass-osd libav libavcodec libavdevice libbluray libdl libm libmpv-shared lua nanosleep optimize osx-thread-name plain-gl posix posix-or-mingw posix-spawn pthreads shm stdatomic subprocess termios tv uchardet videotoolbox-gl videotoolbox-hwaccel zlib zsh-comp
[   0.025][v][bdmv/bluray] Opening /Applications/IINA.app/Contents/Resources/config/input.conf
[   0.025][v][file] Opening /Applications/IINA.app/Contents/Resources/config/input.conf
[   0.025][v][file] Stream opened successfully.
[   0.026][v][input] Parsing input config file /Applications/IINA.app/Contents/Resources/config/input.conf
[   0.027][v][input] Input config file /Applications/IINA.app/Contents/Resources/config/input.conf parsed: 123 binds
[   0.027][v][global] user path: '/Users/shou/Library/Application Support/com.colliderli.iina/log/2017-4-21-2-42-36_7DAzZe.log' -> '/Users/shou/Library/Application Support/com.colliderli.iina/log/2017-4-21-2-42-36_7DAzZe.log'
[   0.027][v][ytdl_hook] Loading lua script @ytdl_hook.lua...
[   0.027][v][ytdl_hook] loading mp.defaults
[   0.028][v][ytdl_hook] loading @ytdl_hook.lua
[   0.029][v][cplayer] Run command: hook-add, flags=0, args=[on_load, 1, 10]
[   0.029][v][cplayer] Run command: hook-add, flags=0, args=[on_preloaded, 2, 10]
[   0.029][v][cplayer] Done loading @ytdl_hook.lua.
[   0.029][v][autoload] Loading lua script /Applications/IINA.app/Contents/Resources/scripts/autoload.lua...
[   0.030][v][autoload] loading mp.defaults
[   0.032][v][global] user path: '/Applications/IINA.app/Contents/Resources/scripts/autoload.lua' -> '/Applications/IINA.app/Contents/Resources/scripts/autoload.lua'
[   0.032][v][autoload] loading file /Applications/IINA.app/Contents/Resources/scripts/autoload.lua
[   0.034][v][autoload] lua-settings/autoload.conf not found. 
[   0.034][v][cplayer] Done loading /Applications/IINA.app/Contents/Resources/scripts/autoload.lua.
[   0.034][v][cplayer] Set property: vo="opengl-cb" -> 1
[   0.034][v][cplayer] Set property: keepaspect="no" -> 1
[   0.035][v][cplayer] Set property: opengl-hwdec-interop="auto" -> 1
[   0.170][v][opengl-cb] GL_VERSION='4.1 INTEL-10.24.45'
[   0.170][v][opengl-cb] Detected desktop OpenGL 4.1.
[   0.170][v][opengl-cb] GL_VENDOR='Intel Inc.'
[   0.170][v][opengl-cb] GL_RENDERER='Intel(R) Iris(TM) Graphics 6100'
[   0.170][v][opengl-cb] GL_SHADING_LANGUAGE_VERSION='4.10'
[   0.170][v][opengl-cb] Loaded extension GL_APPLE_rgb_422.
[   0.170][v][opengl-cb] 16 bit texture depth: 16.
[   0.170][v][opengl-cb] Reported display depth: R=8, G=8, B=8
[   0.170][v][opengl-cb] Testing FBO format 0x805b
[   0.170][v][opengl-cb] Create FBO: 16x16 (16x16)
[   0.170][v][opengl-cb] Using FBO format 0x805b.
[   0.170][v][opengl-cb] No advanced processing required. Enabling dumb mode.
[   0.170][v][opengl-cb] Loading hwdec driver 'videotoolbox'
[   0.191][v][lavf] avformat_find_stream_info() finished after 152417 bytes.
[   0.191][v][demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[   0.191][v][cplayer] Opening done: /Users/shou/Downloads/demo.mp4
[   0.191][v][find_files] Loading external files in /Users/shou/Downloads/
[   0.194][v][cplayer] Running hook: ytdl_hook/on_preloaded
[   0.194][v][cplayer] Run command: hook-ack, flags=0, args=[on_preloaded]
[   0.194][i][cplayer]  (+) Video --vid=1 (*) (h264)
[   0.194][i][cplayer]  (+) Audio --aid=1 --alang=eng (*) (ac3)
[   0.195][v][vd] Container reported FPS: 25.000000
[   0.195][v][vd] Codec list:
[   0.195][v][vd]     h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[   0.195][v][vd]     h264_vda (h264) - H.264 (VDA acceleration)
[   0.195][v][vd] Opening video decoder h264
[   0.195][v][vd] Probing 'videotoolbox'...
[   0.195][v][vd] Trying hardware decoding.
[   0.195][v][vd] Selected video codec: h264 (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
[   0.196][v][ad] Codec list:
[   0.196][v][ad]     ac3 - ATSC A/52A (AC-3)
[   0.196][v][ad]     ac3_fixed (ac3) - ATSC A/52A (AC-3)
[   0.196][v][ad]     ac3_at (ac3) - ac3 (AudioToolbox)
[   0.196][v][ad] Opening audio decoder ac3
[   0.196][v][ad] Detected 4 logical cores.
[   0.196][v][ad] Requesting 5 threads for decoding.
[   0.196][v][ad] Selected audio codec: ac3 (ATSC A/52A (AC-3))
[   0.196][v][cplayer] Starting playback...
[   0.196][v][af] Audio filter chain:
[   0.196][v][af]   [in] 48000Hz stereo 2ch floatp
[   0.196][v][af]   [out] 48000Hz stereo 2ch floatp
[   0.196][v][af]   [ao] 48000Hz stereo 2ch floatp
[   0.196][v][ao] Trying audio driver 'coreaudio'
[   0.197][v][ao/coreaudio] requested format: 48000 Hz, stereo channels, floatp
[   0.296][v][ao/coreaudio] selected audio output device: Built-in Output (202)
[   0.302][v][ao/coreaudio] input channel layout:
[   0.302][v][ao/coreaudio] layout: tag: <0>, bitmap: <0>, descriptions <2>
[   0.302][v][ao/coreaudio]  - description 0: label <4294967295, 64>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[   0.302][v][ao/coreaudio]  - description 1: label <4294967295, 64>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[   0.302][v][ao/coreaudio] mp chmap: unknown2
[   0.302][v][ao/coreaudio] input channel layout:
[   0.302][v][ao/coreaudio] layout: tag: <0>, bitmap: <0>, descriptions <2>
[   0.302][v][ao/coreaudio]  - description 0: label <1, 0>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[   0.302][v][ao/coreaudio]  - description 1: label <2, 1>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[   0.302][v][ao/coreaudio] mp chmap: stereo
[   0.302][v][ao/coreaudio] Channel layouts:
[   0.302][v][ao/coreaudio]  - stereo
[   0.302][v][ao/coreaudio]  - mono
[   0.302][v][ao/coreaudio] result: stereo
[   0.342][v][ao/coreaudio] Latency property cntl: 672 frames
[   0.342][v][ao/coreaudio] Latency property zisf: 512 frames
[   0.342][v][ao/coreaudio] Latency property tfas: 93 frames
[   0.342][v][ao/coreaudio] audiounit latency [us]: 354
[   0.342][v][ao/coreaudio] device latency [us]: 26604
[   0.343][v][ao/coreaudio] using soft-buffer of 9600 samples.
[   0.343][i][cplayer] AO: [coreaudio] 48000Hz stereo 2ch floatp
[   0.343][v][cplayer] AO: Description: CoreAudio AudioUnit
[   0.343][v][vd] Pixel formats supported by decoder: vda_vld vda videotoolbox_vld yuv420p
[   0.343][v][vd] Codec profile: High (0x64)
[   0.376][v][ffmpeg/video] h264: Reinit context to 1280x720, pix_fmt: videotoolbox_vld
[   0.388][i][vd] Using hardware decoding (videotoolbox).
[   0.388][v][vd] Decoder format: 1280x720 videotoolbox[nv12] ycgco/auto/auto/limited CL=mpeg2/4/h264
[   0.388][v][vf] Video filter chain:
[   0.388][v][vf]   [in] 1280x720 videotoolbox[nv12] ycgco/bt.709/bt.1886/limited CL=mpeg2/4/h264
[   0.388][v][vf]   [out] 1280x720 videotoolbox[nv12] ycgco/bt.709/bt.1886/limited CL=mpeg2/4/h264
[   0.388][i][cplayer] VO: [opengl-cb] 1280x720 videotoolbox[nv12]
[   0.388][v][cplayer] VO: Description: OpenGL Callbacks for libmpv
[   0.388][v][cplayer] set video colors output-levels=0 
[   0.392][v][opengl-cb] Window size: 1360x710
[   0.392][v][opengl-cb] Video source: 1280x720 (1:1)
[   0.392][v][opengl-cb] Video display: (0, 0) 1280x720 -> (0, 0) 1360x710
[   0.392][v][opengl-cb] Video scale: 1.062500/0.986111
[   0.392][v][opengl-cb] OSD borders: l=0 t=0 r=0 b=0
[   0.392][v][opengl-cb] Video borders: l=0 t=0 r=0 b=0
[   0.392][v][opengl-cb] Testing FBO format 0x805b
[   0.392][v][opengl-cb] Create FBO: 16x16 (16x16)
[   0.392][v][opengl-cb] Using FBO format 0x805b.
[   0.392][v][opengl-cb] No advanced processing required. Enabling dumb mode.
[   0.393][v][cplayer] first video frame after restart shown
[   0.393][v][cplayer] starting audio playback
[   0.394][v][cplayer] playback restart complete
[   0.398][v][opengl-cb] recompiling a shader program:
[   0.398][v][opengl-cb] [  1] color.r = 1.000000 * vec4(texture(texture0, texcoord0)).r;
[   0.398][v][opengl-cb] [  2] color.gb = 1.000000 * vec4(texture(texture1, texcoord1)).rg;
[   0.398][v][opengl-cb] [  3] // color conversion
[   0.398][v][opengl-cb] [  4] color.rgb = mat3(colormatrix) * color.rgb + colormatrix_c;
[   0.398][v][opengl-cb] [  5] color.a = 1.0;
[   0.398][v][opengl-cb] [  6] // color mapping
[   0.403][v][opengl-cb] shader link log (status=1): WARNING: Output of vertex shader 'texcoord5' not read by fragment shader
[   0.403][v][opengl-cb] WARNING: Output of vertex shader 'texcoord2' not read by fragment shader
[   0.403][v][opengl-cb] WARNING: Output of vertex shader 'texcoord3' not read by fragment shader
[   0.403][v][opengl-cb] WARNING: Output of vertex shader 'texcoord4' not read by fragment shader
[   0.403][v][opengl-cb] 
[   0.437][v][cplayer] Set property: pause=true -> 1
[   0.549][v][cplayer] Set property: pause=false -> 1
[   0.551][v][cplayer] get video colors contrast 
[   0.551][v][cplayer] get video colors gamma 
[   0.551][v][cplayer] get video colors brightness 
[   0.552][v][cplayer] get video colors hue 
[   0.552][v][cplayer] get video colors saturation 
[   0.562][v][opengl-cb] Window size: 1280x720
[   0.562][v][opengl-cb] Video source: 1280x720 (1:1)
[   0.562][v][opengl-cb] Video display: (0, 0) 1280x720 -> (0, 0) 1280x720
[   0.562][v][opengl-cb] Video scale: 1.000000/1.000000
[   0.562][v][opengl-cb] OSD borders: l=0 t=0 r=0 b=0
[   0.562][v][opengl-cb] Video borders: l=0 t=0 r=0 b=0

Steps to reproduce:

Just play the demo video I provided with mpv.

mpv demo.mp4

Attachment

Decompress this zip file to get the mp4 video for demonstration.

demo.mp4.zip

Related issue

iina/iina#585

[kodawah]

usually a mp4 file is already compressed so there is no need to zip it again
the input colorspace is ycocg: support might be limited, and setting the other color properties (transfer and primaries) might help

[Akemi]

seems like a wrongly tagged file/stream? setting --vf=format=colormatrix=bt.709 fixes it. here ffprobe output

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Volumes/AkemiData/Users/Akemi/Downloads/demo.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp41mp42isom
    creation_time   : 2017-04-20T18:54:42.000000Z
  Duration: 00:00:35.16, start: 0.000000, bitrate: 2523 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, ycgco/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 2098 kb/s, 24.95 fps, 25 tbr, 100 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2017-04-20T18:54:42.000000Z
      handler_name    : Core Media Video
    Stream #0:1(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, stereo, fltp, 384 kb/s (default)
    Metadata:
      creation_time   : 2017-04-20T18:54:42.000000Z
      handler_name    : Core Media Audio
    Side data:
      audio service type: main

[shouya]

Akemi:

Setting --vf=format=rgba also fixes it.

This video file could be a wrongly tagged mp4, but no idea why VLC and QuickTime handle it well.

kodabb: I compressed it in a zip file because GitHub doesn't allow me to upload an mp4 file.

[Akemi]

they might handle it correctly in this case because they don't read the tag and just fall back to a default or assume something based on some heuristic.

[haasn]

I don't think the video is necessarily wrongly tagged; if --vf=format=rgba makes it play correctly then it sounds to me as if vo_opengl doesn't handle YCgCo correctly.

Indeed, -vo vdpau and -vo x11 also play it fine.

[haasn]

Is it possible that the matrix in mp_get_csp_matrix is misordered? Swapping it from

    case MP_CSP_YCGCO: {
        *m = (struct mp_cmat) {
            {{1,  -1,  1},
             {1,   1,  0},
             {1,  -1, -1}},
        };
        break;
    }

to

    case MP_CSP_YCGCO: {
        *m = (struct mp_cmat) {
            {{1,  -1,  1},
             {1,  -1, -1},
             {1,   1,  0}},
        };
        break;
    }

Seems to have done the trick, without breaking -vo vdpau either. Which actually sort of surprises me, because I thought vo_vdpau used the same function to calculate the matrix which it then uploads to vdpau?

Anyway, it seems like this was a bug in the original source code, because wikipedia confirms the “corrected” matrix order:
https://en.wikipedia.org/wiki/YCgCo

It's possible that the original code may have confused YCgCo with YCoCg.

Edit: Never mind, actually wikipedia matches the order in the original code.

[haasn]

The code in question goes back to commit 2cb147a and the order has not been touched since then. Do we have any other test samples for YCgCo files?

Another possibility would be that this file requires an RGB swizzle to be set for vo_opengl.

[mc4man]

you could probably use ffmpeg to create test file of sorts, maybe with a parameter something like
-x264-params colormatrix=YCgCo

The previously posted demo.mp4 above would display ok with either bt.709 or bt.601

[haasn]

Some more investigation:

1. @wm4 claims that swscale doesn't support YCgCo (afahk), but with a vf_scale filter it seemingly plays correctly
2. The verbose output from such a case looks like this:

[vd] Decoder format: 1280x720 yuv420p ycgco/auto/auto/limited CL=mpeg2/4/h264
[vf] Using conversion filter.
[scale] Setting option 'warn' = 'no' (flags = 0)
[ffmpeg] SwScaler: reducing / aligning filtersize 1 -> 4
[ffmpeg] SwScaler: reducing / aligning filtersize 1 -> 4
[ffmpeg] SwScaler: reducing / aligning filtersize 1 -> 1
[ffmpeg] SwScaler: reducing / aligning filtersize 5 -> 4
[ffmpeg] swscaler: bicubic scaler, from yuv420p to rgba using MMXEXT
[ffmpeg] swscaler: 1280x720 -> 1280x720
[ffmpeg] swscaler: using unscaled yuv420p -> rgba special converter
[vf] Video filter chain:
[vf]   [in] 1280x720 yuv420p ycgco/bt.709/bt.1886/limited CL=mpeg2/4/h264
[vf]   [scale] 1280x720 rgba rgb/bt.709/bt.1886/full CL=mpeg2/4/h264 [a]
[vf]   [format] "format.00" 1280x720 rgba rgb/bt.709/bt.1886/full CL=mpeg2/4/h264
[vf]   [out] 1280x720 rgba rgb/bt.709/bt.1886/full CL=mpeg2/4/h264
[cplayer] VO: [opengl] 1280x720 rgba

3. By the looks of things, swscale is literally just treating the file as “yuv420p”. If it really doesn't know YCgCo, then maybe it's applying the BT.601/BT.709 YCbCr matrix instead?
4. After some comparison, I've come to the conclusion that the “flipped” YCgCo matrix from my earlier post (which produces a seemingly-correct output) is suspiciously similar to the YCbCr matrices.
5. It's entirely possible that this file is YCbCr in disguise (mistagged as YCgCo), and that it works in QuickTime etc. because they ignore the YCgCo tag; this combined with the previous fact would explain why it produced a seemingly correct output when I swapped the B/G channels for the YCgCo matrix.
6. The YCgCo matrix in the source code is consistent with the other material I've found for YCgCo, and also round-trips with the inverse conversion matrix I've found on wikipedia and elsewhere.

My overall conclusion is that YCgCo support seems to be fine in mpv, and that this file is either YCgCo with a messed up channel order, or (more likely) YCbCr mistagged as YCgCo, as others have suggested. Due to lack of an authoritative source on what the file is supposed to look like, it's probably impossible to know for certain.

[atomi]

I'm having the same issue with ie. VO: [opengl] 1920x1080 yuv420p on every video I've tried including youtube videos and --vf=format=rgba fixes this.

[shumvgolove]

Linux, RX590, amdgpu driver loaded
Problem still persist.

[cplayer] mpv 0.29.1 Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
[cplayer]  built on Sat Sep 21 11:13:22 UTC 2019
[cplayer] ffmpeg library versions:
[cplayer]    libavutil       56.31.100
[cplayer]    libavcodec      58.54.100
[cplayer]    libavformat     58.29.100
[cplayer]    libswscale      5.5.100
[cplayer]    libavfilter     7.57.100
[cplayer]    libswresample   3.5.100
[cplayer] ffmpeg version: 4.2.1

[ghost]

The sample file in the first post is tagged as ycgco, which if it's unintended, but makes it a broken file. It was explained that other players (and even mpv VOs) display it "correctly" because they incorrectly ignore the colorspace tag, i.e. 2 bugs cancel out each other. A manual workaround was provided for vo_gpu. Nothing to do here.

For all other problems, report a new issue with sample files and full logs, if it persists with mpv 0.30.0.

[haasn]

What chrome and ffplay do is really off-topic.