VideoToolBox hwaccel color problem

[typcn]

Use FFmpeg with videotoolbox hwaccel

Use the same FFmpeg to build mpv

The video only have green and red color

LOG（Removed some uselessu info）

➜  build git:(master) ✗ ./mpv ~/Movies/*.mp4 -hwdec=auto -vo=opengl-hq -v
[cplayer] Command line options: '/Users/typcn/Movies/4457031-1hd.mp4' '-hwdec=auto' '-vo=opengl-hq' '-v'
[cplayer] mpv git-392ae68 (C) 2000-2015 mpv/MPlayer/mplayer2 projects
[cplayer]  built on Mon Sep  7 03:54:38 CST 2015
[cplayer] ffmpeg library versions:
[cplayer]    libavutil       55.0.100
[cplayer]    libavcodec      57.0.100
[cplayer]    libavformat     57.0.100
[cplayer]    libswscale      4.0.100
[cplayer]    libavfilter     6.0.100
[cplayer]    libswresample   2.0.100
[cplayer] ffmpeg version: git-2015-09-06-63974bd
[cplayer]
[cplayer] Configuration: ./waf configure --enable-static-build --enable-libmpv-shared --disable-vda-hwaccel --enable-videotoolbox-hwaccel --enable-videotoolbox-gl
[cplayer] List of enabled features: apple-remote asm atomic-builtins atomics av-pix-fmt-mmal av-version-info avcodec-chroma-pos-api avframe-metadata avframe-skip-samples bsd-fstatfs build-date cocoa coreaudio cplayer debug-build dlopen encoding fchmod gl gl-cocoa glob iconv jpeg lcms2 libass libass-osd libav libavdevice libavfilter libdl libm libmpv-shared libswresample lua nanosleep optimize osx-thread-name posix posix-or-mingw posix-spawn pthreads resampler shm static-build subprocess termios tv videotoolbox-gl videotoolbox-hwaccel videotoolbox-vda-gl zlib
[cplayer] Playing: /Users/typcn/Movies/4457031-1hd.mp4
[cplayer] Running hook: ytdl_hook/on_load
[cplayer] Run command: hook-ack, flags=0, args=[on_load]
[file] Opened: /Users/typcn/Movies/4457031-1hd.mp4
[demux] Trying demuxer: disc (force-level: normal)
[demux] Trying demuxer: edl (force-level: normal)
[demux] Trying demuxer: cue (force-level: normal)
[demux] Trying demuxer: rawaudio (force-level: normal)
[demux] Trying demuxer: rawvideo (force-level: normal)
[demux] Trying demuxer: tv (force-level: normal)
[demux] Trying demuxer: libass (force-level: normal)
[demux] Trying demuxer: mkv (force-level: normal)
[demux] Trying demuxer: rar (force-level: normal)
[demux] Trying demuxer: lavf (force-level: normal)
[lavf] Found 'mov,mp4,m4a,3gp,3g2,mj2' at score=100 size=2048.
[lavf] avformat_find_stream_info() finished after 376760 bytes.
[demux] Detected file format: mov,mp4,m4a,3gp,3g2,mj2 (libavformat)
[cplayer]  (+) Video --vid=1 (*) (h264)
[cplayer]  (+) Audio --aid=1 --alang=und (*) (aac)
[vo/opengl-hq] Initializing OpenGL backend 'cocoa'
[vo/opengl-hq] can't find an ambient light sensor
[vo/opengl-hq] GL_VERSION='4.1 NVIDIA-10.5.2 346.02.02f03'
[vo/opengl-hq] Detected desktop OpenGL 4.1.
[vo/opengl-hq] GL_VENDOR='NVIDIA Corporation'
[vo/opengl-hq] GL_RENDERER='NVIDIA GeForce GTX 980 OpenGL Engine'
[vo/opengl-hq] GL_SHADING_LANGUAGE_VERSION='4.10'
[vo/opengl-hq] Loaded functions for 210/builtin.
[vo/opengl-hq] Loaded functions for 210/builtin.
[vo/opengl-hq] Loaded functions for 300/builtin.
[vo/opengl-hq] Loaded functions for 110/GL_EXT_unpack_subimage.
[vo/opengl-hq] Loaded functions for 300/GL_ARB_framebuffer_object.
[vo/opengl-hq] Loaded functions for 300/GL_ARB_vertex_array_object.
[vo/opengl-hq] Loaded functions for 300/GL_ARB_texture_float.
[vo/opengl-hq] Loaded functions for 300/GL_ARB_texture_rg.
[vo/opengl-hq] Loaded functions for 0/GL_APPLE_rgb_422.
[vo/opengl-hq] 16 bit texture depth: 16.
[vo/opengl-hq] Display depth: R=8, G=8, B=8
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Assuming 60.000000 FPS for framedrop.
[vd] Container reported FPS: 23.976107
[vd] Codec list:
[vd]     lavc:h264 - H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
[vd] Opening video decoder lavc:h264
[vd] Probing 'videotoolbox'...
[vo/opengl-hq] Loading hwdec driver 'videotoolbox'
[vd] Trying hardware decoding.
[vd] Selected video codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [lavc:h264]
[ad] Codec list:
[ad]     lavc:aac - AAC (Advanced Audio Coding)
[ad]     lavc:aac_fixed (aac) - AAC (Advanced Audio Coding)
[ad] Opening audio decoder lavc:aac
[ad] Requesting 1 threads for decoding.
[ad] Selected audio codec: AAC (Advanced Audio Coding) [lavc:aac]
[cplayer] Starting playback...
[af] Audio filter chain:
[af]   [in] 48000Hz stereo 2ch floatp
[af]   [out] 48000Hz stereo 2ch floatp
[af]   [ao] 48000Hz stereo 2ch floatp
[ao/coreaudio] requested format: 48000 Hz, stereo channels, floatp
[ao/coreaudio] selected audio output device: USB Audio Device (190)
[ao/coreaudio] converted input channel layout:
[ao/coreaudio] layout: tag: <0>, bitmap: <0>, descriptions <2>
[ao/coreaudio]  - description 0: label <4294967295, 64>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio]  - description 1: label <4294967295, 64>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio] converted input channel layout:
[ao/coreaudio] layout: tag: <0>, bitmap: <0>, descriptions <2>
[ao/coreaudio]  - description 0: label <1, 0>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio]  - description 1: label <2, 1>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio] Latency property 'ltnc': 44 frames
[ao/coreaudio] Latency property 'fsiz': 512 frames
[ao/coreaudio] Latency property 'saft': 22 frames
[ao/coreaudio] audiounit latency [us]: 354
[ao/coreaudio] device latency [us]: 12041
[ao/coreaudio] using soft-buffer of 9600 samples.
[cplayer] AO: [coreaudio] 48000Hz stereo 2ch floatp
[cplayer] AO: Description: CoreAudio AudioUnit
[vd] Pixel formats supported by decoder: videotoolbox_vld yuv420p
[vd] Using hardware decoding (videotoolbox).
[vd] Using container aspect ratio.
[vf] Video filter chain:
[vf]   [vd] 1280x720 videotoolbox auto/auto CL=mpeg2/4/h264
[vf]   [in] 1280x720 videotoolbox bt.709/limited CL=mpeg2/4/h264
[vf]   [out] 1280x720 videotoolbox bt.709/limited CL=mpeg2/4/h264
[cplayer] VO: [opengl-hq] 1280x720 videotoolbox
[cplayer] VO: Description: Extended OpenGL Renderer (high quality rendering preset)
[vo/opengl-hq] Resize: 1280x720
[vo/opengl-hq] Window size: 1280x720
[vo/opengl-hq] Video source: 1280x720 (1280x720)
[vo/opengl-hq] Video display: (0, 0) 1280x720 -> (0, 0) 1280x720
[vo/opengl-hq] Video scale: 1.000000/1.000000
[vo/opengl-hq] OSD borders: l=0 t=0 r=0 b=0
[vo/opengl-hq] Video borders: l=0 t=0 r=0 b=0
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Testing user-set FBO format (0x805b)
[vo/opengl-hq] Create FBO: 16x16
[vo/opengl-hq] Texture for plane 0: 1280x720
[vo/opengl-hq] Reinit rendering.
[vo/opengl-hq] Resize: 1280x720
[vo/opengl-hq] Window size: 1280x720
[vo/opengl-hq] Video source: 1280x720 (1280x720)
[vo/opengl-hq] Video display: (0, 0) 1280x720 -> (0, 0) 1280x720
[vo/opengl-hq] Video scale: 1.000000/1.000000
[vo/opengl-hq] OSD borders: l=0 t=0 r=0 b=0
[vo/opengl-hq] Video borders: l=0 t=0 r=0 b=0
[vo/opengl-hq] Create FBO: 1280x720
[vo/opengl-hq] recompiling a shader program:
[vo/opengl-hq] [  1] vec4 color = texture(texture0, texcoord0);
[vo/opengl-hq] [  2] // color conversion
[vo/opengl-hq] [  3] color = color.gbra;
[vo/opengl-hq] [  4] color.rgb = mat3(colormatrix) * color.rgb + colormatrix_c;
[vo/opengl-hq] [  5] color.a = 1.0;
[vo/opengl-hq] [  6] // scaler pre-conversion
[vo/opengl-hq] [  7] color.rgb = clamp(color.rgb, 0.0, 1.0);
[vo/opengl-hq] [  8] color.rgb = pow(color.rgb, vec3(1.961));
[vo/opengl-hq] [  9] // main scaling
[vo/opengl-hq] shader link log (status=1): WARNING: Output of vertex shader 'texcoord5' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord1' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord2' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord3' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord4' not read by fragment shader
[vo/opengl-hq]
[vo/opengl-hq] Create FBO: 1280x768
[vo/opengl-hq] recompiling a shader program:
[vo/opengl-hq] [  1] #define tex texture0
[vo/opengl-hq] [  2] vec2 pos = texcoord0;
[vo/opengl-hq] [  3] vec2 size = texture_size0;
[vo/opengl-hq] [  4] vec2 pt = vec2(1.0) / size;
[vo/opengl-hq] [  5] // pass 1
[vo/opengl-hq] [  6] vec4 color = vec4(0.0);
[vo/opengl-hq] [  7] {
[vo/opengl-hq] [  8] vec2 dir = vec2(0, 1);
[vo/opengl-hq] [  9] pt *= dir;
[vo/opengl-hq] [ 10] float fcoord = dot(fract(pos * size - vec2(0.5)), dir);
[vo/opengl-hq] [ 11] vec2 base = pos - fcoord * pt - pt * vec2(2);
[vo/opengl-hq] [ 12] vec4 c;
[vo/opengl-hq] [ 13] vec4 c1 = texture(lut, vec2(0.25, fcoord));
[vo/opengl-hq] [ 14] vec4 c2 = texture(lut, vec2(0.75, fcoord));
[vo/opengl-hq] [ 15] float weights[6] = float[](c1.r, c1.g, c1.b, c2.r, c2.g, c2.b);
[vo/opengl-hq] [ 16] // scaler samples
[vo/opengl-hq] [ 17] c = texture(tex, base + pt * vec2(0));
[vo/opengl-hq] [ 18] color += vec4(weights[0]) * c;
[vo/opengl-hq] [ 19] c = texture(tex, base + pt * vec2(1));
[vo/opengl-hq] [ 20] color += vec4(weights[1]) * c;
[vo/opengl-hq] [ 21] c = texture(tex, base + pt * vec2(2));
[vo/opengl-hq] [ 22] color += vec4(weights[2]) * c;
[vo/opengl-hq] [ 23] c = texture(tex, base + pt * vec2(3));
[vo/opengl-hq] [ 24] color += vec4(weights[3]) * c;
[vo/opengl-hq] [ 25] c = texture(tex, base + pt * vec2(4));
[vo/opengl-hq] [ 26] color += vec4(weights[4]) * c;
[vo/opengl-hq] [ 27] c = texture(tex, base + pt * vec2(5));
[vo/opengl-hq] [ 28] color += vec4(weights[5]) * c;
[vo/opengl-hq] [ 29] }
[vo/opengl-hq] shader link log (status=1): WARNING: Output of vertex shader 'texcoord5' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord1' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord2' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord3' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord4' not read by fragment shader
[vo/opengl-hq]
[vo/opengl-hq] Dither to 8.
[vo/opengl-hq] recompiling a shader program:
[vo/opengl-hq] [  1] #define tex texture0
[vo/opengl-hq] [  2] vec2 pos = texcoord0;
[vo/opengl-hq] [  3] vec2 size = texture_size0;
[vo/opengl-hq] [  4] vec2 pt = vec2(1.0) / size;
[vo/opengl-hq] [  5] // pass 2
[vo/opengl-hq] [  6] vec4 color = vec4(0.0);
[vo/opengl-hq] [  7] {
[vo/opengl-hq] [  8] vec2 dir = vec2(1, 0);
[vo/opengl-hq] [  9] pt *= dir;
[vo/opengl-hq] [ 10] float fcoord = dot(fract(pos * size - vec2(0.5)), dir);
[vo/opengl-hq] [ 11] vec2 base = pos - fcoord * pt - pt * vec2(2);
[vo/opengl-hq] [ 12] vec4 c;
[vo/opengl-hq] [ 13] vec4 c1 = texture(lut, vec2(0.25, fcoord));
[vo/opengl-hq] [ 14] vec4 c2 = texture(lut, vec2(0.75, fcoord));
[vo/opengl-hq] [ 15] float weights[6] = float[](c1.r, c1.g, c1.b, c2.r, c2.g, c2.b);
[vo/opengl-hq] [ 16] // scaler samples
[vo/opengl-hq] [ 17] c = texture(tex, base + pt * vec2(0));
[vo/opengl-hq] [ 18] color += vec4(weights[0]) * c;
[vo/opengl-hq] [ 19] c = texture(tex, base + pt * vec2(1));
[vo/opengl-hq] [ 20] color += vec4(weights[1]) * c;
[vo/opengl-hq] [ 21] c = texture(tex, base + pt * vec2(2));
[vo/opengl-hq] [ 22] color += vec4(weights[2]) * c;
[vo/opengl-hq] [ 23] c = texture(tex, base + pt * vec2(3));
[vo/opengl-hq] [ 24] color += vec4(weights[3]) * c;
[vo/opengl-hq] [ 25] c = texture(tex, base + pt * vec2(4));
[vo/opengl-hq] [ 26] color += vec4(weights[4]) * c;
[vo/opengl-hq] [ 27] c = texture(tex, base + pt * vec2(5));
[vo/opengl-hq] [ 28] color += vec4(weights[5]) * c;
[vo/opengl-hq] [ 29] }
[vo/opengl-hq] [ 30] color.a = 1.0;
[vo/opengl-hq] [ 31] // scaler post-conversion
[vo/opengl-hq] [ 32] // color management
[vo/opengl-hq] [ 33] color.rgb = clamp(color.rgb, 0.0, 1.0);
[vo/opengl-hq] [ 34] color.rgb = pow(color.rgb, vec3(1.0/1.961));
[vo/opengl-hq] [ 35] // dithering
[vo/opengl-hq] [ 36] vec2 dither_pos = gl_FragCoord.xy / 64;
[vo/opengl-hq] [ 37] float dither_value = texture(dither, dither_pos).r;
[vo/opengl-hq] [ 38] color = floor(color * 255 + dither_value + 0.5 / (64 * 64)) / 255;
[vo/opengl-hq] shader link log (status=1): WARNING: Output of vertex shader 'texcoord5' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord1' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord2' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord3' not read by fragment shader
[vo/opengl-hq] WARNING: Output of vertex shader 'texcoord4' not read by fragment shader
[cplayer] first video frame after restart shown
[cplayer] Run command: ignore, flags=9, args=[]
[cplayer] playback restart complete
[statusline] AV: 00:00:00 / 00:07:44 (0%) A-V:  0.000
[statusline] AV: 00:00:00 / 00:07:44 (0%) A-V:  0.000
[vo/opengl-hq] recompiling a shader program:
[vo/opengl-hq] [  1] // OSD (libass)
[vo/opengl-hq] [  2] vec4 color = vec4(ass_color.rgb, ass_color.a * texture(osdtex, texcoord).r);
[vo/opengl-hq] [  3] // color management
[statusline] AV: 00:00:05 / 00:07:44 (1%) A-V:  0.000

Hackintosh with GTX980 , iMac 5K with M295X , Macbook with Intel Iris Pro , All can be reproduced .

[typcn]

Sorry, I have build FFmpeg and mpv with both VDA + VideoToolBox support , It works now ..

This only happen if VDA is disabled

[pigoz]

What was disabled? If there is a bug in mpv or FFmpeg we can fix it.

[typcn]

Build mpv and ffmpeg with --disable-vda --enable-videotoolbox , the video will only have green and red color .

If build with --enable-vda --enable-videotoolbox , the video play correctly.

[pigoz]

Ok thanks, I will look into it.

That's really damn odd. Does videtoolbox send the wrong CV pixel format in this case?

[typcn]

Sorry I can't find the pixel format related code,

But I think it's not fallback to VDA .

if build with --disable-vda --enable-videotoolbox , play with -hwdec=videotoolbox , I can see the "VTDecoderXPCService" is using CPU , the video only has green and red color .

if build with --enable-vda --enable-videotoolbox , play with -hwdec=videotoolbox , I can also see the "VTDecoderXPCService" is using CPU , and the video playback is ok , no color problem .

A sample file might help too.

[typcn]

Sorry for late response , I have built mpv/master , the color is correct now.(old hwdec has color problem with every file)