"Could not find matching colorspace" for some H.264 videos using vdpau, x11 and xv

[sutol]

mpv and mplayer display error messages for certain videos when using the vdpau, x11 and xv outputs, but not when using opengl (or opengl-hq).

Example with vdpau:

Playing [Mazui] Hyōka/[Mazui]Hyouka-01v2[F2B20F65].mkv.
Detected file format: Matroska
[stream] Video (+) --vid=0 (avc1/V_MPEG4/ISO/AVC)
[stream] Audio (+) --aid=0 () (MP4A/A_AAC)
[stream] Subs (+) --sid=0 () (ASS/S_TEXT/ASS)
Opening video filter: [screenshot]
Selected video codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [libavcodec]
Selected audio codec: Advanced Audio Coding [libavcodec]
AO: [pulse] 48000Hz 2ch s16le (2 bytes per sample)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
Unsupported format 444p14le
Unsupported format 444p14be
Unsupported format 444p12le
Unsupported format 444p12be
Unsupported format 422p14le
Unsupported format 422p14be
Unsupported format 422p12le
Unsupported format 422p12be
Unsupported format 444p14le
Unsupported format 444p14be
Unsupported format 444p12le
Unsupported format 444p12be
Unsupported format 422p14le
Unsupported format 422p14be
Unsupported format 422p12le
Unsupported format 422p12be
[swscaler @ 0x7f460f886100]using unscaled yuv420p10le -> yuv420p special converter
VO: [vdpau] 1280x720 => 1280x720 yv12
[vdpau] Got display refresh rate 60.020 Hz.
[vdpau] If that value looks wrong give the -vo vdpau:fps=X suboption manually.
AV: 1.0 (00:00:01) / 1539.7 (00:25:39) (0%) A-V: 0.000

Versus opengl-hq:

Playing [Mazui] Hyōka/[Mazui]Hyouka-01v2[F2B20F65].mkv.
Detected file format: Matroska
[stream] Video (+) --vid=0 (avc1/V_MPEG4/ISO/AVC)
[stream] Audio (+) --aid=0 () (MP4A/A_AAC)
[stream] Subs (+) --sid=0 () (ASS/S_TEXT/ASS)
Opening video filter: [screenshot]
Selected video codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [libavcodec]
Selected audio codec: Advanced Audio Coding [libavcodec]
AO: [pulse] 48000Hz 2ch s16le (2 bytes per sample)
VO: [opengl-hq] 1280x720 => 1280x720 420p10le
AV: 1.0 (00:00:00) / 1539.7 (00:25:39) (0%) A-V: -0.000

This file, for which mpv detects the same codecs does not have this problem:

Playing Yojou-han Shinwa Taikei/[J-pwq]The_Tatami_Galaxy-03[9EA3A184].mkv.
Detected file format: Matroska
[stream] Video (+) --vid=0 (avc1/V_MPEG4/ISO/AVC)
[stream] Audio (+) --aid=0 --alang=jpn () (MP4A/A_AAC)
[stream] Subs (+) --sid=0 --slang=eng () (ASS/S_TEXT/ASS)
Opening video filter: [screenshot]
Selected video codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 [libavcodec]
Selected audio codec: Advanced Audio Coding [libavcodec]
AO: [pulse] 48000Hz 2ch s16le (2 bytes per sample)
VO: [vdpau] 1280x720 => 1280x720 yv12
[vdpau] Got display refresh rate 60.020 Hz.
[vdpau] If that value looks wrong give the -vo vdpau:fps=X suboption manually.
AV: 1.4 (00:00:01) / 1379.9 (00:22:59) (0%) A-V: -0.000

It's the same on mplayer2, but the mpv output is more verbose.
Video card: GeForce 8600 GT
NVIDIA driver version: 310.19

[ghost]

These messages are harmless.

Explanation: you're probably using Libav, and that doesn't support these pixel formats. Also, the video you're playing is 10 bit, which is not natively supported by vdpau. So the video filter chain tries to convert it to a supported format. It does so by trying higher quality formats to lower quality formats. Formats like 444p14le are naturally included in the list of attempted formats, because it's obviously higher quality than 420p8 (aka yv12), and if vf_scale finds that this format is not supported, it will cause this warning about unsupported pixel formats (even if the format would actually not be used).

But I agree that it shouldn't print these useless warnings. I'll think about what's the best way to silence them.

[sutol]

As I have seen the (less verbose) error message for a long time while the files play seemingly fine, I figured it was harmless, but I never found an explanation for it. I mainly opened the issue to find out what's going on.
So when vdpau/x11/xv play a 10bit video and try to convert it, is there any difference in quality compared to opengl, even if unnoticeable? Or is it completely transparent?

[ghost]

So when vdpau/x11/xv play a 10bit video and try to convert it, is there any difference in quality compared to opengl, even if unnoticeable?

Yes, there will be a difference. I'm not sure how noticable it is in general. And even with yv12 there will be a difference, because everything does YUV->RGB conversion slightly differently.

With vdpau/xv, libswscale will be used to convert 10 bit to 8 bit. I think this uses dithering by default, although I'm not quite sure.

With --vo=opengl, the conversion is done within a shader. Default settings don't use dithering (they just cut off the extra precision), while --vo=opengl-hq does. Also, there's some chance that --vo=opengl will be slightly faster than the other outputs, because the conversion is done on the GPU. Not quite sure about that, though, it probably depends.

With --vo=x11, all software conversion has to be done by libswscale. And it uses the fastest (lowest quality) libswscale settings to do that. The quality is probably the worst of all VOs. It shouldn't be actually used, it's just a fallback.

Dithering is quite visible if you take a screenshot of the mpv window with an external tool (not mpv's builtin screenshot function), and turn up the gamma or brightness. You see patterns where you'd normally see banding.

[ghost]

I just disabled the error messages.

This was actually caused by vf_screenshot. You don't need vf_screenshot to take screenshots, because all VOs support taking screenshots natively. In fact, vf_screenshot is completely unused now, because VO screenshot support is preferred over the filter.

[sutol]

I see, thanks for the explanation.