undesirable color convertsion to YCbCr with X2RGB10 (A2RGB10)

[MartinPulec]

Hi, I'm encountering a problem with patched FFmpeg and oneVPL when using pixel format X2RGB10/A2RGB10. Something is performing unwanted RGB->YUV conversion but pretending that the output stream is in RGB. I'd like to have a 10-bit RGB HEVC, anyways. Steps to reproduce:

ffmpeg -f lavfi -i smptebars -pix_fmt x2rgb10le -t 1 -c:v hevc_qsv rgb10.mp4
ffplay rgb10.mp4    # <- colors are shifted here; it has been converted to YUV but displayed as RGB
# mediainfo '--Output=Video;%ColorSpace%' rgb10.mp4  # confirms RGB

I am using this repo submodule FFmpeg with patches applied. iGPU is the one integrated on Intel i7-1260P. Upstream graphics libraries are installed as described here.

I've tried also "bgra" pixfmt but it says it is unsupported. vuyx and xv30le were OK (but encoding YCbCr obviously).

[MartinPulec]

Just for illustration how it should look:

and how the output actually looks:

[MartinPulec]

AFAIK, the opposite problem holds also for hevc_qsv decoder - if input HEVC stream is a RGB, the decoder decompresses the data depending on bit-depth into XV36, XV30 or VUYX pixfmt (thus YCbCr) but no actual color conversion happens. The decompressed data are original RGB but presented in YCbCr pixel format.

[xhaihao]

I've tried also "bgra" pixfmt but it says it is unsupported.

bgra should be supported but you should boot your Linux kernel with HuC support. According to https://github.com/intel/media-driver/blob/master/docs/media_features.md#supported-encoding-input-format-and-max-resolution, HuC is required for RGB input.

[MartinPulec]

Thanks a lot, I didn't know about this stuff at all, I'll give it a try.

Anyways, perhaps more interesting for our users would to keep/compress 10-bit RGB directly, without CS conversion - should it work? The rightmost column of this table says 'N' for all "Inputs'. But if Input means encode, it contradicts what I've seen - the RGB->YUV conversion was actually performed, but the stream itself identifies as internally RGB.

[wenbinc-Bin]

I can reproduce the problem and I am working on it.

[MartinPulec]

@wenbinc-Bin thanks a lot!

@xhaihao I can confirm that 8-bit RGB (bgra) works correctly in the HuC mode with patched FFmpeg when set AVCodecContext::low_power == 1.

[wenbinc-Bin]

The cause is that ffmpeg pass sRGB in MatrixCoefficient to VPL but VPL doesn't encode video as RGB. It convert frame to YUV. However VPL still set this flag in stream header. FFmpeg will treat this stream as RGB but actually it is YUV.

[alatteri]

Hi @wenbinc-Bin .... I'm not sure from your reply if this is something your team is going to be able to fix or not.
Thanks.

The cause is that ffmpeg pass sRGB in MatrixCoefficient to VPL but VPL doesn't encode video as RGB. It convert frame to YUV. However VPL still set this flag in stream header. FFmpeg will treat this stream as RGB but actually it is YUV.

[wenbinc-Bin]

I will submit patch to fix this.

[wenbinc-Bin]

I submitted a patch to upstream: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230213055545.535005-1-wenbin.chen@intel.com/

[MartinPulec]

Great, thanks for that. I've quickly tested it and it works, the that encoded stream has now correct colors when decoded. Not a big deal but I guess that comment in patch is inverted (RGB gets converted to YCbCr, not vice versa)?

[wenbinc-Bin]

Thank you for pointing it out.

[wenbinc-Bin]

patch is merged: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230213084926.543371-1-wenbin.chen@intel.com/.