[QSV][windows/linux]decode pipeline fails on linux but works on windows

[fulinjie]

This issue occurs on linux only in ffmpeg qsv decode, and could be reproducde with any clips(H264/H265).
However, it works on windows.

Since there seems to be no difference in FFmpeg level, it may be relevant with the difference between windows and linux.

cmdline:
ffmpeg -v verbose -init_hw_device qsv=hw -filter_hw_device hw -c:v hevc_qsv -i Tears_400_x265.mp4 -y -vframes 2 out.yuv

Error msg:
[hevc_qsv @ 0x1471fc0] Initialized an internal MFX session using hardware accelerated implementation
[hevc_qsv @ 0x1471fc0] Current input bitstream is not supported by QSV decoder.
[hevc_qsv @ 0x1471fc0] Error initializing the MFX video decoder: unsupported (-3)
Error while decoding stream #0:0: Function not implemented

Analysis:
This error may be relevant with FactoryCORE::CreateCORE and VAAPIVideoCORE::SetHandle.
On Linux, the created CORE is used in MFXVideoDECODE_Init/MFX_Utility::CheckVideoParam without calling SetHandle.

However since windows works well, I'm wondering how MSDK for windows cope with this in MFXInit or MFXVideoDecode_Init?

For example, when allocating video core, there is difference between windows and linux.

MediaSDK/_studio/mfx_lib/shared/src/mfx_session.cpp

Line 854 in 690341f

#if defined(MFX_VA_LINUX)

[dmitryermilov]

Application shall call SetHandle on Linux. Does it call it?
Please refer to manual
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#working-with-va-api-applications

[fulinjie]

Yes, it should be called, and currently it seems it wasn't called in Linux pipeline.

The interesting thing is it works for windows, and that's what I'm curious about.

[dmitryermilov]

Okay. Then I'm 99% sure this is a root cause.
On the past we intentionally didn't implement initialization of vaDisplay inside MSDK library on Linux because on Linux there were many back-ends(X11, Wayland, headless drm, etc) and that time we didn't want to bring this logic inside MediaSDK.

[fulinjie]

Thanks a lot for this information.
And is it possible to implement the initialization inside MSDK for Linux now?

[dmitryermilov]

Unfortunately no (since it's already captured in the manual)

[dvrogozh]

I would say that as of now we still does not want to bring dependency from the backends into mediasdk. Which adapter to initialize and how to initialize is application responsibility. There could potentially be some additional mediasdk level helper library which will assist in adapter selection, but ultimately that's an application choice.

[lizhong1008]

https://patchwork.ffmpeg.org/patch/14898/ has been sent to resolve this issue.

[fulinjie]

Fixed, and closed.
Thanks.