hwcontext_vaapi: Add support for vaGetDisplayWin32 initialization path

[sivileri]

Starting from libva v2.17, there's a new display node called libva-win32, allowing support for libva on Windows OS. A Windows VAAPI driver based on Direct3D 12 Video APIs is also available as part of Mesa 22.3.0 release or later, via their VA frontend and gallium d3d12 driver.

The changes in this PR add support for hwcontext_vaapi VAAPI initialization using vaGetDisplayWin32, also allowing for device (adapter) selection via "-device" command line argument. With this change, it's possible to run VAAPI workloads on FFMpeg on Windows. For example, using the Windows driver mentioned above, H264/HEVC/VP9/AV1 8/10bit Decode and H264/HEVC 8/10bit encode as well as some video processing effects are supported on ffmpeg/vaapi.

Basic test instructions

1. Download libva and mesa vaon12 VA driver
2. Rename .nupkg to .zip and uncompress the file.
3. From build\native<arch>\bin>, copy vaon12_drv_video.dll, dxil.dll, va.dll, va_win32.dll to c:\bin (or any preferred directory)
4. Add c:\bin to your PATH environment user variable
5. Set the following user environment variables

    // Prefix of VA driver dll name
    LIBVA_DRIVER_NAME=vaon12
    // Path to directory containing vaon12_drv_video.dll
    LIBVA_DRIVERS_PATH=C:\bin

6. Build ffmpeg from this PR source code with --enable-vaapi and copy the produced binaries to c:\bin
   a. You can find the associated dependencies lib/ and include/ folders on the same package described in (1).
7. Run ffmpeg/vaapi commands as usual like for example from a powershell terminal:

VAAPI H264 Decode + Encode

ffmpeg.exe -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device 0 -i input_720p.h264 -vcodec h264_vaapi output.h264 -loglevel trace

VAAPI H264 Decode + Encode with HW accelerated scale VPP effect

ffmpeg.exe -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device 0 -i input_720p.h264 -vf 'scale_vaapi=w=1280:h=720' -vcodec h264_vaapi output.h264 -loglevel trace

[sivileri]

I don't seem to have permissions to add reviewers, so tagging them here. Could a mantainer please add them to reviewers? Thanks!

@xhaihao @feiwan1 @xuguangxin @galinart

[sivileri]

The configure argument "--enable-vaapi" should be added to the intel/ci/windows to add coverage for the new code here, but I don't seem to have access to the pipelines to add that, how could I help adding this argument flag?

[dvrogozh]

I would add some description to commit message including short note how to get working driver and few cmdline examples.

[tong1wu]

If enable-vaapi and enable-libvpl are both specified, it will have compile error with this PR. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L38. Since there's no va/va_drm.h on Windows and CONFIG_VAAPI now is true, compile errors are introduced.

Should be very careful with not only vaapi itself but also vaapi related parts such as libvpl vulkan opencl.

[sivileri]

If enable-vaapi and enable-libvpl are both specified, it will have compile error with this PR. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L38. Since there's no va/va_drm.h on Windows and CONFIG_VAAPI now is true, compile errors are introduced.

Should be very careful with not only vaapi itself but also vaapi related parts such as libvpl vulkan opencl.

Thanks for pointing this out and for your patience, just coming back to the office. I'd be happy to include extra commits to fix these issues in this PR.

Looks like qsv_internal.h:38 should not need to include va_drm.h as vaGetDisplayDRM (the only defined thing in that header) is only used in ffmpeg\libavutil\hwcontext_vaapi.c which includes va_drm.h guarded by #if HAVE_VAAPI_DRM? Then creating a VAAPI device context should be transparent to the actual VAAPI backend (ie. X11, DRM, Win32, etc) by calling av_hwdevice_ctx_create(..., AV_HWDEVICE_TYPE_VAAPI, ...)?

For the other places that use CONFIG_VAAPI, they look fine to me as they refer to texture formats and generic VAAPI concepts like VASurfaceIDs and VADisplayIDs common across all VAAPI backends (X11, DRM, Win32, etc). There are some parts in vulkan that use CONFIG_VAAPI with DRM, but they do inside a CONFIG_LIBDRM ifdef.

Does this make sense to you?

[sivileri]

I would add some description to commit message including short note how to get working driver and few cmdline examples.

I've updated the PR description also with some basic testing steps. Please let me know if you have any issues or questions.

[sivileri]

I don't seem to have permissions to re-request for all reviewers after new changes, could a mantainer please add them to reviewers? Thanks!

@xhaihao @feiwan1 @xuguangxin @galinart @tong1wu

[tong1wu]

If enable-vaapi and enable-libvpl are both specified, it will have compile error with this PR. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L38. Since there's no va/va_drm.h on Windows and CONFIG_VAAPI now is true, compile errors are introduced.
Should be very careful with not only vaapi itself but also vaapi related parts such as libvpl vulkan opencl.

Thanks for pointing this out and for your patience, just coming back to the office. I'd be happy to include extra commits to fix these issues in this PR.

Looks like qsv_internal.h:38 should not need to include va_drm.h as vaGetDisplayDRM (the only defined thing in that header) is only used in ffmpeg\libavutil\hwcontext_vaapi.c which includes va_drm.h guarded by #if HAVE_VAAPI_DRM? Then creating a VAAPI device context should be transparent to the actual VAAPI backend (ie. X11, DRM, Win32, etc) by calling av_hwdevice_ctx_create(..., AV_HWDEVICE_TYPE_VAAPI, ...)?

For the other places that use CONFIG_VAAPI, they look fine to me as they refer to texture formats and generic VAAPI concepts like VASurfaceIDs and VADisplayIDs common across all VAAPI backends (X11, DRM, Win32, etc). There are some parts in vulkan that use CONFIG_VAAPI with DRM, but they do inside a CONFIG_LIBDRM ifdef.

Does this make sense to you?

If enable-vaapi and enable-libvpl are both specified, it will have compile error with this PR. https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L38. Since there's no va/va_drm.h on Windows and CONFIG_VAAPI now is true, compile errors are introduced.
Should be very careful with not only vaapi itself but also vaapi related parts such as libvpl vulkan opencl.

Thanks for pointing this out and for your patience, just coming back to the office. I'd be happy to include extra commits to fix these issues in this PR.

Looks like qsv_internal.h:38 should not need to include va_drm.h as vaGetDisplayDRM (the only defined thing in that header) is only used in ffmpeg\libavutil\hwcontext_vaapi.c which includes va_drm.h guarded by #if HAVE_VAAPI_DRM? Then creating a VAAPI device context should be transparent to the actual VAAPI backend (ie. X11, DRM, Win32, etc) by calling av_hwdevice_ctx_create(..., AV_HWDEVICE_TYPE_VAAPI, ...)?

For the other places that use CONFIG_VAAPI, they look fine to me as they refer to texture formats and generic VAAPI concepts like VASurfaceIDs and VADisplayIDs common across all VAAPI backends (X11, DRM, Win32, etc). There are some parts in vulkan that use CONFIG_VAAPI with DRM, but they do inside a CONFIG_LIBDRM ifdef.

Does this make sense to you?

Yes it looks like we don't have to include va_drm.h for qsv_internal.h.

My other concerns:

1. Previously we regarded VAAPI as Linux only and this assumption appears in a lot of places. For example, https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L27 might need a new name. https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_qsv.c#L2123. This will make AV_HWDEVICE_TYPE_VAAPI as qsv's child_device_type when you don't explicitly specify child_device_type. Previously the default should be AV_HWDEVICE_TYPE_D3D11VA.

2. Besides qsv, opencl and vulkan have vaapi related code inside(CONFIG_VAAPI for vulkan and HAVE_OPENCL_VAAPI_INTEL for opencl). Could you please enable vaapi with them to see if there's other compile or logic problems?

I think after that we still need to run some tests in our CI to make sure there's no unexpected behaviors during execution.

[sivileri]

@tong1wu

Yes it looks like we don't have to include va_drm.h for qsv_internal.h.

I've added a new commit for removing this include.

My other concerns:

1. Previously we regarded VAAPI as Linux only and this assumption appears in a lot of places. For example, https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/qsv_internal.h#L27 might need a new name.

In another new commit, I renamed it to AVCODEC_QSV_VAAPI_SESSION_HANDLE. Please note that ff_qsv_set_display_handle sets some driver default names and AFAIK those are not available on Windows yet. The other places look well to me, as they just refer to a generic VAAPI device/session and are not assuming the backend (ie. win32, x11, drm, etc). Is the (QSV on top of VAAPI using the vaon12 driver) scenario included in the CI tests you mentioned below?

https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/hwcontext_qsv.c#L2123. This will make AV_HWDEVICE_TYPE_VAAPI as qsv's child_device_type when you don't explicitly specify child_device_type. Previously the default should be AV_HWDEVICE_TYPE_D3D11VA.

Great catch. I've swapped the if/else cases so it stays VAAPI as default for linux, and stays DXVA2/D3D11_VA default in Windows too even when VAAPI is present.

2. Besides qsv, opencl and vulkan have vaapi related code inside(CONFIG_VAAPI for vulkan and HAVE_OPENCL_VAAPI_INTEL for opencl). Could you please enable vaapi with them to see if there's other compile or logic problems?

I looked at that code and, for vulkan it seems ok as DRM assumptions for map/unmap are guarded by #ifdef LIBDRM. For opencl, looks good also, as all the logic is delegated onto the cl_intel_va_api_media_sharing extension without assuming backend (ie. win32, x11, drm, etc).

I successfully built vaapi, opencl, libvpl and vulkan together. For opencl, this fix is needed intel/libva#680 as I was missing an exported function vaDisplayIsValid.

I think after that we still need to run some tests in our CI to make sure there's no unexpected behaviors during execution.

Sounds good, are the qsv/opencl/vulkan scenarios with VAAPI included in this CI tests?