VA-API Encoding not working on Linux (AMD)

[itaranto]

Hardware encoding for H264 is not working, it takes about 1h30m on my Arch Linux partition but only around 30/40min on Windows to encode an 8GB video.
(These numbers are not very precise since I'm relying on shotcut's estimations).

Hardware decoding seems to work on Firefox and video players.

Shotcut Version: ARCH-22.01.30
OS: Arch Linux
DE: i3 (X11)
GPU: AMD Radeon RX 570
CPU: AMD Ryzen 7 2700X

vainfo:

vainfo: VA-API version: 1.13 (libva 2.13.0)
vainfo: Driver version: Mesa Gallium driver 21.3.5 for AMD Radeon RX 570 Series (POLARIS10, DRM 3.44.0, 5.16.8-zen1-1-zen, LLVM 13.0.0)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

I haven't seen any errors in the log either:
shotcut.log

I know there's #1210, but this is different since there's no error message.

I also tried with Sway (Wayland) but encoding fails at some point, I didn't bothered with debugging it since that seems to be addressed in #1210 too.

[sergiomb2]

maybe is about ffmpeg 5 see https://code.videolan.org/videolan/vlc/-/issues/26772

[itaranto]

Hmmm it seems that VLC doesn't support FFmpeg 5, but what that has to do with Shotcut?

For example, in my system (Arch Linux), VLC uses FFmpeg 4.4 while Shotcut uses 5.0.

[ddennedy]

Version 22.03 and later removes a connection_type=x11 parameter passed to FFmpeg, which may have resolved this. This needs retest. Please be aware that the developers do not run arch or maintain or test its package. Also, you need to look at the job's log, and not View > Application Log for any error message. I use an AMD-based Ubuntu 20.04 system where it works.

[itaranto]

Version 22.03 and later removes a connection_type=x11 parameter passed to FFmpeg, which may have resolved this. This needs retest. Please be aware that the developers do not run arch or maintain or test its package. Also, you need to look at the job's log, and not View > Application Log for any error message. I use an AMD-based Ubuntu 20.04 system where it works.

OK, I'll test it, but take into account that I tested it both with X11 (i3wm) and Wayland (Sway).

Beside the time that takes, is there any way I can verify if hardware decoding is actually being used?

[ddennedy]

Forgot to mention: test again on newer version. See the Export job log to tell if it is being used.

[ddennedy]

Here is how the export job log looks on my system - very obvious it is using VA-API:

[AVHWDeviceContext @ 0x7fec94006ac0] Trying to use DRM render node for device 0.
[AVHWDeviceContext @ 0x7fec94006ac0] libva: VA-API version 1.1.0
[AVHWDeviceContext @ 0x7fec94006ac0] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x7fec94006ac0] libva: Trying to open /home/ddennedy/Desktop/Shotcut-22.04/Shotcut.app/lib/va/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x7fec94006ac0] libva: Found init function __vaDriverInit_1_1
[AVHWDeviceContext @ 0x7fec94006ac0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7fec94006ac0] Initialised VAAPI connection: version 1.1
[AVHWDeviceContext @ 0x7fec94006ac0] VAAPI driver: Mesa Gallium driver 20.0.8 for Radeon RX 560 Series (POLARIS11, DRM 3.35.0, 5.4.0-109-generic, LLVM 10.0.0).
[AVHWDeviceContext @ 0x7fec94006ac0] Driver not found in known nonstandard list, using standard behaviour.
[mlt_buffer @ 0x7fec942ab480] w:1280 h:720 pixfmt:nv12 tb:1001/30000 fr:30000/1001 sar:88889/50000
[h264_vaapi @ 0x7fec940020c0] Input surface format is nv12.
[h264_vaapi @ 0x7fec940020c0] Using VAAPI profile VAProfileH264Main (6).
[h264_vaapi @ 0x7fec940020c0] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x7fec940020c0] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x7fec940020c0] RC mode: CQP.
[h264_vaapi @ 0x7fec940020c0] RC quality: 23.
[h264_vaapi @ 0x7fec940020c0] RC framerate: 30000/1001 (29.97 fps).
[h264_vaapi @ 0x7fec940020c0] Using intra and P-frames (supported references: 1 / 0).
[h264_vaapi @ 0x7fec940020c0] Driver does not support some wanted packed headers (wanted 0xd, found 0).
[h264_vaapi @ 0x7fec940020c0] Driver does not support packed sequence headers, but a global header is requested.
[h264_vaapi @ 0x7fec940020c0] No global header will be written: this may result in a stream which is not usable for some purposes (e.g. not muxable to some containers).
[h264_vaapi @ 0x7fec940020c0] Using level 3.1.

[itaranto]

Here is how the export job log looks on my system - very obvious it is using VA-API:

[AVHWDeviceContext @ 0x7fec94006ac0] Trying to use DRM render node for device 0.
[AVHWDeviceContext @ 0x7fec94006ac0] libva: VA-API version 1.1.0
[AVHWDeviceContext @ 0x7fec94006ac0] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x7fec94006ac0] libva: Trying to open /home/ddennedy/Desktop/Shotcut-22.04/Shotcut.app/lib/va/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x7fec94006ac0] libva: Found init function __vaDriverInit_1_1
[AVHWDeviceContext @ 0x7fec94006ac0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7fec94006ac0] Initialised VAAPI connection: version 1.1
[AVHWDeviceContext @ 0x7fec94006ac0] VAAPI driver: Mesa Gallium driver 20.0.8 for Radeon RX 560 Series (POLARIS11, DRM 3.35.0, 5.4.0-109-generic, LLVM 10.0.0).
[AVHWDeviceContext @ 0x7fec94006ac0] Driver not found in known nonstandard list, using standard behaviour.
[mlt_buffer @ 0x7fec942ab480] w:1280 h:720 pixfmt:nv12 tb:1001/30000 fr:30000/1001 sar:88889/50000
[h264_vaapi @ 0x7fec940020c0] Input surface format is nv12.
[h264_vaapi @ 0x7fec940020c0] Using VAAPI profile VAProfileH264Main (6).
[h264_vaapi @ 0x7fec940020c0] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x7fec940020c0] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x7fec940020c0] RC mode: CQP.
[h264_vaapi @ 0x7fec940020c0] RC quality: 23.
[h264_vaapi @ 0x7fec940020c0] RC framerate: 30000/1001 (29.97 fps).
[h264_vaapi @ 0x7fec940020c0] Using intra and P-frames (supported references: 1 / 0).
[h264_vaapi @ 0x7fec940020c0] Driver does not support some wanted packed headers (wanted 0xd, found 0).
[h264_vaapi @ 0x7fec940020c0] Driver does not support packed sequence headers, but a global header is requested.
[h264_vaapi @ 0x7fec940020c0] No global header will be written: this may result in a stream which is not usable for some purposes (e.g. not muxable to some containers).
[h264_vaapi @ 0x7fec940020c0] Using level 3.1.

So, I encoded the same video I did before, it took 41 mins so it seems to be using hardware acceleration.

The logs also mention it's using VAAPI:

[AVHWDeviceContext @ 0x7f8330006780] Trying to use DRM render node for device 0.
[AVHWDeviceContext @ 0x7f8330006780] libva: VA-API version 1.14.0
[AVHWDeviceContext @ 0x7f8330006780] libva: Trying to open /usr/lib/dri/radeonsi_drv_video.so
[AVHWDeviceContext @ 0x7f8330006780] libva: Found init function __vaDriverInit_1_14
[AVHWDeviceContext @ 0x7f8330006780] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7f8330006780] Initialised VAAPI connection: version 1.14
[AVHWDeviceContext @ 0x7f8330006780] VAAPI driver: Mesa Gallium driver 22.1.0 for AMD Radeon RX 570 Series (polaris10, LLVM 13.0.1, DRM 3.44, 5.17.9-zen1-1-zen).
[AVHWDeviceContext @ 0x7f8330006780] Driver not found in known nonstandard list, using standard behaviour.
[mlt_buffer @ 0x7f8330239a00] w:1920 h:1080 pixfmt:nv12 tb:1/60 fr:60/1 sar:0/1
[h264_vaapi @ 0x7f8330001f80] Input surface format is nv12.
[h264_vaapi @ 0x7f8330001f80] Using VAAPI profile VAProfileH264Main (6).
[h264_vaapi @ 0x7f8330001f80] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x7f8330001f80] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x7f8330001f80] RC mode: CQP.
[h264_vaapi @ 0x7f8330001f80] RC quality: 23.
[h264_vaapi @ 0x7f8330001f80] RC framerate: 60/1 (60.00 fps).
[h264_vaapi @ 0x7f8330001f80] Using intra and P-frames (supported references: 1 / 0).
[h264_vaapi @ 0x7f8330001f80] Driver does not support some wanted packed headers (wanted 0xd, found 0).
[h264_vaapi @ 0x7f8330001f80] Driver does not support packed sequence headers, but a global header is requested.
[h264_vaapi @ 0x7f8330001f80] No global header will be written: this may result in a stream which is not usable for some purposes (e.g. not muxable to some containers).
[h264_vaapi @ 0x7f8330001f80] Using level 4.2.
[h264 @ 0x7f833020a040] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0x7f83301be100] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0x7f833020cc00] Reinit context to 1920x1088, pix_fmt: yuv420p