vaapi doesn't work and Garbled output in vaapi-copy , waylandvk and vulkan (radv driver)

[gitpubber]

Important Information

• mpv version : mpv 0.34.0
• Linux Distribution and Version : Void Linux 5.15.12_1
• Source of the mpv binary : distro
• If known which version of mpv introduced the problem : No
• Window Manager and version : sway 1.6.1
• GPU driver and version:
  mesa 21.3.2
  mesa-vulkan-radeon-21.3.2_1
  amdvlk-2021.Q3.7_1

Context

I use ThinkPad E14 with i7-10510u processor which comes with AMD radeon rx 640 GPU discrete GPU . I run sway (v1.6.1) wayland compositor on it . I use mpv in sway.

Also , I can assure you that same problem persists in both voidlinux and archlinux ( I have tried mesa-git and mpv-git too).

With Intel GPU in sway , hardware acceleration works properly.

vaapi, wayland ,opengl-> works
vaapi, waylandvk , vulkan -> works
vaapi-copy , waylandvk , vulkan -> works
vaapi-copy , wayland , opengl -> works

And this issue is about problems all regarding amd radeon rx640 .

With amdgpu in sway though, I have two choices for vulkan driver (which works with mesa amdgpu driver) => RADV and amdvlk .

Here are the results of hardware acceleration with amdgpu + radv drivers .

amd radeon rx 640(radeon):
amdgpu + RADV: 

vaapi, wayland ,opengl-> doesn't work(fallbacks to no hwaccel; radeonsi_drv_video.so init failed)
vaapi, waylandvk , vulkan -> doesn't work(garbled output and no vaapi)

vaapi-copy , waylandvk , vulkan -> works but garbled output
vaapi-copy , wayland , opengl -> works


----
vaapi , drm , opengl -> doesn't work; fallback to no hwaccel
vaapi, displayvk, vulkan -> doesn't work ; fallback to no hwaccel
vaapi-copy, drm , opengl -> works
vaapi-copy, displayvk , vulkan -> works

----
vaapi,x11egl, opengl -> works
vaapi, x11vk, vulkan -> doesn't work ; fallback to no hwaccel
vaapi-copy, x11egl, opengl -> works
vaapi-copy , x11vk , vulkan -> works

With amdgpu + amdvlk , things are little different ,

amdgpu + amdvlk:  

vaapi, wayland ,opengl-> doesn't work(fallbacks to no hwaccel ; error message: radeonsi_drv_video.so init failed)
vaapi, waylandvk , vulkan -> doesn't work( no vaapi)

vaapi-copy , waylandvk , vulkan -> works
vaapi-copy , wayland , opengl -> works


----
vaapi , drm , opengl -> doesn't work; fallback to no hwaccel
vaapi, displayvk, vulkan -> doesn't work ; fallback to no hwaccel
vaapi-copy, drm , opengl -> works
vaapi-copy, displayvk , vulkan -> works

----
vaapi,x11egl, opengl -> works
vaapi, x11vk, vulkan -> doesn't work ; fallback to no hwaccel
vaapi-copy, x11egl, opengl -> works
vaapi-copy , x11vk , vulkan -> works

Briefly,

1. vaapi in amdgpu in both drivers radv/amdvlk in both opengl/vulkan doesn't work in wayland sway (but vaapi-copy works in few cases that i mentioned above). Is it because my amd radeon rx640 is dedicated GPU ? But , vaapi,x11egl, opengl works though ( why?)
2. vaapi-copy , wayland , opengl works in amdgpu (mesa driver) in sway . It has nothing to do with vulkan. So, this point(2) is not a issue .
3. This point might be bug with RADV or with waylandvk gpu context in mpv.

vaapi-copy , waylandvk , vulkan in amdgpu+radv shows that vaapi is working . But, it produces garbled output.

Screenshot of vaapi-copy,waylandvk, vulkan , amdgpu+radv :

But, if we use amdvlk instead of radv here , it produces correct output.

Screenshot of vaapi-copy,waylandvk, vulkan , amdgpu+amdvlk :

4. I thought it was bug of vulkan implementation in radv. But, if we use
   vaapi-copy, displayvk , wayland it uses wayland vaapi vulkan and displays properly under both radv and amdvlk. So, it might be more related to waylandvk with radv driver. Why displayvk works and why not waylandvk ?

5. vaapi, x11vk, vulkan doesn't work . It fallbacks to sw rendering. Why doesn't it work(it must display through XWayland)?

6. However vaapi,x11egl, opengl , vaapi-copy, x11egl, opengl and vaapi-copy , x11vk , vulkan work in Xwayland in sway using amdgpu in both vulkan drivers.

How I have run?

VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.x86_64.json DRI_PRIME=1 LIBVA_DRIVER_NAME=radeonsi mpv --no-config --hwdec=vaapi --gpu-context=waylandvk --gpu-api=vulkan --vaapi-device=/dev/dri/renderD129 filename
I have changed VK_ICD_FILENAMES to /usr/share/vulkan/icd.d/amd_icd64.json if i need to use amdvlk driver( i used to delete this driver when i needed to try radv ) and changed other parameters corresponding to situation.

[Dudemanguy]

I don't think vaapi works with amd (not my card anyway) At least I get VAAPI Vulkan interop requires support for dma_buf import in Vulkan in the log. vaapi-copy on radv works fine here though.

[LaserEyess]

It works for AMD, log from mpv --no-config --hwdec=vaapi --gpu-api=vulkan: https://0x0.st/osbW.txt

It's in the logs but I guess I should state I'm using RADV on 21.3.2, as well as a 5700 XT.

[gitpubber]

@Dudemanguy , vaapi x11egl opengl works fine on x11/xwayland in amd

[LaserEyess]

Also I think there are two bugs here.

1. RADV with garbled output on wayland
2. vaapi/vulkan interop is broken on your hardware somehow

[gitpubber]

Hello , I have uploaded all logs here: https://github.com/gitpubber/problemlog

[gitpubber]

@LaserEyess @Dudemanguy , Is there anything I can do ? Any updates on this ?

[nyanmisaka]

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6176
https://patchwork.freedesktop.org/series/80262/

dmabuf modifier support is only available for GFX9+(Vega and RDNA) but RX640 is GFX8(Lexa).

[Dudemanguy]

Yup, so that would explain vaapi not working on his hardware (and mine) on vulkan. The garbled output thing is probably a separate thing but maybe it's related. It's not likely to be an mpv issue in any case.

[gitpubber]

@nyanmisaka @Dudemanguy ,

can't those patches backported to support GFX8 ? anything can be done from my side(i don't know much driver programming)? i needed my hardware to support it

[LaserEyess]

If you clicked on the gitlab link, which it looks like you did since you made a comment, you can see that nothing can be done on your side short of buying a new laptop. You'd need to be very knowledgeable of AMD hardware as well as dma-bufs to get it to work on your hardware. Sorry but there's nothing anyone but AMD (or maybe valve) can do here.

[gitpubber]

@LaserEyess , @Dudemanguy ,

Can someone figure out why vaapi-copy displayvk vulkan works with RADV while vaapi-copy waylandvk vulkan doesn't work?

[Dudemanguy]

That's on mesa.