Hardware accelerated video decoding with context/texture sharing #1381
Description
Windows build number:
10.0.26100.0
Your Distribution version:
24.04
Your WSL versions:
WSL version: 2.6.1.0
Kernel version: 6.6.87.2-1
WSLg version: 1.0.66
MSRDC version: 1.2.6353
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.26100.1-240331-1435.ge-release
Windows version: 10.0.26100.6584
Steps to reproduce:
Just install a wsl2 version of ubuntu 24.04, and do the usual steps to prepare it for hardware acceleration for video decoding (https://devblogs.microsoft.com/commandline/d3d12-gpu-video-acceleration-in-the-windows-subsystem-for-linux-now-available/)
There's a couple of not well-known c.q. undocumented steps, like activating systemd, video mod in order to mount /dev/dri/* devices, but is out of scope of the actual problem. Ones the environment is configured for hardware accelerated rendering you should be able to run GALLIUM_DRIVER=d3d12 LIBVA_DRIVER=d3d12 eglinfo or GALLIUM_DRIVER=d3d12 LIBVA_DRIVER=d3d12 glxinfo.
You should be able to see all of the supported extension etc, in my case, my machine has two gpu's, so I also have MESA_D3D12_DEFAULT_ADAPTER_NAME=nvidia to nudge it in the right direction because by default, it uses integrated gpu but the dedicated gpu is actually mounted for the wayland/x11 session.
WSL logs:
No response
WSL dumps:
No response
Expected behavior:
I'd expect that packages like https://pub.dev/packages/fvp, https://pub.dev/packages/media_kit, https://pub.dev/packages/flutter_gstreamer_player are able to work without problems, and use hardware accelerated decoding and rendering (when properly configured).
Actual behavior:
The reallity is, that a lot of these packages don't seem to work on wslg. They all seem to fall short or fail during composition when uing a hardware accelerated video output. They have screenshots of it running on Linux and other platforms, but they clearly don't seem to be compatible with wslg.
So I'll give you a concrete example, I have mpv installed and am able to run MESA_D3D12_DEFAULT_ADAPTER_NAME=nvidia GALLIUM_DRIVER=d3d12 LIBVA_DRIVER_NAME=d3d12 mpv 5103988-hd_1920_1080_30fps.mp4 it renders a very smooth video.
https://pub.dev/packages/media_kit uses libmpv internally, when i use this the place where the video would be diisplayed in the flutter application is either transparent or stays black, indicating some problem with the gpu-surface. I tried manually configuring the hwdec and vo parameters to 'vaapi' and 'gpu' and the an window with a video surface rendered smoothly like the mpv application, but the video is forced to render in an seperate window and not actually composited into the main application.
I'm using an nvidia rtx 3060 (mobile), and opengl profile 4.5/4.6 on the wsl side most interop related extensions should be supported. I'd expect that these interop api's would be used, and the textures can be shared accross contexts, and the application can composite it. Flutter has a built-in way to consume texture like this that rendered by an external process or context, and this package also seems to setup such a texture.
In a lot of places where you're trying to utulize the opengl contexts always warnings about screen:0 not having DRI3 support, and I believe this is a problem. DRI3 seems to be related to zero-copy style transfers, basically similair to opengl/vulkan interop api's, so I suspect that because of the gallium driver not supporting the DRI3 interface, basic compositing is failing.