Consider wgpu as future renderer backend (abstracts vk & gl)

[heavyrain266]

It's somehow related to #129 and #134, for example bevyengine/bevy uses gfx-rs/wgpu to abstract graphics APIs on every platform, on Linux it handles OpenGL and Vulkan without bigger effort. Smithay could use this library for renderer to automatically use either OpenGL or Vulkan, depending which one is supported by user's hardware or/and provide simple way to switch between them.

In my honest opinion, this will make future development much easier and allow you focus more on e.g. #363 and #423.

[PolyMeilex]

I'm pretty sure that wgpu is too abstract for internal compositor work, as it does not give us direct access to extensions that compositors use. For OpenGL, the first ones that come to my mind would be EGL_WL_bind_wayland_display or EGL_EXT_image_dma_buf_import
For Vulkan situation is probably a lot more complex than that.

But that aside, I would love to use wgpu in downstream compositors, so I think the easiest way to achieve that would be to swap one of wgpu-hal backends to use Smithay context as it's backend (either opengl or vulkan).

TLDR: compositors act a bit differently than typical clients, so it's not a matter of adding a wgpu dep and calling it a day.
EDIT: I kinda hope that someone will prove me wrong, as I love wgpu 😄

[heavyrain266]

Actually, now I see that while writing this, completely forgot about all vulkan extensions which I'm using in my compsoitor, they're not accessible in wgpu though 🤦

VK_KHR_wayland_surface, VK_EXT_physical_device_drm and few others

[i509VCB]

Other than dependent extensions and maybe things we don't know we need yet, I think these are needed:

• VK_EXT_image_drm_format_modifier
• VK_EXT_physical_device_drm
• VK_EXT_external_memory_dma_buf
• VK_KHR_external_memory_fd
• VK_KHR_external_memory
• VK_KHR_external_memory_capabilities
• VK_KHR_image_format_list

Also VK_KHR_wayland_surface is really intended for clients.

[heavyrain266]

The only thing which I got from vulkan renderer is that, it works much better with Nvidia GPU compared to OpenGL based compositor which tends to crash randomly and have weird graphical glitches + Nvidia uses linear vertical rendering on Wayland which is really really weird...

[i509VCB]

Well nvidia isn't the main reason I would adopt a vulkan renderer. With Vulkan you can render with less overhead and in a more performant fashion. Combining that with optimizations like damaged rendering and you could squeeze out some extra battery life.

[heavyrain266]

Battery life is excelent, compared Sway (gl) and RoseQuartz (implemented damage tracking with vulkan for that) and battery was 3-4h on Sway and 5,5h with RoseQuartz

[i509VCB]

Oh another note, @heavyrain266 I would recommend joining the smithay matrix channel.

[heavyrain266]

I was there, but for some reason I can't login to matrix anymore and my account doesn't exists, so I don't really want to bother with it again after trying to create new account for like 3 times (no, I wasn't banned or anothing, just got logged of randomly)

[i509VCB]

Hmm you don't appear to be on the member list. Is element doing some weird volatile account thing?

[heavyrain266]

I was using @branwright:matrix.org as account name before.

[elinorbgr]

The general impression I have wrt to that is that Smithay will always need to do low-level interaction with OpenGL and Vulkan, for setting things up with appropriate extensions and such indeed.

But we can provide the necessary hooks to use a high-level API like wgpu on top of the Smithay infrastructure, so that compositors writers can use Smithay's API to import client buffers as textures (and such), and then manipulate them using wgpu for doing all the rendering they wish. That is mostly the idea that emerged in the discussion of #129

[heavyrain266]

But we can provide the necessary hooks to use a high-level API like wgpu on top of the Smithay infrastructure, so that compositors writers can use Smithay's API to import client buffers as textures (and such), and then manipulate them using wgpu for doing all the rendering they wish. That is mostly the idea that emerged in the discussion of #129

Really interesting idea, I would like to see it in the future so will learn smithay codebase and try to do something about it 😁.

[Sroka]

I think it is possible to to set your own required vulkan extensions in wgpu. Please take a look at this example. It is about openXr integration with wgpu but the principle is the same:
https://github.com/philpax/wgpu-openxr-example/blob/main/src/xr.rs

[Drakulix]

I am not entirely sure, what the advantage of using wgpu is, when you add a hard-dependency on vulkan anyway by requiring a given set of extensions?
It is not like we want wgpu to enable running game engines on top of smithay like bevy, which might be a good reason to use wgpu + openxr.
But using vulkan directly gives us more control over specific performance characteristics and integration with our rendering api and is not necessarily much more complicated then using wgpu for this rather simple use case.

[heavyrain266]

We have discussed that on matrix many times before, the only adventage could be higher level abstraction for user which don't want raw vulkan but design of rendering API changed and I don't think that we need it anymore.

However, OpenXR requires stable graphics and for compositor you need entirely custom rendering logic anyways.

Personally, after using wgpu for long time, it's too high level for compositors because you want full control over pipeline and optimization often requires newest extensions while you cannot simply implement them in wgpu which is limited to specific vulkan extensions.

Another problem with wgpu on Wayland is that it seems to not work on Nvidia GPUs because they surface formats which are unsupported by drivers like sRGB.

[i509VCB]

I think wgpu's value isn't as a whole vulkan replacement for smithay, but something to complement it. Like the OpenXR example, it should also be possible to run wgpu off an existing instance (assuming the instance is properly configured).

There are some aspects like external memory extensions wgpu needs to handle that are currently work on progress, but it is better now than it was a year ago.