Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
DRM leasing #1723
I'm working on implementing DRM leasing (and with it, VR support) and wanted to start the discussion here on how it should shake out. For those unaware, DRM leasing allows us to "lease" some DRM resources (connector, crtc, encoder) to another process, allowing us to share the DRM master. This is used by VR applications to have exclusive and low-latency control over the HMD (head mounted display). I think a comprehensive solution for VR on Wayland (and wlroots) involves the following broad steps, in no particular order:
At that point, we can write our own Wayland clients which take advantage of DRM leasing to provide a VR experience. I'll write some basic demo as part of this workstream. However, to be useful, we'll probably want it to work with OpenVR (which is a misnomer, because it's closed source) - this is what Steam uses. It only supports X11, so my plan is to extend Xwayland with support for the DRM leasing protocol. DRM leasing via X11 was added to xrandr 1.6.
In the long term, I want to expand wlroots to support VR compositors better. This will involve at least:
It may be worth making the protocol multi-GPU aware. DRM object IDs are guaranteed to be unique across different devices, but I don't think we can create leases involving multiple devices (although I can't say I've tried). It could also help a client choose which device it wants to render with. I don't know if I client actually has a way to determine what device it has a lease for otherwise.
I'm don't think
Looking at the DRM API, it doesn't seem that there is a way to actually modify existing leases. We have to destroy and create them again.
Agreed. The DRM object matching logic is already stupidly complicated enough, and I'm sure that users don't really want to deal with that or have much they can add.
XR is not something I know a lot about or how it works, as well as these aren't complete technologies yet, but surely this is relevant.
Another potential (non-VR) use-case is running multiple Wayland/X11 servers on the same GPU, independent of each other. Kind of like how logind multiseat works.
Hm, good point. I'll ponder this a bit.
Yeah, I don't think that this is ideal, either, but I also don't know of a good way to expand on it elegantly. I think this is good enough for v1 because the usual case is not going to be affected - most compositors will only offer VR headsets for lease, and there'll probably only be one of them. And if not - the name is probably Good Enough to disambiguate them until we figure out something better.
Hm, drmModeChangeLease was proposed, but I guess it didn't make it. It looks like it almost did, I found a kernel with the ioctl implemented. I'll dig and see why this didn't land and update the protocol accordingly. Bleh.
See also: OpenHMD. I'd like to support these as well, but they'll almost certainly rely on DRM leasing as well so I think the work here translates quite directly to that. This stuff hasn't materialized in an entirely practical way yet, so I'm thinking of Steam only in terms of something useful to test with. Of course, I would like to dive into OpenHMD and get it working on Wayland as well.
Aye. I plan on adding support for this to the Wayland backend, where Wayland bootstraps a DRM lease and hands it over to the DRM backend, for this very purpose. It should be a pretty cut-and-dry change.
Works for me.
We don't have
They'll probably want to keep doing so, to minimize latency. However some VR Wayland clients would need such a protocol to benefit from compositor multiplexing.
Indeed, that needs to be considered. But see below, maybe not in this end.
I wonder if the client should choose at all.
I'm starting to think that maybe the client should indicate why it needs a lease. For instance a VR client could say "I want non-desktop outputs", instead of saying "I want this particular output". In case multiple DRM nodes or non-desktop outputs are available compositors could provide configuration options to choose what to lease.
Also, another note: surely this protocol should be privileged, all clients shouldn't be able to randomly create leases. As such the protocol might be rejected from wayland-protocols (or at least the current wayland-protocols).
This makes more sense for e.g. head tracking than e.g. controller input, imo
With some rejiggering (a lot, rather), we could allow the client to become a DRM master with no leased resources, scan connectors themselves for the outputs they want, then ask for those in particular.
I'm unfamiliar with leases. What happens when a client gets a lease? Does the leased FD expose all connectors? Or does the client only have a restricted view of the DRM node?
Update on drmModeChangeLease: I reached out to Keith to ask about why it didn't make the cut, if it wasn't a fundamental issue I'll adopt the effort and try to get it merged. I updated the protocol regardless, to make it more vague and support both the case where drmModeChangeLease exists and the case where it doesn't in a reasonably backwards-compatible way.
Update v2: it looks like it wasn't added just because it wasn't necessary for VR.
@ddevault asked for a description of the DRM object matching, so I'll post it here. It should probably be transferred over to a comment in the source code itself.
For a basic intro to the DRM object types, see https://github.com/ascent12/drm_doc. However, I never did get around to talking about planes or the atomic API.
We want to configure each pipeline like
I can definitely think of a few bits of unnecessary bits of complexity with our current solution. It would probably make your life significantly easier to address these before moving on to changing anything else.
There is half-assed and incomplete overlay plane handling in the code. The way I've done it is shit, and will definitely be changed when we get proper support. We should remove any code currently mentioning overlays.
We run the
The DRM Atomic API allows us to very cheaply test if a mapping should work (TEST_ONLY), which honestly could just let us take a more brute force approach. Unfortunately the legacy API doesn't have this, but I'm not going to let any legacy issues hold any atomic code back. I'd be fine with giving the legacy code a suboptimal solution if it's reduce a bunch of complexity. This is quite a rework, and may not be something you'd want to do right now.