Abstraction over input and output backends

[i509VCB]

Currently writing a compositor requires wiring up a backend to the compositor. Compositor developers may want to also have more than just udev, so that a compositor could be more easily tested inside a window.

Right now to manage these backends, compositor developers need to write code to link udev and possibly a windowed backend or two up to the compositor. In Anvil, the udev backend is about 1000 lines, and then winit and x11 are about 300 lines each. The future proposed Wayland backend will further increase this amount.

The common api proposal would provide something like the following:

• Abstract over input creation and forward events to consumers.
• Present dmabufs or memory buffers.
  • Rendering is intentionally excluded from this because rendering outside of Smithay's 2D pipelines may be desired.
• Handle creation and destruction of outputs and input devices.

Questions:

• Some features such as session management need to be exposed via this abstraction.
• Windowed backends operate differently to udev. You create windows with a windowed backend, but udev involves configuration of a fixed amount of display hardware.
• Windowed backends recieve input relative to a within a window. udev needs to map the input to some place in the compositor coordinate's coordinate space.
• Multigpu support
  • udev via KMS will probably need gpu info to present to each output, possibly on different display hardware.
  • Windowed backends could rely on the host compositor to handle routing presented buffers to the appropriate gpu in some manner.
• It would be ideal to support direct scanout.
  • For KMS this would be via overlay, underlay and cursor planes.
  • For the proposed Wayland output backend, subsurfaces could be utilized to allow the host compositor to optimize presentation and test scanout capabilities without switching ttys.
  • The X11 backend is not capable of subsurfaces.

cc @PolyMeilex, @cmeissl, @Drakulix

[Drakulix]

The general problem I see with this suggestions is, that the udev-backend pretty much falls out of line.
E.g. it likely need access to some kind of session (though given on the use-case not necessarily as shown by #310).

Abstracting over that part of the backend leads to either:

• Oversimplification of session-handling, that might not accommodate for more advanced use-cases, but does provide an easy and similar api for all backends.
• Or a full-blown session-api, that looks like the udev-backend today + a lot of technically unnecessary code to emulate this api for the window backends.

So while I am generally in favour of building some backend abstraction, I don't think we should attempt to make the udev-backend neatly fit into that. I would generally focus on building some common abstractions mostly for window-backed backends at first and adapt what makes sense for udev afterwards.

Abstract over input creation and forward events to consumers.

Yes absolutely. But I think we are half the way already there here.

Present dmabufs or memory buffers.

Yes, but we might wanna look in the nvidia situation first and then decide, if we want to keep nvidia compatibility for this abstraction (of course all lower level parts would still work) or mostly focus on mesa to upheld this design goal.

Handle creation and destruction of outputs

Outputs is the most interesting part here, imo. If we find a good abstraction for outputs, we can use that for udev and for better multi-window support in our windowed backends.

Some features such as session management need to be exposed via this abstraction.

As stated above, I am against doing this given the current state of the discussion.

You create windows with a windowed backend, but udev involves configuration of a fixed amount of display hardware.

Configuration might also be a non-goal of the abstraction, no? If additional hardware is plugged, output-events should be generated. If additional windows are created, the same thing holds true. But I would for now keep the way to trigger a hardware-scan or create a window specific to a given backend. (We can always introduce a WindowedBackend-trait later to unify window creation, but again wayland-windows and x11-windows can be very different in terms of properties they support.)

Multigpu support

For udev the interesting part here is scan-out. If we abstract over outputs, how can a user receive the scan-out DrmNode from an output? Maybe the Output should have a supported_nodes function? For the wayland backend this would probably work nicely for linux-dmabuf v4 as well, for the x11-backend that means selecting a screen, but for udev this might be a little tricky, given we would need to test the buffer. In general the scan-out api (however that may look) should likely not expect scan-out to succeed and possibly even provide a dedicated api for testing given dmabufs.

It would be ideal to support direct scanout.

If we support testing buffers, we can relatively easily do that. However we need to figure out how this will fit together with @cmeissl's DrmPlaneRenderer. I guess that more complicated and advanced use-case might not be covered by a simpler backend-abstraction, but it would be nice, if we find a way.

[PolyMeilex]

How about Udev Abstraction, instead of backend abstraction?
Windowed backends are quite obvious to initialize anyway.
And that would eliminate the problem of trying to fit all backends into one box.

[Drakulix]

If we are just talking about adding more helpers to ease initialization and management of the udev-backend, then I am all for it.