Pixman software renderer

[emersion]

After renderer v6, we'll be able to start working on a pixman software renderer. It'll probably work this way:

• Allocator: add a dumb buffer allocator. The returned wlr_buffer can be exported to a DMA-BUF via drmPrimeHandleToFD.
• Renderer: import the DMA-BUF via drmPrimeFDToHandle and mmap the dumb buffer.
• DRM backend: import the DMA-BUF via drmPrimeFDToHandle (or GBM?) and scan it out
• Other backends: ???

Need to be careful wrt. GEM handle ref-counting.

[bl4ckb0ne]

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?

Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

[emersion]

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?

Probably in wlr_gles2_renderer yes.

Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

By default, if there's a render node, try GLES2. Otherwise try pixman. We probably want an env var to force a specific renderer.

[emersion]

More thoughts:

• As discussed on IRC, the DRM dumb buffer allocator will get GEM handles and somehow needs to share them with the DRM backend. We need to be careful wrt. ref'counting issues with GEM handles.
  • One solution is to open the primary DRM node multiple times (via the session) and then the allocator can do some drmPrimeHandleToFD calls to implement wlr_buffer_impl.get_dmabuf.
  • Another solution would be to have a wlr_buffer_impl.get_gem_handle function, but this would be fragile as it would completely break if both parties don't have the same file description.
• The DRM dumb buffer allocator should probably check for DRM_CAP_DUMB_BUFFER.
• For the Wayland/X11/headless backends, DRM dumb buffers are not appropriate. We'll probably need a shm allocator (which uses shm_open), use wl_shm on the Wayland backend, use MIT-SHM's AttachFd on X11, and a new wlr_buffer_impl.get_shm_fd function to share the FDs between the allocator and the backends.
• We need a way to access the buffers from the Pixman renderer.
  • We could special-case the DRM dumb buffer allocator in the Pixman renderer, but it would prevent the shm allocator and third-party allocators to work.
  • We could add a wlr_buffer_impl.map function to create a memory mapping for the buffer.

[emersion]

We need a way to access the buffers from the Pixman renderer.

To expand on this a little bit more: if we want something that works with both the DRM dumb allocator and the shm allocator, we have a few solutions:

1. Add a wlr_buffer_impl.get_data_ptr function that returns a pointer to an mmap'ed region. This would be pretty close to what @bl4ckb0ne has implemented in the PR: mmap happens in the allocator, there's only a single mmap'ed region, and the renderer just creates a pixman surface with the data pointer.
2. Add wlr_buffer_impl.begin_access and wlr_buffer_impl.end_access. This would be similar to (1), however it would let the allocator know when the mmap'ed region is being accessed by the renderer. In case the buffer comes from an untrusted client, this is necessary to make sure SIGBUS doesn't take the whole compositor down. See https://wayland.freedesktop.org/docs/html/apc.html#Server-structwl__shm__buffer_1a809cb5d6b33338c62bbca6daa4138667. We may want to render to a client buffer for the screencopy protocol. However screencopy could just call wl_shm_buffer_begin_access before asking the renderer to perform the copy. Or it could just do the memcpy itself from the wlr_buffer without having to interact with the renderer at all.
3. Add wlr_buffer_impl.map and wlr_buffer_impl.unmap functions, and let the renderer create the memory mappings. This would allow us to have potentially more efficient mappings, e.g. a write-only mapping for rendering and a read-only mapping for screencopy.

[bl4ckb0ne]

I like the option 1 for now, I think it's the easiest/quickest one. Do you want a separate patch for it or can I add it to the drm dumb allocator one and patch the gbm allocator after?

[emersion]

GBM doesn't need to implement it. The best for a review would be to add to your DRM dumb buffer allocator a commit that adds the wlr_bufferand wlr_buffer_impl functions, then a commit that implements it in the DRM dumb buffer allocator.

[bl4ckb0ne]

Done in #2700. Though I'm getting segfaults when I rebase #2661 to use it, still investigating.