vo_dmabuf_wayland: wayland VO displaying dmabuf buffers using vaapi or drm hwdec

[boxerab]

Wayland VO that can display images from either vaapi or drm hwdecs.

The PR adds the following changes:

1. a context_wayland Wayland context with no gl dependencies
2. no-op ra and dmabuf_interop objects - no-op because there is no need to map/unmap the drmprime buffer, and there is no need to manage any textures.

Tested on both x86_64 and rk3399 AArch64

cc @philipl

[CounterPillow]

Tested on an AMD Ryzen 7 Pro 3700U, appears to work (with usual no osc/osd caveats) on Weston aside from fullscreening refusing to cover Weston's launcher bar. KWin seemingly lacks NV12 support. Will test on aarch64 when I get the time.

[boxerab]

I wish github would fix their webui spacing. It's really confusing. Feel free to rebase that one.

nods done rebased that one

[Dudemanguy]

Thanks for bearing with all this. General VO/wayland code looks good to me. I'll just have @philipl ACK the hwdec stuff real quick.

[philipl]

I still have a couple of questions.

And also, what is the mechanism that allows the VO compatibility check to pass when hwupload is deciding what formats should be available? You previously commented all that code out but I don't see anything that will make it work here. That code relies on having supported ra formats being provided and you're not doing that either.

[Dudemanguy]

And also, what is the mechanism that allows the VO compatibility check to pass when hwupload is deciding what formats should be available?

I think that's just the query_format? It'll only accept IMGFMT_VAAPI and IMGFMT_DRMPRIME. Unless I'm misunderstanding the question.

[philipl]

And also, what is the mechanism that allows the VO compatibility check to pass when hwupload is deciding what formats should be available?

I think that's just the query_format? It'll only accept IMGFMT_VAAPI and IMGFMT_DRMPRIME. Unless I'm misunderstanding the question.

https://github.com/mpv-player/mpv/blob/master/filters/f_hwtransfer.c#L377

[Dudemanguy]

https://github.com/mpv-player/mpv/blob/master/filters/f_hwtransfer.c#L377

Thanks. Is that not taken care of by this part?

mpv/video/out/vo_dmabuf_wayland.c

Lines 273 to 280 in 46c37f2

	case VOCTRL_LOAD_HWDEC_API:
	assert(p->hwdec_ctx.ra);
	struct hwdec_imgfmt_request* req = (struct hwdec_imgfmt_request*)data;
	if (!is_supported_fmt(req->imgfmt))
	return 0;
	ra_hwdec_ctx_load_fmt(&p->hwdec_ctx, vo->hwdec_devs, req);
	return (p->hwdec_ctx.num_hwdecs > 0);
	break;

[philipl]

No. This is the logic that compares sw formats.

I just tested the latest code from this PR and the hwupload formats that it ultimately accepts is weird. It rejects some of the basic ones (like yuv420p which we know works) and accepts weirder ones. The list also doesn't reflect the actual compositor capabilities. ie: GNOME only supports bgra and friends but this weird list we get is the same as I see on weston so it tries to send nv12 and just shows black. If I force a conversion in software to a format that GNOME supports, it will hwupload and display correctly.

This goes back to - in principle, the ra format list needs to reflect the compositor's supported formats for hwupload to work correctly.

[Dudemanguy]

Oh yeah I'm stupid. This hardcodes the drmplanes no wonder.

[philipl]

As we discussed, let's not block merging this on the hwupload behaviour, but we do need to fix the lack of error if there are no usable formats in f_autoconvert

[boxerab]

As we discussed, let's not block merging this on the hwupload behaviour, but we do need to fix the lack of error if there are no usable formats in f_autoconvert

I've added this error log.

[Dudemanguy]

Thanks for all the work on this!