video/out/opengl: Prioritize Wayland over X11

[vanvugt]

This fixes bug LP: #1698287. The problem was that in a Gnome Shell
Wayland session both X11 (Xwayland) and Wayland are available. But
choosing X11 results in an unusable VO for VAAPI. The simple fix is
to choose Wayland over X11 if both are present.

[CounterPillow]

This was changed on purpose because people kept complaining that they didn't have window decorations under GNOME on Wayland.

The real fix would be to fix opening a VAAPI context on xwayland, and not this.

[vanvugt]

Yes I can see the problem you're talking about. Although both Wayland and X11 will need to coexist for years to come so it would be better if we can find a way to use Wayland over X11. If only it had decorations... Is that a Wayland feature or a bug?

[vanvugt]

Oh great, it's a feature. Wayland is slightly less great than I expected...

[wm4]

Is that a Wayland feature or a bug?

Depends who you ask. For the Wayland devs, the topic is over.

One thing is clear though: Wayland is not ready for desktop, unless you throw tons of vendor-specific extensions at it.

[haasn]

I personally don't see wayland surviving long without significant future improvements to the protocol being made (to give the compositor more control over windows), so I fully expect red hat to cave in here eventually anyway.

Anyway, it sounds like this specific issue is a bug, especially considering the upstream launchpad report is about vaapi failing to work, which has absolutely nothing to do with whether or not mpv decides to use wayland or X11.

[vanvugt]

Well there has to be a fix somewhere eventually. Intel has already disowned the problem too (01org/intel-vaapi-driver#203). I'll keep investigating...

[montvid]

mpv works on gnome wayland with --opengl-backend=wayland tag, If anyone wants window decorations they can install gnome-mpv - problem solved.

[jcjordyn130]

@montvid not everyone wants to use a GTK frontend to a command line video player.

[vanvugt]

I looked into gnome-mpv a while back. It wasn't looking great, but is fixable:
https://community.ubuntu.com/t/hardware-accelerated-video-playback/304/4