gpu-context=wayland performance drop

[ghost]

mpv 0.29.1, linux, swaywm (wayland)

independent of other settings, running mpv with --gpu-context=wayland kills performance. Barely any video can be played when fullscreen, lagging with 5-10 fps. When it a small window, it's fine again. Whether --opengl-es is set or not, does not matter. Running mpv in X mode, thus being displayed via XWayland gives ironically almost good performance again.

Additional info:
GM45 chipset (GMA 4500MHD) - 2.1 Mesa 19.2.1
In X plays 1080p60 video 100% smooth
In XWayland plays 1080p30 almost smooth
With gpu-context=wayland, fails to play 1080p30 over 15 fps

[Dudemanguy]

Try the master branch. It should be better I think (at least it seems that way for me).

[laichiaheng]

It really performs worse on wayland with the git version of MPV.

[ghost]

Compiled the master branch ( mpv 0.29.0-851-g2fcd5271eb )
It didn't change anything. The main culprit is the overload on the GPU, that makes everything a stop motion movie.
Quite ironically, forcing vo=x11 aaaaalmost fixes this. It baarely falls below Vsync on 1080p60 videos. I guess this is because with vo=x11 the gpu is left alone?
Is there any useful info I can provide?

offtopic:
Kinda makes me which there was some way to autoswitch to TTY and let DRM run the video. Now that I think about it, shouldn't there be a way to start mpv with root priviliges, let it render into a fbdev and just CTRL-ALT F1 switch to a fullscreen video? That way even 4k60 videos play 100% fluently. (on a 1080p screen

[Dudemanguy]

To be clear, how does the performance of --gpu-context=wayland compare to --gpu-context=x11egl and --gpu-context=x11 (with vo=gpu of course)? They should be more or less the same. At least my brief tests with them have suggested that.

vo=x11 and vo=xv are completely different and for compatibility reasons. You shouldn't compare them with any of the vo=gpu options. They might perform much better on lower end hardware (the tradeoff, of course, is lower quality) that . Also vo=wlshm was literally just merged which is a compatibility wayland output you might find useful.

Kinda makes me which there was some way to autoswitch to TTY and let DRM run the video.

There is a --gpu-context=drm and vo=drm as well.

[ghost]

@Dudemanguy I'm aware of --vo=drm, that's why I mentioned it :P
On wayland, gpu-context=X11egl and gpu-context=x11 perform the same. However, gpu-context=wayland drops more frames. In a 9 second 1080p60 video, gpu-context=wayland drops 300 frames and both the x11 options drop 150 frames. Again, on X, gpu-context=X11egl and gpu-context=x11 drop none. (except 8-9 at video startup).
This was tested, by letting a video play and reading the dopped frames value in stdout.

Will try out wlshm now :]

edit: wlshm works, looks to be about the same as x11 / x11egl, if not a smidgen better.
However, wlshm breaks the frames dropped metric, so I can't say for sure. Definitely still more of a 30fps with a 60fps video....

[Dudemanguy]

On wayland, gpu-context=X11egl and gpu-context=x11 perform the same. However, gpu-context=wayland drops more frames. In a 9 second 1080p60 video, gpu-context=wayland drops 300 frames and both the x11 options drop 150 frames. Again, on X, gpu-context=X11egl and gpu-context=x11 drop none. (except 8-9 at video startup).

Yeah this seems pretty bad. Hopefully I can just cop out and blame drivers here. Doing some more testing on my Haswell, I seem to get basically the same fps (it isn't possible to disable vsync for wayland and benchmark in the master branch atm, but in one of my local branches it is) in sway regardless of x11egl/x11/wayland. Actually testing in i3, I seem to get slightly lower fps than in wayland which is interesting.

However, wlshm breaks the frames dropped metric, so I can't say for sure.

Hmm, I guess I should investigate further into this too.

[aufkrawall]

I finally got my new notebook battery from China and could give it a go on that Gemini Lake 2 core system: Playback seems to work flawlessly on Sway-git with VAAPI, OGL and Vulkan. Vsync jitter of ~0.
That was with 1440p 60fps VP9 streamed from YouTube & recent mpv-git.

[Dudemanguy]

Btw @SairesArt, you can benckmark the wayland backend in the master branch now if you use --wayland-disable-vsync along with the other options listed here. If you compare it to x11egl's fps, how different is the performance?

P.S. Oh and use sway-git/wlroots-git not 1.2. Sway's master branch has some additional fixes.

[ghost]

@Dudemanguy I built sway, mpv and ffmpeg from git and magically, the performance is vastly improved!
I actually manage to hit vsync! Finally no more switching to tty for smooth video <3

First of all: gpu-context=wayland has a major bug now: If A-V desync happens, it does not resync back by default! The default --video-sync=audio suddenly behaves like desync. It is not broken under gpu-context=x11

For some reason neither osd-msg1 nor term-status-msg report any dynamic data. It all remains just a static message like "fps", although pressing I shows information like estimated-display-fps properly. So I measured with two other methods: 1. I timed a long video without vsync and calculated fps based on time passed and total frames in the video file. This was done 5 times and times were averaged. 2. recorded dropped frames when vsynced.

Now for the numbers...
in a 960x600 window x11 and x11egl (~118fps) performed the same and wayland performed slightly worse (~113fps).
However, fullscreen @1920x1200 when Vsynced + display-resample, it's a whole other story. Wayland drops 6x-8x more frames. All of this is still in wayland, so ironically, running XWayland is better! I cannot wrap my head around this.
In a 5000frame block of 60fps video, wayland drops 70-80 frames, whilst x11egl drops only 6-11. x11 looks and feels undoubtedly smoother (no micro stutter / small pause because of display-resample) because of this.

[Dudemanguy]

First of all: gpu-context=wayland has a major bug now: If A-V desync happens, it does not resync back by default! The default --video-sync=audio suddenly behaves like desync. It is not broken under gpu-context=x11

The fallback mechanism was removed 273cc30. Are you sure it's not just this? Before that commit getting a large enough A-V desync would automatically disable video-sync=display-resample and go back to the default. Now it just stays on all the time no matter what your A-V is. So if you're constantly getting bad A-V, you'll have lots of dropped/mistimed/etc frames the entire time.

For some reason neither osd-msg1 nor term-status-msg report any dynamic data. It all remains just a static message like "fps", although pressing I shows information like estimated-display-fps properly.

Something is wrong here. If stats.lua can display the estimated-display-fps property then so should anything else that requests it. The main reason to avoid using stats.lua is because it can be kind of slow.

Glad that the performance is at least a little better at least. This is a weird problem though. I can't really wrap my head around it either.

[aufkrawall]

I spoke too soon, there are occasional jitter spikes on Gemini Lake causing mistimed frames.

[xantoz]

@SairesArt Try this branch and see if it helps any: https://github.com/xantoz/mpv/tree/fence-magic--v2

That reshuffling of fences helps performance in weston on sandy bridge for me, at least. And I'm not 100% sure why...
Seems driver related, somehow.

[xantoz]

offtopic:
Kinda makes me which there was some way to autoswitch to TTY and let DRM run the video. Now that I think about it, shouldn't there be a way to start mpv with root priviliges, let it render into a fbdev and just CTRL-ALT F1 switch to a fullscreen video? That way even 4k60 videos play 100% fluently

Should be theoretically possible using DRM leasing, but for wayland there is currently only the experimental/non-standardized support in experimental branches on sway/wlroots. It seems mainly geared towards use with VR headsets and other "non-desktop" displays as well. There's also the question on how to handle input, as vo=drm/gpu-context=drm have no actual input code, and they just rely on mpv running in the underlying terminal you started mpv from (try starting mpv --gpu-context=drm over ssh to see what I mean)

Note: You do not need root privileges to use either vo=drm or gpu-context=drm, you just need rw permission to /dev/dri/cardX which can usually be gotten by adding your user to the video group

[aufkrawall]

@xantoz Your commit unfortunately doesn't improve my jitter spike issue.
Reminds me an awful lot of the regular stutter with modesetting DDX on Xorg. :(

[ghost]

using DRM leasing

Wow, thanks for introducing me to this! I'm baffled how I managed to miss this. I recently started porting my OpenGl Application to DRM for performance reasons and this is awesome news. Since I'm on sway, I'll definitely start using those experimental branches.

Try this branch and see if it helps any: https://github.com/xantoz/mpv/tree/fence-magic--v2

Thanks for the amazing effort and digging. Unfortunately this does not change the outcome.
In a 5 minute 60fps video, with video-sync=display-resample, gpu-context=x11 drops 38 frames (mostly on start-up) and gpu-context=wayland drops 376 frames throughout the watch duration.

Without display-resample the dropped frames aren't that bad and x11 without display-resample continues to be the best option. The combination of wayland's "every frame is perfect" and the 1920x1200 display just are too much for that poor GMA 4500 MHD and the weak drivers.

[Soundtoxin]

I'm getting video desync on Wayland with mpv 30.0. Using --vo=wlshm trades desync for 88 frame drops, and using --vo=x11 gets me down to 50 frame drops. I'm using a ThinkPad T440p with a mobile Haswell i7 and integrated graphics. I have an older ThinkPad X220T with a mobile Sandy Bridge i5 that plays back the same file with no desync and only dropping 10 frames. Both machines are using Sway on Wayland, but the Sandy Bridge machine is on Guix System and the Haswell machine is on NixOS.

Here are some logs:

• Haswell
• Sandy Bridge

---

• Haswell --vo=wlshm
• Haswell --vo=x11
• Sandy Bridge --vo=wlshm
• Sandy Bridge --vo=x11

[ghost]

(For x11 and wlshm, try without subtitles and OSD, because these are handled extremely inefficiently on these VOs.)

[multiflexi]

I think I am experiencing same problem with Wayland and nvdec - more here: https://www.reddit.com/r/mpv/comments/dpqn3l/nvdec_on_wayland/
I compiled ffmpeg 4.1.4 with nv-codec-headers installed and also git version of mpv but neither VDPAU or NVDEC work.

[Dudemanguy]

Does nvdec even work on wayland? I'd be surprised if it does. vdpau doesn't at least.

nvidia+wayland is just bad news and should be avoided (AKA just use x11 and hope that one day nvidia stops being a crappy company). I guess nouveau might work somewhat OK but I wouldn't count on good performance. No idea what the state of eglstreams is in gnome, but I'd be surprised if it was anywhere close to GBM.

Edit: Regardless it's not really the same problem. Nvidia's wayland """support""" is inherently broken. Just use x11.

[philipl]

Technically, nvdec with interop does work with wayland. I’ve tested it. You must use OpenGL and must force GL-ES as desktop gl silently fails and results in a blank texture. Vdpau does not work at all.

But yes, wayland on Nvidia is mostly unusable and you should just use x11.

[sl1pkn07]

and now with the new nvidia beta 495?

greetings

[Dudemanguy]

There's an FAQ entry on it. The gist is basically "who knows but don't expect much".

[sl1pkn07]

but both GNOME and KDE have hard coded driver detection that means they will still use EGLStreams despite the new GBM support.

now in KDE:
https://invent.kde.org/plasma/kwin/-/merge_requests/1526#note_325121
https://invent.kde.org/plasma/kwin/-/merge_requests/1526#note_325139

[Dudemanguy]

In mutter too actually. I'll update that.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2052

[andrewfader]

I've got wlshm working alright in nvidia/wayland/gnome but I don't think it's using nvdec. I think I'm on nvidia gbm, using newer mutter.

[Dudemanguy]

Don't use wlshm (nvdec won't work on it). opengl (the default) should work on the master branch with nvidia.

[andrewfader]

Nevermind, it works fine if I delete everything and just do gpu-context=wayland

 (+) Video --vid=1 (*) (h264 1280x720 29.970fps)
 (+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
     Subs  --sid=1 --slang=eng (subrip)
     Subs  --sid=2 (*) (eia_608)
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 1280x720 yuv420p
(Paused) AV: 00:00:09 / 00:21:03 (1%) A-V:  0.000

Thanks for the good work here!

[Whitecatkeke]

It works well when it works, with "gpu-context=wayland". But sometimes, videos can not be played and I see the following errors:

[vo/gpu/opengl] Error: framebuffer completeness check failed (error=36061).
[vo/gpu] Error: texture could not be created.

[andrewfader]

I have nvdec-copy working on nvidia/gnome/wayland with 510 proprietary beta driver and using gbm

❯ mpv /mnt/earth/downloads23/Whose.Line.Is.It.Anyway.US.S16.720p.MIXED.h264-BTN/Whose.Line.Is.It.Anyway.US.S16E15.Brad.Sherwood.4.720p.CW.WEB-DL.AAC2.0.H.264-RTFM.mkv
 (+) Video --vid=1 (*) (h264 1280x720 29.970fps)
 (+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
     Subs  --sid=1 --slang=eng (subrip)
[vo/gpu/wayland] GNOME's wayland compositor lacks support for the idle inhibit protocol. This means the screen can blank during playback.
[vo/gpu] Interpolation now requires enabling display-sync mode.
[vo/gpu] E.g.: --video-sync=display-resample
Using hardware decoding (nvdec-copy).
 (+) Video --vid=1 (*) (h264 1280x720 29.970fps)
 (+) Audio --aid=1 --alang=eng (*) (aac 2ch 44100Hz)
     Subs  --sid=1 --slang=eng (subrip)
     Subs  --sid=2 (*) (eia_608)
AO: [pulse] 44100Hz stereo 2ch float
VO: [gpu] 1280x720 nv12
(Paused) AV: 00:00:01 / 00:21:03 (0%) A-V:  0.000

❯ cat ~/.config/mpv/mpv.conf
profile=gpu-hq
hwdec=auto-copy-safe
gpu-context=wayland
scale=ewa_lanczossharp
cscale=ewa_lanczossharp
interpolation
tscale=oversample

[Dudemanguy]

OP is a ghost now and this is pretty stale so I'll close it. Feel free to open up a new issue if there is one.