SmoothMotion: occasionally stuttering + high CPU usage

[Soukyuu]

A continuation of #731

I tested the new master branch build, and the first time watching the scene (Akame ga kill ep 23, the pan in front of the gates after the opening) was buttery smooth, just like madvr. Replaying the scene a few times had 1-4 stutters, usually a few of them in succession. It's already better than it was before though.

I tried playing the whole file from the start without skipping right to the scene, but that doesn't change anything.

There is also a huge impact on the CPU - enabling SM pushed my AMD Phenom II X4@3.4GHz to ~60% of CPU usage vs ~13% without SM. GPU (260GTX) isn't loaded at all according to GPU-Z.

This is on windows 8.1 x64, build compiled with msys2. However, the same behavior can be seen on my two arch linux x64 systems, one running an AMD Zacate e-350 APU (6130 GPU, open source driver) and another running Intel C2D T7250@2GHz+8600m GT (proprietary driver).

All three systems can handle madvr's smooth motion without high impact on the CPU and they also don't stutter.

[ghost]

Possible causes:

• you have disabled vsync
• the driver doesn't lock to vsync (try the glfinish suboption, which may help, or may make everything worse)
• it doesn't lock to vsync because it's locked to the compositor
• the stutter actually comes from subtitle rendering or so (try with --no-sub)
• the stutter comes from messed up timestamp due to broken audio playback status reporting (try with --no-audio)
• some other bug anywhere

[Soukyuu]

• vsync is enabled
• glfinish didn't change anything (windows)
• glfinish doubles the blur (linux)
• I don't think I can disable DWM... but disabling kwin effects solves it
• no change with --no-sub/--no-audio on windows

So maybe it does "lock to compositor", whatever exactly that means. Is that something you can change? Disabling composition on linux is possible, I guess, but windows...

I also tried playing around with kwin's rendering options and openGL + effects off is the only thing that doesn't stutter. openGL + any strategy for vsync stutters, xrender instead of openGL disables vsync.

[ghost]

The compositor probably accepts new frames as quickly as possible - don't know, really depends on implementation details, which are different on all platforms, and I don't know the details for any of these.

I think on Windows, going fullscreen with OpenGL usually "unredirects" the window, which effectively disables compositing?

[Soukyuu]

I could imagine that my 8600m GT/6130 GPUs on linux are just not powerful enough for kwin effects + smooth motion/opengl-hq (kwin is known for being heavy, anyway), so I'm willing to just accept having to disable the effects before watching stuff.

No idea about windows getting "unredirected", I'm actually running madvr in overlay mode because it's fast enough for me. I see the screens flicker with mpv, so it does go into exclusive fullscreen mode. I thought that maybe having multiple screens affects it, so I turned the second one off in the control panel - at first it seemed to solve the stutters, but they then re-appeared, so that's not it.

There is also the high CPU issue, though, so maybe it's related to that somehow.

[pigoz]

Does setting --display-fps=60 affect the CPU usage? IIRC, that should stop mpv to keep waking even if vsync doesn't block on your platform.

[Soukyuu]

No, but setting that option has nearly eliminated the stutters on windows! They are almost unnoticeable unless you're looking for them.

[AirPort]

Same on Os X: I have no CPU usage problem, but there were observable stutters with smooth motion enabled. --display-fps=60 makes them barely noticeable.

[qmega]

@Soukyuu In case you didn't know, for KWin, you could set a application or window rule for mpv to block compositing, so you wouldn't have to manually disable and re-enable effects. It's under "Appearance & Fixes".

[Soukyuu]

@qmega: Thanks, I don't know why, but I took that as "sector/region/zone compositing" not "blocking compositing" when I saw the option. It's working great on linux now. And that's why love KDE/KWin...

Also, it seems that the CPU usage issue is somehow windows related - on the 8600m GT on linux, I'm getting about the same CPU usage in both SM and non-SM mode now.

(I should have probably posted the CPU usage as its own issue now that I think about it...)

[pigoz]

If the CPU issue is windows only it could be a threading issue?

[Soukyuu]

FWIW, I tried rebuilding mpv with --enable-win32-internal-pthreads, but no change.

[ghost]

Now it would be nice if there was a working debugger or profiler for windows/mingw.

[bodayw]

I have a high CPU usage on OS X as well...1080p playback on a 1440p screen usually cost ~50% of CPU usage, but with smoothmotion it goes up to 120-150%. Tested on the lowest model of Mac mini 2014 with i5-4260U/HD5000. Not sure this is expected or not...

Otherwise it looks good...Panning scenes are buttery smooth now, very satisfied :D

Will report again when a new Windows build including smoothmotion is available.

[bodayw]

OK, I have tried smoothmotion on my Windows machine (Windows 8.1, i7-3520M/HD4000) with the new build, and the impact on CPU usage is minimal, if at all. GPU wise the difference of usage was huge: 70% with SM vs. 15% without.

Also, with the new OS X build on 20150127 my CPU usage with smoothmotion goes down a bit, with 80-100% instead of 120-150% before while playing the same file.

[Soukyuu]

Tried the latest x64 build by lachs0r on my Windows 8.1, Phenom II X4 970/260GTX machine, stats unchanged:

• smoothmotion: CPU:53% GPU:10%
• no smoothmotion: CPU: 3,5% GPU:4%

For comparison, mpc-hc + madvr (jinc3, light debanding):

• smoothmotion: CPU: 6,1% GPU: 53%
• no smoothmotion: CPU: 5,5% GPU: 49%

Is the code intel-optimized?

[pigoz]

@Soukyuu did you try to use glfinish in the --vo=opengl arguments?

[ghost]

Also you need --display-fps.

[Soukyuu]

Using glfinish cuts CPU usage to 23%, but the result is subjectively less smooth.

The scale of smoothness is madvr > opengl-hq:smoothmotion > opengl-hq:smoothmotion:glfinish, with madvr being slightly blurier. Going off on a tangent here, is there a way to vary the blurryness with current code? Blur is easier on the eyes than small stutters, imho.

@wm4: Yes, I am using --display-fps=60, otherwise there would be random stuttering.

[pigoz]

It should not stutter, if it does there's probably a bug. What is your test video? Can you upload it somewhere?

[Soukyuu]

The video I'm currently using is Akame ga Kill ep 23 (by vivid-asenshi), scene in front of the gates, from 00:03:40 to 00:03:53 I've extracted the scene and uploaded it here.

(Ignore the image quality, the motion didn't get damaged in the converting process.)

[Cyberbeing]

I'm also noticing that mpv smoothmotion seems to cause small stutters on Win7 SP1 x64 + GTX 770 when the blending factor is very small, like when I playback 23.976fps video on a near-match ~96Hz display. These stutters are less noticeable with mismatch refresh rates, but still exists if you look from them. This is with display-fps set.

CPU usage is also extremely high with 2 CPU cores running at nearly 100% load in Fullscreen when smoothmotion is enabled, unless I enable glfinish. Though as Soukyuu noted, the stuttering increases with smoothmotion + glfinish. Disabling Desktop Composition doesn't seem to eliminate these smoothmotion stutters either.

I suspect this may be at least partially related to whatever causes Issue #98. Since for things like my 23.976fps 96hz example, madVR (smoothmotion disabled) playback is still smoother than anything which I can accomplish with mpv. Currently mpv (smoothmotion enabled) is actually less smooth than mpv (smoothmotion disabled) for this use case, which shouldn't be so if smoothmotion was functioning properly.

[ghost]

So how is it with --no-audio?

[Cyberbeing]

Good point, I forgot to test that.

Unfortunately, it seems --no-audio only changes the appearance of the stutter with smoothmotion enabled to more of a constant micro-stutter. By comparison --no-audio with smoothmotion disabled is almost perfectly smooth, similar to my Issue #98 conclusions.

This leads me to conclude that the underlying cause of the smoothmotion stutters I'm seeing must exist primarily on the OpenGL video renderer side of things, while any stuttering caused by Issue #98 is a separate unrelated problem.

[ghost]

@Cyberbeing: how is your GPU usage?

[Cyberbeing]

~11% GPU load at low-3D clocks with smoothmotion enabled & ~3% with smoothmotion disabled.

[Cyberbeing]

Another thing which comes to mind, is how/when/where/if you call the opengl flush() command. Flushing can potentially have a negative or positive effect in terms of stuttering and general GPU stability, so you would need to experiment.

[pigoz]

CPU usage is still a mystery, but 3931544 may help with the stuttering.

[Soukyuu]

Updated to newest lachs0r build, I don't see any difference (on windows), sadly.

[haasn]

Does this commit help the performance? haasn@861c852

Also, you should test master again, using version after 4356e89. That ought to fix the stuttering issues, assuming your --display-fps is correctly set up.

[bodayw]

Just tested with the latest OS X build. Now panning scenes look perfectly smooth, no more occasional stuttering.

Performance wise, CPU usage is a bit lower now, but not much. It's still considerably higher than smoothmotion disabled. Well compared with the numbers I posted here earlier, it's more like no difference.

Will report again when the new Windows build including the new commit is available.

[Soukyuu]

I built a fresh msys2 windows build, the random stuttering seems to be gone now - yay.
As @bodayw says, the CPU usage is still unchanged for me also.

[Cyberbeing]

I've only tested lachs0r's 2015-02-24 dc3c718 build, just prior to the performance commit, but the earlier 4356e89 commit does seem to resolve the stuttering issues I was seeing before.

One thing I've discovered about this particular build though, is SmoothMotion CPU usage on my i5-3570K is very high (90%|90%|40%|40%) in fullscreen only when NVIDIA Threaded optimization is enabled or auto (default).

If you create a profile for mpv.exe in NVCPL and explicitly disable threaded optimization, CPU usage is much more reasonable in fullscreen (30%|10%|5%|2%). Still much higher than smoothmotion disabled (8%|8%|2%|1%), but who knows if anything can be done about that.

No idea if this will still hold true for the latest git after 010cf18, but it seems somewhat promising.

[bodayw]

Tested with the new Windows build as well. The CPU usage issue doesn't appear which is same as before. Notably the GPU usage is a bit lower now (60% vs. 70% before).

So in my case, all of my computers are running on Intel CPU and integrated GPU, but I only have high CPU usage on OS X.

[bodayw]

Today while watching something on my Windows machine (I rarely do this since I mostly watch on my Mac mini as the HTPC), I noticed quite a few stuttering during panning scenes. I further tested and confirmed it with my favorite scene for testing smoothmotion. Well I guess I was only paying attention on the CPU and GPU usage back then so I didn't notice it.

So it seems that on my Windows machine(s) which doesn't have the high CPU usage issue, smoothmomtion doesn't actually work as it should as well...

And I don't think it's because the GPU is too weak to handle this, since GPU usage were always below 70%, also I can get perfectly smooth results on my Mac mini which has an even weaker GPU.

[haasn]

GPU usage percentage can be a misleading measurement here. Just because your GPU is idle for 30% of the time doesn't mean it can render a frame in time, due to the way it's implemented. (Since we draw once per vsync, there's at most one vsync of time you can squeeze a render operation into - any more than that and you start delaying a blended frame, which causes stutter)

Either way, @avih has also independently confirmed general vsync and timing issues and frame drops on Windows, so maybe this is the underlying cause.

Can you test on a Linux-based machine, for good measure? When using hardware decoding (for testing purposes), I get CPU usages around 1% per core in the new code path (current master), which is also more taxing on the CPU than the old one.

[avih]

Depending on your system/driver, you might also get better performance/smoothness in full screen with OpenGL on Windows (even if it's harder to monitor cpu/gpu usage when in full screen).

The main reason is that windows has desktop composition (DWM) when running in a window which might mess with OpenGL vsync, but the driver might override it when going full screen. YMMV.

On windows 7 you can disable DWM by choosing some of the themes (the "Aero" ones have DWM), and on windows 8 and up you can't disable DWM (but full screen might or might not have DWM).

[Soukyuu]

Can you test on a Linux-based machine, for good measure? When using hardware decoding (for testing purposes), I get CPU usages around 1% per core in the new code path (current master), which is also more taxing on the CPU than the old one.

I somehow completely overlooked this part. I've just looked at CPU usage on linux and it's the same (~9% according to ksysguard) whether the interpolation is on or not.

[Soukyuu]

...it's back to randomly stuttering here. About the only thing I changed was updating to the latest git snapshot, but sadly I don't remember which revision I was on before. Was there anything changed code-wise that could have triggered it?

[Soukyuu]

Or rather, not random. It seem it lock on vsync correctly about 3 times, stutters, locks on correctly 3 times again, stutters. I'm at a loss here, I tried fiddling with compositing/displayfps and different scalers, but nothing helps. It's barely better than no interpolation - and the same file plays smoothly on windows with madvr =/

[Soukyuu]

..I swear. I'm starting to hate KWin about as much as I love it. Apparently something changed and now I have to completely disable compositing, or not use kwin to get smooth playback. So nvm, not mpv's problem.

[mia-0]

You can tell KWin to disable compositing for all fullscreen windows, or mpv in particular. For the latter, use the menu accessible from the title bar; there should be an advanced settings dialog to allow you to set rules for mpv’s window class.

[Soukyuu]

Yes, though that's not all. You must also convince your GPU to vsync on the monitor where mpv is playing. This seems to work out of the box on windows, but for linux I had to do this. I was getting nasty tearing without it.

[Soukyuu]

I think we can close this, unless someone on windows is still getting this behavior.
I've fully moved to linux (and it works as intended for me), so I have no real interest in the windows version anymore.

[ghost]

OK then.