Smooth-motion interpolation without display-resample

[Nojevah]

Expected behavior of the wanted feature

You certainly have heard of madVR Smooth Motion. It also exists in another less-known app (MPDN).
It creates blended frames to improve motion. For the 3:2 pulldown, in 60 Hz, it does not change 48 frames but it creates 12 blended frames.
This solution is CPU and GPU friendly.
If you want to use display-resample and oversample interpolation in mpv (which is supposed to be the equivalent), first, it is CPU and GPU heavy (because of display-resample), plus it causes more flickering.
Display-resample also has some problems, and is totally overkill when having a correct (high) display refresh rate.

I hope this request won't be close, I just created this thread in order to motivate someone who might think Smooth Motion already exists whereas it's not the same (perfect) Smooth Motion as madVR.

[Akemi]

i doubt anyone knows what exactly you are requesting here. you have to be far more specific in what you actually want.

[Nojevah]

It would help if you tell me what you don't understand, because to me it is very clear. :/
haasn and wm4 who have worked on display-resample and interpolation would understand it well (I guess ?).
Is it Smooth Motion you don't understand ? Or the difference with/without video-sync=display-resample ?

[Dudemanguy]

I don't understand. How is display-resample overkill? If you care about accurate frame timings (which is baked into the entire premise of this question), you need this. Are you asking for the ability to use interpolation but without setting one of the display- modes? It's not technically impossible, but why would you want this? If you want to eliminate the appearance of judder as much as possible (one of the reasons to use interpolation in the first), you need to care about your display's actual refresh rate which is what the display-resample option does.

It creates blended frames to improve motion. For the 3:2 pulldown, in 60 Hz, it does not change 48 frames but it creates 12 blended frames.

Frame blending is how oversample works in mpv.

[Akemi]

I guess ?

why are you guessing instead of properly explaining what you want?

Is it Smooth Motion you don't understand ?

linking a google search is not an explanation. obviously you should explain what Smooth Motion is, because that's what you want?

Or the difference with/without video-sync=display-resample ?

i am well aware what display-resample does.

Display-resample also has some problems, and is totally overkill when having a correct (high) display refresh rate.

what problems?

if you want interpolation without display-resample does that mean you want to interpolate to arbitrary destination fps? instead of arbitrary, should it interpolate to the smallest divisible integer fps, eg 24fps src, ~60hz monitor, interpolate from 24fps->~30fps since 60/30=2 (would be same case for 120hz). so it would safe some CPU cycles?

[Nojevah]

I don't understand. How is display-resample overkill?

I just check my CPU and GPU stats and can see more ressources are used. It is also confirmed by developers ("- thus increasing power usage by a factor of 2x-3x in such a case")
Use that much ressources has no sense, especially if you have an alternative interpolation methode which can use way less ressources.

If you care about accurate frame timings (which is baked into the entire premise of this question), you need this.

No. There are several ways to achieve accurate frame timings, for exemple, good refresh rates.

Are you asking for the ability to use interpolation but without setting one of the display- modes?

Looks like the title, except I don't need t_scale settings, just a basic frames blender when needed.

It's not technically impossible, but why would you want this? If you want to eliminate the appearance of judder as much as possible (one of the reasons to use interpolation in the first), you need to care about your display's actual refresh rate which is what the display-resample option does.

display-resample doesn't aim to reduce 3:2 pulldown. And if your refresh rate is good, display-resample is not needed (except if you want to use interpolation, but it is not recommended if you don't want blur/flickering).
As madshi (I think he is someone who can be trusted) explains in the past, he does not want to touch audio, so it's a first reason not wanting to use display-resample. The second reason is the ressources, since it is not needed when you have perfect refresh rate.
The "need to care about your display's actual refresh rate" has nothing to do with display-resample. It's the user task to select/create the right refresh rate.

Frame blending is how oversample works in mpv.

But it works only with display-resample, which uses more ressources and causes more flickering.

why are you guessing instead of properly explaining what you want?

linking a google search is not an explanation. obviously you should explain what Smooth Motion is, because that's what you want?

Well. I don't know how to put it nicely without create a drama... If you don't understand and don't ask precise question, maybe you can ignore this thread and we'll be fine. OK ?

what problems?

Well, that's not the aim of this ticket, but here is an example: #6271

if you want interpolation without display-resample does that mean you want to interpolate to arbitrary destination fps? instead of arbitrary, should it interpolate to the smallest divisible integer fps, eg 24fps src, ~60hz monitor, interpolate from 24fps->~30fps since 60/30=2 (would be same case for 120hz). so it would safe some CPU cycles?

There's no "arbitrary destination fps" since this algorithm doesn't change fps. It just modifies some frames (by blending 2).
Well, another way to understand Smooth Motion (for those interested) is to read madshi's posts from this thread.

[Dudemanguy]

I just check my CPU and GPU stats and can see more ressources are used. It is also confirmed by developers ("- thus increasing power usage by a factor of 2x-3x in such a case")

Of course it uses more resources, it renders a frame on every vsync for timing reasons. The 2x-3x is an absolute worse case scenario (same frame being repeated over and over again). But yes, better video playback taxes your computer a bit more. That's how it works.

Use that much ressources has no sense, especially if you have an alternative interpolation methode which can use way less ressources.

display-resample is not an interpolation method. It is a setting that is required to use various interpolation methods. Don't confuse the two.

No. There are several ways to achieve accurate frame timings, for exemple, good refresh rates.

I do not know about madVRs internals, but simply setting a custom display mode does not guarantee that the audio clock and video clock will never drift from each other. It is true that setting a "good" refresh rate (like an integer multiple of your video fps) will probably have less issues than the standard 60hz display, but it is not a solution to audio/video clocks drifting from each other. Also this isn't the job of mpv to set your display modes, we're crossplatform. Use whatever your OS provides for this (I do this on my own machine).

Looks like the title, except I don't need t_scale settings, just a basic frames blender when needed.

Sure but asking to do temporal interpolation without knowing when frames actually occur doesn't make sense. You're asking for the A A A+B B C C C+D.... etc. pattern (aka oversample in tscale). How do you want us to blend frames correctly together if we aren't rendering on every vsync (which is what display-resample does)? Yes, this uses more resources than the standard audio-clock guided playback. That's just how it works if you want more accurate playback and/or do fancy interpolation tricks.

display-resample doesn't aim to reduce 3:2 pulldown

I wasn't suggesting this, but see the above for the explanation on why it's needed.

blur/flickering

I don't understand what you mean by this. display-resample doesn't cause flickering. Using interpolation will blur the convoluted frame to a certain degree depending on your tscale. If you don't want this, don't use it.

It's the user task to select/create the right refresh rate.

If you are on the correct refresh rate (i.e. an integer multiple of the video), then you don't need interpolation.

[Akemi]

Well. I don't know how to put it nicely without create a drama... If you don't understand and don't ask precise question, maybe you can ignore this thread and we'll be fine. OK ?

it's not my responsibility to ask overly precise questions if you can't put your wanted behaviour into precise words.

you do realise i am asking because we (developers) don't know what you want from us? if you want me to ignore this feature request that's fine by me, though it will either get closed or ignored that way.

There's no "arbitrary destination fps" since this algorithm doesn't change fps. It just modifies some frames (by blending 2).
Well, another way to understand Smooth Motion (for those interested) is to read madshi's posts from this thread.

maybe you should read the comment you just linked yourself:

Introducing the new "smooth motion" frame rate changer (FRC) with the following main features / benefits:

(1) Can convert any source frame rate to any display refresh rate, while maintaining smooth motion.

i dunno maybe you are misunderstanding

[Nojevah]

Of course it uses more resources, [...] But yes, better video playback taxes your computer a bit more. That's how it works.

Well, that's not how madVR Smooth Motion works. Negligeable ressources. display-resample is using not negligeable ressources.

display-resample is not an interpolation method. It is a setting that is required to use various interpolation methods. Don't confuse the two.

Did I say it is an interpolation method ? No. I know what display-resample does. The alternative interpolation method I mentionned is "madVR Smooth Motion" which does NOT need to resample audio or change video speed or other things display-resample do.

How do you want us to blend frames correctly together if we aren't rendering on every vsync (which is what display-resample does)?

Well, that's the reason of my ticket in fact. To motivate a developer who knows what it needs to make the correct algorithm, like madVR did (or Zachs with his Fluid Motion in MPDN).

I don't understand what you mean by this. display-resample doesn't cause flickering. Using interpolation will blur the convoluted frame to a certain degree depending on your tscale. If you don't want this, don't use it.

madVR smooth Motion doesn't flicker as much as display-resample + t_scale = oversample. And that's because of display-resample.

it's not my responsibility to ask overly precise questions if you can't put your wanted behaviour into precise words.

you do realise i am asking because we (developers) don't know what you want from us? if you want me to ignore this feature request that's fine by me, though it will either get closed or ignored that way.

Well, I explained myself clearly, I can't explain Smooth Motion better than madshi did. Smooth Motion is quite known now, and you'll find references to this in other tickets, and it's a requested feature also in other applications like VLC for example.
I hope this ticket won't be closed though, I prefer it to be in sleepy state, in case a developer reads feature requests by google and catches this one. ;-)

i dunno maybe you are misunderstanding

Well, if you don't understand me, and if you don't understand explanation from original developers, I don't know what to say.

But if you don't understand, that's not a crime, I just hope one developer (who understands what I'm talking about) is interested in implement this in mpv.

[Akemi]

we prefer to not deal with this like this then.

[ghost]

If you need something primitive that "upconverts" to a fixed rate, you can just use one of those ffmpeg filters.

[Nojevah]

"We"... Yeah. w/e

@wm4: I'm not sure to understand, since Smooth Motion is not primitive to me, but if you know a command/syntax with ffmpeg which allows to mimic madVR Smooth Motion, I'm interested (just remember I'm a basic user and no developer).

[Dudemanguy]

Well, that's not how madVR Smooth Motion works. Negligeable ressources. display-resample is using not negligeable ressources.

madshi said that smooth motion also increases resources in the thread you linked. display-resample on default settings should not significantly increase resources either. Please quantify what the difference is between the two and make sure the settings are the same otherwise (scaling, etc.).

The alternative interpolation method I mentionned is "madVR Smooth Motion" which does NOT need to resample audio or change video speed or other things display-resample do.

Oversample is "smooth motion". It's not a different interpolation method. Also, display-resample does not change the video speed (quite the opposite actually). It resamples the audio so it stays in sync with the video. Again, I don't know the internals of madvr (nobody does except madshi), but since smooth motion/oversample is a temporal interpolation (i.e. along time), I would also expect it to render a frame on every vsync for timing reasons and so it correctly knows what frames to blend together. This is a necessarily more expensive operation (which madshi said as well).

madVR smooth Motion doesn't flicker as much as display-resample + t_scale = oversample. And that's because of display-resample.

What do you mean by "flickering"? When someone says their display flickers, that normally means it blinks off and on rapidly. display-resample doesn't do this, so I still have no idea what you mean.

[Nojevah]

Please quantify what the difference is between the two and make sure the settings are the same otherwise (scaling, etc.).

Here is a comparison. Scaling settings are not important, we just want to compare Smooth Motion (enabled/disabled):
((Edit: I forgot to mention, you have to look at the right part of graphs, the left part is before I launch video))

The difference is really visible (higher GPU Clock and D3D usage, higher CPU).
I have a powerful GPU as you can see, but with an Intel GPU, display-resample is a no-go.

Also, display-resample does not change the video speed (quite the opposite actually).

It does. Yes it resamples audio but it also changes video speed: "this mode will also minimize unnecessary frame drops by slightly changing the video playback speed "

This is a necessarily more expensive operation (which madshi said as well).

Yes, it costs a bit, but it's negligible, as you can see on my screenshot. On an Intel GPU, it is noticeable, but not that much.

What do you mean by "flickering"? When someone says their display flickers, that normally means it blinks off and on rapidly. display-resample doesn't do this, so I still have no idea what you mean.

Smooth Motion implies flickering, and it is noticeable in end credits for example. If you compare madVR Smooth Motion and tscale=oversample, you'll see there's more flickering in mpv. If you want less flickering, you have to change tscale, but then all tscales will have blur (a lot depending on the setting).

[Dudemanguy]

Scaling settings are not important

Of course it matters. If you don't start at the same baseline, then comparing the increases between the two is meaningless. You can probably spit out bilinear super fast but other settings may not be as simple.

I have a powerful GPU as you can see, but with an Intel GPU, display-resample is a no-go.

I can use display-resample on my crappy intel laptop just fine.

It does. Yes it resamples audio but it also changes video speed: "this mode will also minimize unnecessary frame drops by slightly changing the video playback speed "

This is my mistake. I forgot that it did a slight adjustment on refresh rates.

Yes, it costs a bit, but it's negligible, as you can see on my screenshot. On an Intel GPU, it is noticeable, but not that much.

A 7% increase isn't negligible.

Smooth Motion implies flickering, and it is noticeable in end credits for example. If you compare madVR Smooth Motion and tscale=oversample, you'll see there's more flickering in mpv. If you want less flickering, you have to change tscale, but then all tscales will have blur (a lot depending on the setting).

I still have no idea what you mean by flickering. Do mean to say that the motion has more judder? You can change the tscale parameters to your liking.

[Nojevah]

Of course it matters. If you don't start at the same baseline, then comparing the increases between the two is meaningless. You can probably spit out bilinear super fast but other settings may not be as simple.

Scaling algorithms, debanding, dithering etc. are nearly the same, they don't have impact for this comparison. Don't you find strange that without SM, mpv uses less ressources than madVR, then using SM, mpv uses more ressources than madVR ?

A 7% increase isn't negligible.

There's not 7% increase. I guess you took the last values on graph, but if you compare CPU curves in time, you can see they are identical (my CPU, an i3, is too powerful to show difference). For GPU, it's the same clock frequencies, and again, D3D usage is identical (powerful card).

I still have no idea what you mean by flickering. Do mean to say that the motion has more judder?

Well, on end credits (black screen, white text), it's easily perceived, it seems images are blinking, like if there was a black screen, white text, black screen, white text, etc. That's a side effect of judder indeed, but the blinking/flickering is amplified with smooth motion (I can't find a link, but I though I read it from haasn for example, but maybe it was madshi).
And I can tell you it is more noticeable on mpv than on madVR (with SM On for both of course, no differences otherwise).

You can change the tscale parameters to your liking.

Except oversampling, others ("Convolution-based") cause way too much blur to be usable.

[Dudemanguy]

Scaling algorithms, debanding, dithering etc. are nearly the same, they don't have impact for this comparison.

Nearly the same? They need to be exactly the same otherwise, again, this comparison is meaningless.

Don't you find strange that without SM, mpv uses less ressources than madVR, then using SM, mpv uses more ressources than madVR ?

No, because mpv's defaults are not equal to madVR's defaults and I have no idea what settings you are using otherwise for either program. According to your screenshot, the CPU clock is always lower in mpv though. But again see above.

There's not 7% increase. I guess you took the last values on graph

I have no idea how long this sample lasts and that's the hard number given.

Well, on end credits (black screen, white text), it's easily perceived, it seems images are blinking, like if there was a black screen, white text, black screen, white text, etc.

I have never ever seen this.

Except oversampling, others ("Convolution-based") cause way too much blur to be usable.

It's not just the scale itself. Various other things like tscale-clamp are adjustable. You can play with them.

[kevmitch]

For what it's worth, madvr must be using something like video-sync=display-vdrop by default because it does measure frame timings even with smoothmotion turned off as can be seen by constantly updating refresh rate in the stats screen.

It sounds like display-vdrop is what the this user wants:

1. don't touch audio
2. use minimal resources
3. have enough timing information to run frame interpolation (oversample / smoothmotion in this case)

[ghost]

If it's about random guessing what madvr might do: maybe it doesn't redraw on every display vsync, and does "repeating" of frames simply by waiting, or it switches the framerate, or makes the compositor take in the desired framerate.

mpv's display-vdrop still redraws the frame for each display frame, it's just display-sync without audio resampling. (Redrawing on every display frame is, in my opinion, still the least messy way tp lock on reliable to the display refresh rate.)

[Nojevah]

@Dudemanguy : Let's agree to disagree, I find my comparison very relevant, independent to scaling etc.

@kevmitch: I guess you know the diff between "audio" and "display-vdrop", in my case I don't, since I trust the manual: "(Although it should have the same effects as audio, the implementation is very different."). But anyhow, you understand what I want.

@wm4: You can find information by madshi through this link: https://forum.doom9.org/search.php?do=process&showposts=1&searchuser=madshi&query=smooth+motion&order=ascending&searchdate=2640 (you may need to change 2640 by a higher value in order to have posts from 20th February 2013 if you click this link in some months)
He explains several time for people who does not understand the principle and give details.

[ghost]

I won't crawl through that awful forum just to try to guess things I either don't care about or already know. If you present me the information on a silver plate, then maybe I might care. (That's just how OSS works.)

[Nojevah]

As I'm no developer, and madVR closed source (MPDN too), I can't give you much. My words/posts would be useless boring blabla for you.
But since that algorithm is really well done and useful, I hope one day a developer will find the need to create it in an OS player (mpv, preferred choice, but I guess it could be ported from VLC to mpv).

[haasn]

I have no idea what's going on in this thread (and I stopped reading somepoint midway) but I wanted to quickly point out a few things

1. You can avoid the resampling by using a different DS mode, for example display-vdrop. While I haven't really investigated this closely, I was under the impression that this is effectively what madVR is doing. (With the note, of course, that madVR will also change the display refresh rate to minimize the rate of vdrops that are necessary) That said, I think the CPU cost of resampling is generally negligible during playback, and worth the complexity it avoids.
2. Adding madVR-style custom display clocks might be a possibiity, but it's the kind of thing that could probably be served by a lua script. It's highly platform-specific and afaict on madVR it requires "setup", i.e. it doesn't happen on the fly but rather there's an extra tool you have to run to measure and create these modes. You can effectively do the equivalent of this yourself with e.g. X display lines, plus a script to auto-select the right display mode based on the content you're watching. (This is more or less what madVR is doing, right?)
3. The quote about "2-3x power usage" was misunderstood / taken out of context. It's specifically to avoid this 2-3x power usage increase that we use the frame cache FBO to minimize the cost of frame re-paints. (See the next sentence on the page you quoted) That being said, it's true that blitting is still a nonzero cost.
4. I believe this blit is the main source of the inefficiency in the mpv interpolation algorithm (however I haven't benchmraked it). In theory we could try switching to a zero-copy code path in which existing frames are either simply re-queued for presentation (vulkan would easily allow this, dunno about the other APIs), or even better, we could simply block for another vsync without even queueing a presentation event. (Some APIs have various timing extensions that could theoretically allow this). The tl;dr on this is it's complicated and platform specific, which is the main reason it hasn't been done. There's also the caveat of OSD changes necessitating a repaint that would need to be taken into consideration. (Do we skip the cache FBO and always do a full redraw on OSD changes? Do we cut our losses by always drawing to a cache FBO even when we won't read from it, just in case we need to repaint for the OSD? Do we throttle the frequency at which OSD updates can happen, so that e.g. 24 Hz video can only have 24 Hz OSD updates, rather than 60 Hz? etc.)
5. Measuring CPU/GPU usage is incredibly difficult, since a lot of factors can influence it or make statistics misleading. I think the most useful metric is power consumption, and also, "what resolution can I still play smoothly without framedrops?". Looking at the clock speeds I think is an especially misleading stat, since a CPU spending less time at a higher clock is better for power consumption than a CPU spending more time at a lower clock, and specifics of how this sampling and averaging are done can strongly influence what the "average clock speed" ends up being. It may simply be the case that it's waking up more, or clocking up at a rate disproportionate to the extra load being generated.

Anyway I'd probably recommend closing this issue in favor for more specific issues addressing the individual points above, since such "please make things faster" meta-issues are likely to end up stalled.

[Nojevah]

1. You can avoid the resampling by using a different DS mode, for example display-vdrop. While I haven't really investigated this closely, I was under the impression that this is effectively what madVR is doing. (With the note, of course, that madVR will also change the display refresh rate to minimize the rate of vdrops that are necessary) That said, I think the CPU cost of resampling is generally negligible during playback, and worth the complexity it avoids.

madVR does not touch audio, so indeed, no resampling.
Even if there's a setting in madVR to change refresh rate, madVR Smooth Motion does not change refresh rate at all. It uses display refresh rates offered by system (GPU drivers).

2. Adding madVR-style custom display clocks might be a possibiity, but it's the kind of thing that could probably be served by a lua script. It's highly platform-specific and afaict on madVR it requires "setup", i.e. it doesn't happen on the fly but rather there's an extra tool you have to run to measure and create these modes. You can effectively do the equivalent of this yourself with e.g. X display lines, plus a script to auto-select the right display mode based on the content you're watching. (This is more or less what madVR is doing, right?)

Indeed, madVR has an integrated tool to create custom refresh rate, but it has nothing to do with Smooth Motion. And I don't want mpv to do that. To be honest, this integrated tool in madVR should be in a separate app, since it's particular, can be broken easily (because it relyes a lot on GPU broken drivers).
But we can forget about this point, that's not linked to Smooth Motion

3. The quote about "2-3x power usage" was misunderstood / taken out of context. It's specifically to avoid this 2-3x power usage increase that we use the frame cache FBO to minimize the cost of frame re-paints. (See the next sentence on the page you quoted) That being said, it's true that blitting is still a nonzero cost.
4. Measuring CPU/GPU usage is incredibly difficult, since a lot of factors can influence it or make statistics misleading. I think the most useful metric is power consumption, and also, "what resolution can I still play smoothly without framedrops?". Looking at the clock speeds I think is an especially misleading stat, since a CPU spending less time at a higher clock is better for power consumption than a CPU spending more time at a lower clock, and specifics of how this sampling and averaging are done can strongly influence what the "average clock speed" ends up being. It may simply be the case that it's waking up more, or clocking up at a rate disproportionate to the extra load being generated.

It appears I have a voltage monitor plug, so I can compare graphs and power consumption. I can assure you my graphs are really consistent. The spike in consumption is when GPU clock goes > 1000 MHz (which is not the case in my graphs). It's also true for CPU, but it is less marked.

I kept these 3 graphs after carefully comparing real power consumption and these stats. These 3 are the ones which match consumption. D3D usage has less importance, it is just for comparison when GPU clock are equal between 2 tests (the higher the D3D Usage, the higher the chance to use a higher GPU Clock)

4. I believe this blit is the main source of the inefficiency in the mpv interpolation algorithm (however I haven't benchmraked it).

There's already an increase of GPU ressources without interpolation, just with display-resample.
For the rest of your 4, it's for developers, so I let you between pros.

Anyway I'd probably recommend closing this issue in favor for more specific issues addressing the individual points above, since such "please make things faster" meta-issues are likely to end up stalled.

I didn't ask to make things faster. I've waited several months before to create this ticket because I remembered of a ticket about implementing Smooth Motion and it had been closed because the solution was to use display-resample and oversample, and for me it's not the same as madVR Smooth Motion.
Unfortunately, I expected this request to be closed (hence the last sentence of my first post ;-) )
For me it's an individual feature, since I'm not interested in using display-resample. I'm fine with the default video-sync=audio:

• no GPU ressources used
• no frames dropped/repeated with my TV thanks to matching refresh rates (with audio clock deviation)
• I have a high refresh rate monitor, so dropped/repeated frames are way less noticeable.

But with a 60Hz monitor, Smooth Motion is THE solution to handle judder like 3:2 pulldown.
By the way, maybe you already know this, but with madVR Smooth Motion, there's no repeated/dropped frame, there'll be blended frames at worst.

[ghost]

The fuck?

[haasn]

There's already an increase of GPU ressources without interpolation, just with display-resample.

Any DS mode requires blitting/repainting. It's simply a consequence of how we measure the display clock (by presenting to every vsync event). And actually, I even forgot that the interpolation modes don't even blit, they always undergo a full sample from the FBO, so even an optimization to avoid blitting wouldn't help the performance of interpolation.

Speaking of which, I just saw that even the oversample mode always blends two frames, even in the case in which the mixing coefficient is 1.0 or 0.0. I pushed a change for this to the oversample_mixless branch. Dunno if it helps. Maybe you can benchmark it?

(Side note, I also realized that drivers might have optimizations to make re-queueing the same frame for presentation twice in a row be the equivalent of just blocking for another vsync. (And if they don't, that would arguably be worth a bug report). So regarding my earlier note, simply queueing a frame multiple times might be the most realistic path towards achieving this zero-copy blit. Assuming the blit is a source of significant GPU load.)

By the way, maybe you already know this, but with madVR Smooth Motion, there's no repeated/dropped frame, there'll be blended frames at worst.

This is wrong. The smoothmotion algorithm does repeat frames, exactly the way tscale=oversample does. The two are algorithmically identical. Any inefficiency you're measuring is an inefficiency in the implementation, but there's no fundamental difference in design. Which is sort of exactly why this issue isn't likely to get far, since you seem to be arguing from sort of standpoint of smooth motion being fundamentally different enough to be worth a feature request.

[Nojevah]

Any DS mode requires blitting/repainting. It's simply a consequence of how we measure the display clock (by presenting to every vsync event)

I don't know if I understand it well but the default display=audio has no cost on GPU.

so even an optimization to avoid blitting wouldn't help the performance of interpolation.

In fact, some optimization would be to have a diffrent mode than display-resample.
Display-resample does 2 thing that I don't need (and might cost some ressources):

• resampling (or trying) audio
• changing (or trying) video speed
  That's why in my request I ask a method which would not use display-resample.

Speaking of which, I just saw that even the oversample mode always blends two frames, even in the case in which the mixing coefficient is 1.0 or 0.0. I pushed a change for this to the oversample_mixless branch. Dunno if it helps. Maybe you can benchmark it?

I'll try to benchmark this in next Shinchiro build, but I don't expect major changes in terms of performance. But it could have a positive impact on flickering. It could explain why I perceived more flickering than madVR.

(And if they don't, that would arguably be worth a bug report).

Report a bug to GPU brands ? Wow. I though nobody had success with this. But anyway, it needs to come from a true developers who knows what he's talking about, not me.

This is wrong. The smoothmotion algorithm does repeat frames, exactly the way tscale=oversample does. The two are algorithmically identical. Any inefficiency you're measuring is an inefficiency in the implementation, but there's no fundamental difference in design. Which is sort of exactly why this issue isn't likely to get far, since you seem to be arguing from sort of standpoint of smooth motion being fundamentally different enough to be worth a feature request.

My bad, I explained it badly. Of course they will be repeated frames (A A A+B B C C C+D). But I meant in cases like 59,94 fps on 60 Hz, let's forget about the 2 repeated frames each time, there will be 1 time when a frame would be repeated 3 times, and that's when Smooth Motion prevents that in blending two frames (hence I said no repeated/dropped frames, it was just in case of mismatch).
I'm aware oversample is an equivalent of Smooth Motion (except the flickering that I though was linked to display-resample), but it is not the same because it needs display-resample which does stuff I don't want.

If you manage to have the "oversample" works without display-resample and if it uses minimal ressources, my request would have been totally fulfilled.

And thanks for your time and the constructive dialog.

[ghost]

That "flickering" you mention (and which we can't know what it's supposed to be) is probably just because your GPU can't keep up with the high refresh rate.

[haasn]

Well anyway it sounds like this issue can remain closed. I'm not reading much else out of this besides questions which have either already been answered (use display-vdrop to avoid resampling), and a request to make interpolation more GPU efficient, which my branch should. (To which extent, though, remains to be determined)

I'll leave it rest for now.

[ghost]

Also, no idea why you'd want to disable audio resampling though. Maybe an audiophile argument? What audiophile dumbasses don't know is that anything audio related probably has a resampler hidden somewhere for synchronization purposes, whether it's in hardware, OS audio code, or when the audio/video was produced.

[Nojevah]

@wm4: No. It's not this kind of issue. In fact, forget about it because I can't explain it clearly. I'll try again but I would not be surprised if you still don't understand (my bad again):
This flickering is visible only with credits ends, with slow moving white text on black background. The slow movement with low fps videos (classic 23,976 fps for example) already causes this flickering even without no settings at all (every players), but Smooth Motion amplifies this (because of blended frames).

@haasn: the default video-sync=audio does not resample too. And my request was not to improve interpolation, but not using display-resample. Thanks again for your time, it was nice to have your opinion.

@wm4: Tbh, I don't care about resampling or sound quality, it's just that I don't need changing speed for audio or video in fact. Let audio plays like it has to, let video follows audio (that's why video-sync=audio is fine to me), and when there will be a need to repeat/drop a frame, just blend it in order to have a better motion sensation instead of a "jerk".

[Hrxn]

@Nojevah What kind of display are you using exactly? 120 Hz? Any FreeSync, G-Sync, Whatever?

[Nojevah]

No, it's a TN 120/144 Hz, I don't use Freesync, G-sync etc. But the flickering is not linked to monitor, it is visible on all 60 Hz LCD panels (my TV included). Maybe it's less marked on OLED panels...
But don't forget, this flickering is noticeable only with end credits, it's impossible to detect it in other situations (but it might have a negative impact on visual quality).

[Hrxn]

Yeah, that was my next question. But if you've already tried this with different displays, 60 Hz included...
I guess that rules it out then.

[Robot-DaneelOlivaw]

I use video-sync=display-resample interpolation=yes tscale=oversample to eliminate judder on my MacBook Pro. This configuration does increase load and drain battery much faster. But it's acceptable since I can disable it when the laptop isn't charged.

However, I find it weird that mpv still uses up a lot of power even if frame-rate of video matches refresh-rate of screen (e.g. playing 60fps video on 60Hz screen). Things remain similar after interpolation is disabled. Currently I use an auto-profile to bypass these situations:

[Judder-free]
profile-desc=cond:((p["estimated-vf-fps"]>29 and p["estimated-vf-fps"]<31) or (p["estimated-vf-fps"]>59 and p["estimated-vf-fps"]<61))
video-sync=audio
interpolation=no

[w-barath]

I think I know how frame resampling could be creating flickering for Nojevah. Gamma/power curve differences for the pixels on blended frames. ie luma 240 as white and 16 as black, 128 for blended between a black frame and a white frame. On a properly calibrated monitor with a gama of 2.2 for SRGB, the two adjacent 128 luma lines will perceptually appear the same brightness as a single luma 240 adjacent to a 16. On uncalibrated displays or with a host OS gamma other than 2.2, there will be apparent brightness changes from frame to frame introduced due to output power*area differences.

This effect is used for subjective gamma calibration of displays for this reason... and TN panels are notoriously difficult if not impossible to get accurate gamma for since gamma changes with viewing angle, esp. vertical viewing angle, resulting in TN panels having an effective gradient of gammas from top to bottom. Many cheaper TVs use TN and/or have poorer calibration.

So @Nojevah I would recommend you check for flickering on (P)VA and IPS panels, and also try adjusting the gamma through your OS display preferences to minimise the flickering.

Gamma (power curve) is frequently mis-labelled or simply not present in display preferences, sadly. I've seen it labelled as Contrast, Brightness, even Saturation. If your display is properly calibrated, gamma adjusts the brightness of mid-tones and leaves the extremes of bright and dark alone. If the display is mis-calibrated then gamma may alter the amount of black crush, highlight clipping, lower the darkness, or raise the brightness. That's how it gets mis-labelled... by developers and manufacturers who don't understand their own hardware.

To check / adjust your calibration for LCD panels, see http://www.lagom.nl/lcd-test/gamma_calibration.php

Ideally make the changes via the monitor's settings rather than the OS display driver settings. If you attach multiple displays you can't separately adjust them from the OS. You'll also get higher image quality doing it at the display. The OS/driver must work within the limitations of the display's gamut (3D volume of reproducible colours.) If the display is calibrated the gamut may be increased, leading to more possible colours and a higher degree of fidelity.

[Nojevah]

Thanks for your investigation but I'm not interested in a solution based on display-resample for performance reason.
Fortunately, there's more and more high-refresh monitors which make smooth-motion interpolation not mandatory.