Results #199
Replies: 3 comments 2 replies
-
I recently ran into this issue trying to upscale fire (a lit "onion volcano" at a visit to Benihana Restaurant). Here's what I think is going on:
I also ran into something like this when this trying a simple test of interpolation stop-motion animation (taking some pictures and moving objects each time by hand). The objects were too far apart for the motion flow estimator to see them as the same object realistically moving. |
Beta Was this translation helpful? Give feedback.
-
|
@jhogsett |
Beta Was this translation helpful? Give feedback.
-
|
In my experience resynthesizing frames (replacing a destroyed frame by interpolating the two original adjacent frames has a kind of cleaning effect, where objects/dirt/noise not found in both frames tends to be ignored. This has the side effect of cleaning up the frames. In a sense, only visual information common to both frames is retained in the interpolated frame. It also seems size dependent. The larger the object is it seems, the more real the engine assumes it is, and it will pop into existence in the interpolated frame instead of being left out. The same is try with artificial slow motion. If you look at this example GIF-to-MP4 interpolation https://www.youtube.com/shorts/PLkOUxiFAuk you can see how the engine handles large objects popping in and out of existence near the edges of the frame. That was created using EMA-VFI. The effect is more pronounced with the same video interpolated using VFIformer https://www.youtube.com/shorts/IZdEITUQx3U . That fact is what convinced me to switch from VFIformer to EMA-VFI as my interpolation engine. |
Beta Was this translation helpful? Give feedback.
-
|
@jhogsett |
Beta Was this translation helpful? Give feedback.
-
I recommend you not use it then. |
Beta Was this translation helpful? Give feedback.
-
@jhogsett
My first impressions of the process are mixed.
Apart from my installation problems, the results of the program are not bad, but this product is not suitable for me.
On my rtx 3060 ti, 20 seconds of video was interpolated for more than 1.5 hours.
While the result obtained cannot be said to be very much better than the work of, for example, the same rife 4.6.
I used the "flowframes" program, where an hour-long video can be interpolated from 30 to 60 frames in 1 hour 30 minutes using rife on vapoursynth.
Yes, interpolation really reduces the quality of almost all models, and here EMA-VFI is comparable to other models as I see it. But most importantly, making one film for several days with not such a big difference in quality is too long.
It may be appropriate to use EMA-VFI in some video laboratories or on Pentagon computers so that the calculation speed is high and the result does not need to wait for days, but at home (especially if you cannot leave the PC turned on at night) it is not advisable I didn't see it.
I would also like to ask you, I noticed that in some videos, on static objects (this is especially visible in your video on YouTube - it’s called something like a “neural network stress test” with a man on a board in the sea with a lot of splashes) there is the effect of “blurring” the picture. As I understand it, the neural network for some reason mistakes some objects that are either static or quickly changing in size for a moving object. Can something be done about this?
Beta Was this translation helpful? Give feedback.
All reactions