Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transition animation #25

Open
lsdlh opened this issue Nov 10, 2023 · 20 comments
Open

transition animation #25

lsdlh opened this issue Nov 10, 2023 · 20 comments

Comments

@lsdlh
Copy link

lsdlh commented Nov 10, 2023

How to add transition animations, such as fading in and out, when video stitching

@rezoo
Copy link
Owner

rezoo commented Nov 11, 2023

Thank you for your interest. The simplest method is to overlay two layers, similar to how it's done in Premiere or After Effects, and then add animation related to opacity.

For example, let's say we are creating a video that totals 7 seconds, combining a 3-second clip and a 5-second clip with a 1-second fade period. In this case, you would apply the animation as follows.

import movis as mv

w, h = 640, 480

img1 = mv.layer.Rectangle(size=(w, h), color='pink', duration=3.0)
img2 = mv.layer.Rectangle(size=(w, h), color='turquoise', duration=5.0)

scene = mv.layer.Composition(size=(w, h), duration=7.0)
scene.add_layer(img1, name='video1')
scene.add_layer(img2, name='video2', offset=2.0)
scene['video2'].opacity.enable_motion().extend(
    keyframes=[0.0, 1.0],
    values=[0.0, 1.0],
)

If you are using an IPython notebook, you can easily check the results with scene.render_and_play(). If not, please output the video using scene.write_video('video.mp4').

However, for simple fade processing, it would be better for me to implement a shortcut function like mv.fade().

@lsdlh
Copy link
Author

lsdlh commented Nov 20, 2023

感谢您的关注。最简单的方法是叠加两个图层,类似于 Premiere 或 After Effects 中的操作方式,然后添加与不透明度相关的动画。

例如,假设我们正在创建一个总长 7 秒的视频,其中将 3 秒剪辑和 5 秒剪辑以及 1 秒淡入淡出周期组合在一起。在这种情况下,您将按如下方式应用动画。

import movis as mv

w, h = 640, 480

img1 = mv.layer.Rectangle(size=(w, h), color='pink', duration=3.0)
img2 = mv.layer.Rectangle(size=(w, h), color='turquoise', duration=5.0)

scene = mv.layer.Composition(size=(w, h), duration=7.0)
scene.add_layer(img1, name='video1')
scene.add_layer(img2, name='video2', offset=2.0)
scene['video2'].opacity.enable_motion().extend(
    keyframes=[0.0, 1.0],
    values=[0.0, 1.0],
)

如果您使用的是 IPython 笔记本,您可以使用 轻松检查结果scene.render_and_play()。如果没有,请使用 输出视频scene.write_video('video.mp4')

不过,对于简单的淡入淡出处理,我最好实现一个像mv.fade()

Great, thank you very much, hope to add more preset special effects.

@rezoo
Copy link
Owner

rezoo commented Nov 20, 2023

感谢您的建议。我们正在考虑如何在当前的界面中添加这些典型的效果,但无论如何,我们都认为这是一个非常重要的问题。如果您有其他任何要求,请随时告诉我们。

Thank you for your suggestion. We are considering how to add such typical effects in the current interface, but in any case, we think it is a very important issue. If you have any other requests, please feel free to let us know.

@lsdlh
Copy link
Author

lsdlh commented Nov 20, 2023

感谢您的建议。我们正在考虑如何在当前的界面中添加这些典型的效果,但无论如何,我们都认为这是一个非常重要的问题。如果您有其他任何要求,请随时告诉我们。

Thank you for your suggestion. We are considering how to add such typical effects in the current interface, but in any case, we think it is a very important issue. If you have any other requests, please feel free to let us know.

Conveniently adding filters to videos and adding transition effects when splicing multiple video segments are extremely useful for video editing.

@rezoo
Copy link
Owner

rezoo commented Nov 20, 2023

#37

我目前正在创建一些快捷函数,以便简单地实现淡入和淡出效果。我认为这些函数可能会直观易用,但如果您有任何不明白的地方,请告诉我。

I am currently creating a set of shortcut functions to easily implement fade-in and fade-out effects. I believe these functions will likely be intuitive to use, but if there is anything unclear, please let me know.

@lsdlh
Copy link
Author

lsdlh commented Nov 24, 2023

#37

我目前正在创建一些快捷函数,以便简单地实现淡入和淡出效果。我认为这些函数可能会直观易用,但如果您有任何不明白的地方,请告诉我。

I am currently creating a set of shortcut functions to easily implement fade-in and fade-out effects. I believe these functions will likely be intuitive to use, but if there is anything unclear, please let me know.

scene1 = mv.layer.Image.from_color(size=(640, 480), color='pink', duration=5.0)
scene2 = mv.layer.Image.from_color(size=(640, 480), color='turquoise', duration=5.0)
scene3 = mv.fade_in(scene1,duration=2.0)
scene4 = mv.fade_out(scene2, duration=2.0)
scene3.write_video(r'D:\video\2\01.mp4')
scene4.write_video(r'D:\video\2\02.mp4')
How come the fade in and fade out feel the same? The effects are always at the end.

@lsdlh
Copy link
Author

lsdlh commented Nov 24, 2023

#37

我目前正在创建一些快捷函数,以便简单地实现淡入和淡出效果。我认为这些函数可能会直观易用,但如果您有任何不明白的地方,请告诉我。

I am currently creating a set of shortcut functions to easily implement fade-in and fade-out effects. I believe these functions will likely be intuitive to use, but if there is anything unclear, please let me know.

layer = mv.layer.Video(r'D:\video\2\07.mp4')
scene1 = mv.trim(layer, start_times=[10.0], end_times=[13.0])
scene2 = mv.trim(layer, start_times=[50.0], end_times=[53.0])
scene3 = mv.concatenate([mv.fade_out(scene1, duration=1), mv.fade_in(scene2, duration=1)])
scene3.write_video(r'D:\video\2\08.mp4')

If there are more clips to trim, is there a simpler way to add fade-ins and fade-outs, And the speed is very slow

@rezoo
Copy link
Owner

rezoo commented Nov 24, 2023

In that case, these codes can be simplified as follows:

layer = mv.layer.Video('input.mp4')
scene = mv.trim(layer, start_times=[10, 50], end_times=[13, 53])
scene = mv.fade_in_out(scene, 1.0, 1.0)
scene.write_video('output.mp4')

the speed is very slow

I need to investigate what is causing the slowness. If the same level of slowness occurs even when outputting with the following type of code, then it is an issue with ffmpeg. If not, then it is a problem on the composition side. I don't often do operations that create many compositions, so there may be an issue there.

layer = mv.layer.Video('input.mp4')
scene = mv.layer.Composition(size=layer.size, duration=layer.duration)
scene.add_layer(layer)
scene.write_video('output.mp4')

@lsdlh
Copy link
Author

lsdlh commented Nov 27, 2023

在这种情况下,这些代码可以简化如下:

layer = mv.layer.Video('input.mp4')
scene = mv.trim(layer, start_times=[10, 50], end_times=[13, 53])
scene = mv.fade_in_out(scene, 1.0, 1.0)
scene.write_video('output.mp4')

速度很慢

我需要调查导致缓慢的原因。如果使用以下类型的代码输出时也出现相同程度的缓慢,那么这是 ffmpeg 的问题。如果不是,那么就是构图方面的问题。我不经常进行创建许多合成的操作,因此那里可能存在问题。

layer = mv.layer.Video('input.mp4')
scene = mv.layer.Composition(size=layer.size, duration=layer.duration)
scene.add_layer(layer)
scene.write_video('output.mp4')

13th Gen Intel(R) Core(TM) i7-13700H 2.40 GHz
image
Is it slow because of repeated video encoding? If it’s the same video, is encoding not necessary?

@lsdlh
Copy link
Author

lsdlh commented Nov 27, 2023

在这种情况下,这些代码可以简化如下:

layer = mv.layer.Video('input.mp4')
scene = mv.trim(layer, start_times=[10, 50], end_times=[13, 53])
scene = mv.fade_in_out(scene, 1.0, 1.0)
scene.write_video('output.mp4')

速度很慢

我需要调查导致缓慢的原因。如果使用以下类型的代码输出时也出现相同程度的缓慢,这就是 ffmpeg 的问题。如果不是,那么就是构图方面的问题。我不经常进行创建许多如此合成的操作,因此那里可能存在问题。

layer = mv.layer.Video('input.mp4')
scene = mv.layer.Composition(size=layer.size, duration=layer.duration)
scene.add_layer(layer)
scene.write_video('output.mp4')

image

image

image

image

@rezoo
Copy link
Owner

rezoo commented Nov 29, 2023

Thank you. I have partially understood the situation.

The reason why we're currently getting 6.24 iter/sec instead of the expected 14.73 iter/sec is likely due to the processing of mv.trim(). Specifically, the problem seems to be on the imageio-ffmpeg side, where intermittent seeking creates more load than continuous playback.
If removing mv.fade_in_out and running write_video() results in the same 6.24 iter/sec performance, it would seem certain that the issue lies with the imageio-ffmpeg processing.

If possible, could you please confirm this? (If not, for instance, if mv.fade_in_out is causing the slowdown, this would be due to the composition, and we can handle it to some extent on our side).
In any case, I intend to write some code to better understand the situation.

@lsdlh
Copy link
Author

lsdlh commented Nov 29, 2023

Thank you. I have partially understood the situation.

The reason why we're currently getting 6.24 iter/sec instead of the expected 14.73 iter/sec is likely due to the processing of mv.trim(). Specifically, the problem seems to be on the imageio-ffmpeg side, where intermittent seeking creates more load than continuous playback. If removing mv.fade_in_out and running write_video() results in the same 6.24 iter/sec performance, it would seem certain that the issue lies with the imageio-ffmpeg processing.

If possible, could you please confirm this? (If not, for instance, if mv.fade_in_out is causing the slowdown, this would be due to the composition, and we can handle it to some extent on our side). In any case, I intend to write some code to better understand the situation.

Do not use mv.fade_in_out
image
use mv.fade_in_out
image

@rezoo
Copy link
Owner

rezoo commented Nov 29, 2023

I see. Currently I am considering to replace movis.ops functions with ones without compositions (#43).

  1. Since it takes some time to do the replication experiment, could you first run it with the current slow implementation?
  2. If a future implementation that doesn't use composition is completed, could you please try again after switching to that branch?

I do think that there is some slow cause in the composition caching mechanism, but since such a situation where it becomes 1/3 slower does not occur in my environment, it may take some time to reproduce the issue.

@lsdlh
Copy link
Author

lsdlh commented Nov 30, 2023

I see. Currently I am considering to replace movis.ops functions with ones without compositions (#43).

  1. Since it takes some time to do the replication experiment, could you first run it with the current slow implementation?
  2. If a future implementation that doesn't use composition is completed, could you please try again after switching to that branch?

I do think that there is some slow cause in the composition caching mechanism, but since such a situation where it becomes 1/3 slower does not occur in my environment, it may take some time to reproduce the issue.

No problem, thank you very much.

@rezoo
Copy link
Owner

rezoo commented Dec 12, 2023

#43 I have experimentally attempted to speed up using layers with mv.concatenate and mv.trim. Could you test from the master branch to see if it speeds up?

@lsdlh
Copy link
Author

lsdlh commented Dec 18, 2023

#43 I have experimentally attempted to speed up using layers with mv.concatenate and mv.trim. Could you test from the master branch to see if it speeds up?

Snipaste_2023-12-18_13-52-47
image
Thank you very much, it did change faster.

@dey-d
Copy link

dey-d commented Feb 26, 2024

Hi,

I'm trying to do a simple transition of a 2nd clip sliding in from the side, so far my code is per below. The issue is the second clip slides in between keyframes 5 and 6 but then it dissapears. What am I doing wrong? Is there a better way to do this? Thanks.

import os
import movis as mv
import numpy as np

w, h = 1080, 1920

scene = mv.layer.Composition(size=(w, h), duration=12.0)

scene.add_layer(mv.layer.Video("/home/deep/movis/browndog.mp4"), scale=1, name="browndog")
scene.add_layer(mv.layer.Video("/home/deep/movis/jackdog.mp4"), scale=1, name="jackdog")

initial_position = np.array([-w, 0])
final_position = np.array([w // 2, h // 2])

scene['jackdog'].position.enable_motion().extend(
keyframes=[5.0, 6.0],
values=[initial_position, final_position],
easings=['ease_in_out2'],
)

@dey-d
Copy link

dey-d commented Feb 26, 2024

I followed the bunny example on the google colab notebook so have kind of got it to work now however is there a way to add motion blur to the slide in/out?

@rezoo
Copy link
Owner

rezoo commented Feb 28, 2024

Hi,

At first glance, there does not seem to be anything odd in the code. I will now try to reproduce it with a simple code.
Motion blur has not yet been implemented. I thought it might be possible to implement it, but we thought that not many people would use it.

@dey-d
Copy link

dey-d commented Feb 28, 2024

Thank you! Motion blur would be great for quick transitions 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants