Can't overlay gizeh animation onto video with transparency/mask

[dannyshisler]

I can't work out how to overlay a gizeh animation onto a video so that the vector graphics are visible but the background is transparent so the video is visible underneath the animation. I've tried lots of different ways and nothing seems to work. All I ever get is the gizeh animation completely hiding the underlying video.

This was my latest effort, just simply trying to draw a red line over the video, I've tried using the mask_color vfx method to create a mask that uses the Surface bg_color, but it doesn't have any effect.

import gizeh
from moviepy.editor import *

def make_frame(t):
    surface = gizeh.Surface(width=720, height=1280, bg_color=(0.5, 0.5, 0))
    line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=3, stroke=(1, 0, 0))
    line.draw(surface)
    return surface.get_npimage()


original_clip = VideoFileClip("test_original_video.mp4")
graphics_clip = VideoClip(make_frame, duration=original_clip.duration)
masked_graphics_clip = vfx.mask_color(graphics_clip, [0.5, 0.5, 0])

final_clip = CompositeVideoClip(
    [original_clip,
     graphics_clip],
    size=(720, 1280))

final_clip.write_videofile("test_output_video.mp4", fps=30))``` 

How do I do this? Thanks

[Zulko]

Two things you did wrong:

• Use surface.get_npimage(transparent=True) for getting a Gizeh image with a mask.
• In moviepy, transparency is achieved via a mask clip which at a time t returns a frame with values in 0-1.

So something like this (not entirely sure about syntax):

def make_frame(t):
    surface = gizeh.Surface(width=720, height=1280, bg_color=(0.5, 0.5, 0))
    line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=3, stroke=(1, 0, 0))
    line.draw(surface)
    return surface.get_npimage(transparent=True)


graphics_clip_mask = VideoClip(lambda t: make_frame[t][3] / 255.0, is_mask=True)
graphics_clip = VideoClip(lambda t: make_frame(t)[:3],
                          duration=original_clip.duration,
                          mask=graphics_clip_mask)

[dannyshisler]

Hi @Zulko thanks for getting back to me so quickly. I've tried to implement the solution you provided but I can't quite make it work.

This code, which is very close to what you've suggested, won't run:

import gizeh
from moviepy.editor import *


def make_frame(t):
    surface = gizeh.Surface(width=720, height=1280)
    line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=3, stroke=(1, 0, 0))
    line.draw(surface)
    return surface.get_npimage(transparent=True)


original_clip = VideoFileClip("test_original_video.mp4")

graphics_clip_mask = VideoClip(lambda t: make_frame(t)[3] / 255.0, 
                                     duration=original_clip.duration, 
                                     ismask=True)
graphics_clip = VideoClip(lambda t: make_frame(t)[:3], 
                           duration=original_clip.duration)
                           .set_mask(graphics_clip_mask)

final_clip = CompositeVideoClip(
    [original_clip,
     graphics_clip],
    size=(720, 1280))

final_clip.write_videofile("test_output_video.mp4", fps=30)

I get the error

...
File "/Users/danny/.local/share/virtualenvs/poc-Fdmdzxs7/lib/python3.7/site-packages/numpy/core/shape_base.py", line 234, in vstack
    return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly

If I modify return surface.get_npimage() so that (transparent=True) for the mask clip but (transparent=False) for the graphics clip then it runs but I only see the underlying video, not the graphics.

I feel like I'm very close but clearly I'm still doing something wrong.

[Zulko]

maybe the indices are wrong in my example, they should be make_frame(t)[:, :, 3] in graphics_clip_mask, and make_frame(t)[:, :, :3] in graphics_clip.

[dannyshisler]

Fantastic, that works!

The final working code is:

import gizeh
from moviepy.editor import *

def make_frame(t):
    surface = gizeh.Surface(width=720, height=1280)
    line = gizeh.polyline(points=[(0, 1180), (720, 1180)], stroke_width=10, stroke=(1, 0, 0))
    line.draw(surface)
    return surface.get_npimage(transparent=True)


original_clip = VideoFileClip("test_original_video.mp4")

graphics_clip_mask = VideoClip(lambda t: make_frame(t)[:, :, 3] / 255.0, duration=original_clip.duration, ismask=True)
graphics_clip = VideoClip(lambda t: make_frame(t)[:, :, :3], duration=original_clip.duration).set_mask(graphics_clip_mask)

final_clip = CompositeVideoClip(
    [original_clip,
     graphics_clip],
    size=(720, 1280)
)

final_clip.write_videofile("test_output_video.mp4", fps=30)

Thanks very much for your help.

[jitendra-koodo]

Getting a single gray/black pixel around the text when placed on a video following this mask approach.

Any solution for this?

import gizeh
from moviepy.editor import *

def make_frame(t):
    surface = gizeh.Surface(width=360, height=640)
    text_surface = gizeh.text(
        "Long String",
        fontfamily="Arial",
        fontsize=50,  # Font size
        fill=(1, 1, 1),  # White color
        xy=(360 / 2, 640 / 2),  # Centered horizontally and vertically
        h_align="center",
        v_align="center"
    )
    text_surface.draw(surface)
    return surface.get_npimage(transparent=True)

original_clip = VideoFileClip("background_video.mp4")

graphics_clip_mask = VideoClip(lambda t: make_frame(t)[:, :, 3] / 255.0, duration=original_clip.duration, ismask=True)
graphics_clip = VideoClip(lambda t: make_frame(t)[:, :, :3], duration=original_clip.duration).set_mask(graphics_clip_mask)

final_clip = CompositeVideoClip(
    [original_clip,
     graphics_clip],
    size=(360, 640)
)

final_clip.write_videofile("../output/test_output_video.mp4", fps=30)

[jitendra-koodo]

This is a trivial and simplistic use case : showing static text on a video. Is this Gizeh + Moviepy not a right solution? Moviepy + textclip have aliasing issues when text is scaled. Are there better combos/alternatives?