Skip to content

Conversation

Aki-07
Copy link

@Aki-07 Aki-07 commented Oct 18, 2025

What does this PR do?

This PR introduces a new community pipeline named VideoInpaintPipeline, which extends the existing StableDiffusionInpaintPipeline to support temporally-coherent video inpainting. The pipeline allows users to perform text-guided video editing and inpainting while maintaining temporal consistency across frames, enabling realistic motion, stable lighting, and coherent textures even on mid-tier GPUs.

It achieves this by:

  • Reusing diffusion noise and latent representations across consecutive frames
  • Optionally applying optical-flow-guided warping (RAFT) for motion-aware latent propagation
  • Adding batched FP16 inference, torch.compile acceleration, and streaming video I/O
  • Providing a clean, high-level API consistent with other Diffusers pipelines

Fixes #12477

Before submitting

Who can review?

@asomoza

@Aki-07 Aki-07 force-pushed the feat/add-community-video-pipeline branch from ff3ed09 to 1310cee Compare October 18, 2025 19:18
@Aki-07 Aki-07 force-pushed the feat/add-community-video-pipeline branch from 1310cee to 09abe15 Compare October 18, 2025 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add VideoInpaintPipeline for temporally-consistent diffusion-based video inpainting

1 participant