Stable Video Diffusion (SVD) Image-to-Video is a diffusion model that takes in a still image as a conditioning frame, and generates a video from it. In this tutorial we consider how to convert and run Stable Video Diffusion using OpenVINO. We will use stable-video-diffusion-img2video-xt model as example. Additionally, to speedup video generation process we apply AnimateLCM LoRA weights and run optimization with NNCF.
This notebook demonstrates how to convert and run stable video diffusion using OpenVINO.
Notebook contains the following steps:
- Create PyTorch model pipeline using Diffusers
- Convert Stable Video Diffusion Pipeline models to OpenVINO
- Convert Image Encoder
- Convert U-Net
- Convert VAE Encoder and Decoder
- Create Stable Video Diffusion Pipeline with OpenVINO
- Optimize pipeline with NNCF
- Compare results of original and optimized pipelines
- Interactive Demo
This is a self-contained example that relies solely on its own code.
We recommend running the notebook in a virtual environment. You only need a Jupyter server to start.
For details, please refer to Installation Guide.
