Pix2Gif: Motion-Guided Diffusion for GIF Generation

🍇 [arXiv]   🍎 [Demo]   🍊 [Project Page]

Hitesh Kandala¹, Jianfeng Gao², Jianwei Yang²
Microsoft Research India¹, Microsoft Research Redmond²

Setup

Set up python virtual environment

python3.10 -m venv .pix2gif
source .pix2gif/bin/activate
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt
bash scripts/download_pretrained_sd.sh

Demo

This demo takes in an image and a caption and generates a Gif following the input caption. To launch the demo, run:

python demo.py

Acknowledgement

We build our work on top of InstructPix2Pix

Citation

@misc{kandala2024pix2gif,
      title={Pix2Gif: Motion-Guided Diffusion for GIF Generation}, 
      author={Hitesh Kandala and Jianfeng Gao and Jianwei Yang},
      year={2024},
      eprint={2403.04634},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}