Add support for PPVCtrl #10625
Description
Model/Pipeline/Scheduler description
I recently came across an impressive work called PPVCtrl, a controllable video generation model. It leverages an auxiliary condition encoder to transform a text-to-video generation model into a customizable video generator, all without retraining the original generator. It's akin to ControlNet, but for video generation.
| Prompt | Reference Image | Control Videos | PP-VCtrl-5B-T2V | PP-VCtrl-5B-I2V |
|---|---|---|---|---|
| Group of fishes swimming in aquarium. |  |
 |
 |
 |
| A boat with a flag on it is sailing on the sea. |  |
 |
 |
 |
| Prompt | Reference Image | Control Videos | PP-VCtrl-5B-T2V | PP-VCtrl-5B-I2V |
|---|---|---|---|---|
| A rider in a dark helmet and white breeches is atop a chestnut horse... |  |
 |
 |
 |
| A dark gray Mini Cooper is parked on a city street... |  |
 |
 |
 |
| Prompt | Reference Image | Pose Videos | PP-VCtrl-5B-I2V |
|---|---|---|---|
| A young man with curly hair and a red t-shirt featuring a white logo is seen in various states of motion... |
|
|
|
| A woman models an Adrianna Papell women's gown, featuring a sleeveless... |
|
|
|
Open source status
- The model implementation is available.
- The model weights are available (Only relevant if addition is not a scheduler).
Provide useful links for the implementation
https://github.com/PaddlePaddle/PaddleMIX/tree/develop/ppdiffusers/examples/ppvctrl