Why Latte Model can‘t support PipeFusion

[philipwan]

Open-Sora-v1.1’s Transformer module come from Latte, I see PipeFusion doesn't support Latte in README.
How does xDit support Open-Sora-v1.1/1.2 module, any suggestions ?

Thank you~

[feifeibear]

We just have no time to support pipefuson on Latte.
Currently, we are focusing on CogVideoX, which seems more popular right now.
I found ColossalAI, the framework developed by the company behind Open-Sora, has already used pipefusion.

https://github.com/hpcaitech/ColossalAI/blob/7cf9df07bcb267c0839d8880b109c4c7d55e80fa/colossalai/inference/modeling/layers/distrifusion.py#L3

Would like to help us on supporting PipeFusion for Latte-based models?

[philipwan]

We just have no time to support pipefuson on Latte. Currently, we are focusing on CogVideoX, which seems more popular right now. I found ColossalAI, the framework developed by the company behind Open-Sora, has already used pipefusion.

https://github.com/hpcaitech/ColossalAI/blob/7cf9df07bcb267c0839d8880b109c4c7d55e80fa/colossalai/inference/modeling/layers/distrifusion.py#L3

Would like to help us on supporting PipeFusion for Latte-based models?

Sorry for my mistake, I want to support for open-sora-plan v1.2 which has same struction with CogVideoX. Open-Sora may be slightly worse video quality， but ColossalAI's PipeFusion is a good example. I will try to keep following up.

[feifeibear]

We just have no time to support pipefuson on Latte. Currently, we are focusing on CogVideoX, which seems more popular right now. I found ColossalAI, the framework developed by the company behind Open-Sora, has already used pipefusion.
https://github.com/hpcaitech/ColossalAI/blob/7cf9df07bcb267c0839d8880b109c4c7d55e80fa/colossalai/inference/modeling/layers/distrifusion.py#L3
Would like to help us on supporting PipeFusion for Latte-based models?

Sorry for my mistake, I want to support for open-sora-plan v1.2 which has same struction with CogVideoX. Open-Sora may be slightly worse video quality， but ColossalAI's PipeFusion is a good example. I will try to keep following up.

We can work together on open-sora-plan v1.2.