Support for multi-view video generation

[wsynuiag]

Dear authors,

How can we input images or videos from multiple cameras and generate multi-view videos just like Cosmos2.5-Predict and Cosmos-Predict2?

[rOtking]

Waiting for replay

[mli0603]

Hi @wsynuiag @rOtking, for robotics datasets such DROID and Agibot, we can already support multi-view generation as shown in https://huggingface.co/nvidia/Cosmos3-Nano.
More cookbooks can be found here: https://github.com/NVIDIA/cosmos/tree/main/cookbooks/cosmos3/generator/action

For other general domains such AV, we are building towards it.

[wsynuiag]

@mli0603 Thanks for your valuable reply! I wonder if the cosmos3 supports multi-view generation for autonomous driving (e.g., input 3-6 cameras data like OpenDWM), or are there any plans for it?

[mli0603]

Yes we have plans for it! The current version will require post-training as it does not natively support AV use cases.

[TSxingke]

I share the same question and eagerly await your future work on this matter.