Extra key in ucf101.pt

[wang-muhan]

When I'm loading the pretrained ucf101 checkpoint, the error appears:
RuntimeError: Error(s) in loading state_dict for Latte:
Unexpected key(s) in state_dict: "y_embedder.embedding_table.weight".
All the other checkpoints can generate videos successfully, can you check this? Thank you

[maxin-cn]

When I'm loading the pretrained ucf101 checkpoint, the error appears: RuntimeError: Error(s) in loading state_dict for Latte: Unexpected key(s) in state_dict: "y_embedder.embedding_table.weight". All the other checkpoints can generate videos successfully, can you check this? Thank you

Hi, can I confirm whether the extras parameter is 2 when you use the ucf101 pre-train model? See:

Latte/configs/ucf101/ucf101_sample.yaml

Line 14 in 9ededbe

extras: 2 # [1, 2] 1 unconditional generation, 2 class-conditional generation

[wang-muhan]

Do you have pretrained models for unconditional generation?

[maxin-cn]

Do you have pretrained models for unconditional generation?

All pre-trained models are unconditional generation except ucf101 and t2v.

[wang-muhan]

Thanks. How can I control the class being sampled, is it "sample_names" in the ucf101_sample.yaml? But I didn't found where you reference this value in your code

[maxin-cn]

Thanks. How can I control the class being sampled, is it "sample_names" in the ucf101_sample.yaml? But I didn't found where you reference this value in your code

You can change the class label in here,

Latte/sample/sample.py

Line 90 in 9ededbe

y = torch.randint(0, args.num_classes, (1,), device=device)

[wang-muhan]

What's you training loss after converge? I want to reproduce but met some problem.

[maxin-cn]

What's you training loss after converge? I want to reproduce but met some problem.

Training loss decreases quickly and oscillates around a value.

[github-actions]

Hi There! 👋

This issue has been marked as stale due to inactivity for 60 days.

We would like to inquire if you still have the same problem or if it has been resolved.

If you need further assistance, please feel free to respond to this comment within the next 7 days. Otherwise, the issue will be automatically closed.

We appreciate your understanding and would like to express our gratitude for your contribution to Latte. Thank you for your support. 🙏