inference memory with torch.set_grad_enabled(True)

[pwwwyyy]

Thank you for your great work! For some reasons ,I need to infer with torch.set_grad_enabled(True),but 80G cuda memory is not enough. I want to know whether this is normal and the configuration you use when training transformer of Latte-1.

[maxin-cn]

You can enable gradient checkpointing to save the GPU memory.

[github-actions]

Hi There! 👋

This issue has been marked as stale due to inactivity for 14 days.

We would like to inquire if you still have the same problem or if it has been resolved.

If you need further assistance, please feel free to respond to this comment within the next 7 days. Otherwise, the issue will be automatically closed.

We appreciate your understanding and would like to express our gratitude for your contribution to Latte. Thank you for your support. 🙏