Assess the performance / memory trade-off

[Maxusmusti]

Assess how much memory is saved and performance is lost when switching from BACKWARD_PRE to BACKWARD_POST or None for torch.distributed.fsdp.BackwardPrefetch.

[github-actions]

This issue has been automatically marked as stale because it has not had activity within 90 days. It will be automatically closed if no further activity occurs within 30 days.

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant!