[FSDP] Default to BACKWARD_PRE

[awgu]

Stack from ghstack:

• [FSDP] Default to limit_all_gathers=True #88432 [FSDP] Default to limit_all_gathers=True
• [FSDP][Docs] Reword sharding_strategy docs and other minor doc changes #88431 [FSDP][Docs] Reword sharding_strategy docs and other minor doc changes
• [FSDP][Docs] Simplify CPU offload docs #88430 [FSDP][Docs] Simplify CPU offload docs
• [FSDP][Docs] Simplify mixed_precision ctor docs #88429 [FSDP][Docs] Simplify mixed_precision ctor docs
• [FSDP] Default to BACKWARD_PRE #88428 [FSDP] Default to BACKWARD_PRE
• [FSDP()][Easy] Make fully_shard() only FULL_SHARD #88260 [FSDP()][Easy] Make fully_shard() only FULL_SHARD
• [FSDP()] Have fully_shard() abide by @contract! #88235 [FSDP()] Have fully_shard() abide by @contract!
• [FSDP()][Easy] Rename _State to _FSDPState #88234 [FSDP()][Easy] Rename _State to _FSDPState
• [FSDP()] Rename to fully_shard() and move to _composable/ #88233 [FSDP()] Rename to fully_shard() and move to _composable/
• [FSDP][Easy] Remove unneeded TrainingState transition #88232 [FSDP][Easy] Remove unneeded TrainingState transition
• [FSDP] Rename unflat_param_name -> fqn for consistency #88123 [FSDP] Rename unflat_param_name -> fqn for consistency
• [FSDP] Simplify _get_buffer_names() #88122 [FSDP] Simplify _get_buffer_names()
• [FSDP] Remove unneeded torch.no_grad() context when offloading to CPU #88121 [FSDP] Remove unneeded torch.no_grad() context when offloading to CPU
• [FSDP][Docs] Add note mentioning rate limiter for backward prefetch #88120 [FSDP][Docs] Add note mentioning rate limiter for backward prefetch
• [FSDP()][27/N] Add forward hook registration #88040 [FSDP()][27/N] Add forward hook registration
• [FSDP()][26/N] Move _lazy_init() into _fsdp_root_pre_forward() #87941 [FSDP()][26/N] Move _lazy_init() into _fsdp_root_pre_forward()
• [FSDP()][25/N] Add _post_forward_reshard() #87940 [FSDP()][25/N] Add _post_forward_reshard()
• [FSDP()][24/N] Refactor _lazy_init() #87939 [FSDP()][24/N] Refactor _lazy_init()
• [FSDP()][23/N] Refactor handle attr initialization #87938 [FSDP()][23/N] Refactor handle attr initialization
• [FSDP()][21/N] Refactor and fix _cast_buffers() #87935 [FSDP()][21/N] Refactor and fix _cast_buffers()
• [FSDP] Rename dtype to buffer_name_to_dtype #87934 [FSDP] Rename dtype to buffer_name_to_dtype
• [FSDP] Remove device arg from _cast_buffers() #87933 [FSDP] Remove device arg from _cast_buffers()
• [FSDP()][20/N][Easy] Move functions in file #87932 [FSDP()][20/N][Easy] Move functions in file
• [FSDP()][18/N] Refactor pre_forward_unshard() #87931 [FSDP()][18/N] Refactor pre_forward_unshard()
• [FSDP()][17/N] Refactor _fsdp_root_pre_forward() #87930 [FSDP()][17/N] Refactor _fsdp_root_pre_forward()
• [FSDP()][16/N] Refactor post-forward/pre-backward #87929 [FSDP()][16/N] Refactor post-forward/pre-backward
• [FSDP()][15/N] Refactor _init_streams() #87928 [FSDP()][15/N] Refactor _init_streams()
• [FSDP()][14/N] Refactor pre-forward/post-backward #87927 [FSDP()][14/N] Refactor pre-forward/post-backward

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/88428

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 9c9492f:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[awgu]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here