Skip to content

Commit

Permalink
[chore] Update the CHANGELOG to add details about the new feature tha…
Browse files Browse the repository at this point in the history
…t enables reduce_scatter overlap in backward propagation (#906)

* updated the change log

* improve the change log
  • Loading branch information
tmarkstrum committed Jan 12, 2022
1 parent 0a526bc commit 0044372
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [0.4.5] - TBD

### Added
- FSDP: Added process_group_reduce_scatter parameter to allow users to pass in the process group that is used for reduce scatter operation. [#897]
- FSDP: Added state_dict_on_rank_0_only flag allow user choose to return full state dict on rank 0 and return empty dict non-rank 0 to prevent OOM [#844]

### Changed
- FSDP: Enabled reduce_scatter operation to overlap with all_gather stream and computation stream in backward propagation.
This change increases FSDP's throughput. To roll back this change, please pass ProcessGroupName.default to the process_group_reduce_scatter API parameter in FSDP [#897]

## [0.4.4] - 2021-12-21

Expand Down

0 comments on commit 0044372

Please sign in to comment.