Skip to content

fsdp fix#20719

Merged
pacman100 merged 1 commit intomainfrom
smangrul/fsdp-fix
Dec 12, 2022
Merged

fsdp fix#20719
pacman100 merged 1 commit intomainfrom
smangrul/fsdp-fix

Conversation

@pacman100
Copy link
Copy Markdown
Contributor

@pacman100 pacman100 commented Dec 10, 2022

What does this PR do?

  1. FSDP fix: Fixes 'UserWarning: Module is put on CPU' when use FSDP by Accelerate #18767. If this argument is not specified and module is on CPU, FSDP issues a warning mentioning that this argument can be specified for faster initialization.

@pacman100 pacman100 requested a review from sgugger December 10, 2022 06:34
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

HuggingFaceDocBuilderDev commented Dec 10, 2022

The documentation is not available anymore as the PR was closed or merged.

Copy link
Copy Markdown
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! Can you expand a bit the description so that users going back to your PR understand what's happening?

@pacman100
Copy link
Copy Markdown
Contributor Author

Thanks for the fix! Can you expand a bit the description so that users going back to your PR understand what's happening?

Done, Thanks!

@pacman100 pacman100 merged commit a413c72 into main Dec 12, 2022
@pacman100 pacman100 deleted the smangrul/fsdp-fix branch December 12, 2022 15:07
mpierrau pushed a commit to mpierrau/transformers that referenced this pull request Dec 15, 2022
@pacman100 pacman100 restored the smangrul/fsdp-fix branch December 22, 2022 08:40
@pacman100 pacman100 deleted the smangrul/fsdp-fix branch December 22, 2022 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

'UserWarning: Module is put on CPU' when use FSDP by Accelerate

3 participants