Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise error when saving with param on meta device #2132

Merged
merged 6 commits into from
Nov 8, 2023

Conversation

SunMarc
Copy link
Member

@SunMarc SunMarc commented Nov 7, 2023

what does this PR do ?

This PR fixes the save_model method when a user tries to use it on a model with some parameters on the meta device. I didn't check the hf_device_map since it is not the only case where a user might move the model to the meta device. However, the warning that we have put in the dispatch function should be enough to let the user know that some parameters were offloaded. Fixes #2128

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Nov 7, 2023

The documentation is not available anymore as the PR was closed or merged.

Copy link
Collaborator

@muellerzr muellerzr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG2M! Small suggestion for efficiency :) Thanks for the blocker

src/accelerate/accelerator.py Outdated Show resolved Hide resolved
Co-authored-by: Zach Mueller <muellerzr@gmail.com>
Copy link
Contributor

@pacman100 pacman100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @SunMarc for adding this!

Copy link
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this update, it should hopefully help many users who are currently confused. I only have a nit, up to you if you want to fix it.

src/accelerate/accelerator.py Show resolved Hide resolved
@SunMarc SunMarc merged commit 0b0d921 into huggingface:main Nov 8, 2023
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

accelerator.save_model(): NotImplementedError: Cannot copy out of meta tensor; no data!
5 participants