Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Meta Tensor checkpoint load for OPT models #2990

Merged
merged 1 commit into from Mar 10, 2023

Conversation

lekurile
Copy link
Contributor

This PR fixes Meta Tensor checkpoint loading for OPT models where the SD keys start with model..

Command:

deepspeed --num_nodes 1 --num_gpus 1 inference-test.py --ds_inference --use_kernel --name <model_name> --use_meta_tensor --dtype <model_dtype>

Model table:

OPT Model SD Key Before Fix After Fix
facebook/opt-125m model.decoder.embed_tokens.weight FAIL PASS
facebook/opt-350m decoder.embed_tokens.weight PASS PASS
facebook/opt-1.3b model.decoder.embed_tokens.weight FAIL PASS
facebook/opt-2.7b model.decoder.embed_tokens.weight FAIL PASS
facebook/opt-6.7b decoder.layers.23.fc2.weight PASS PASS
facebook/opt-13b decoder.embed_tokens.weight PASS PASS
facebook/opt-30b decoder.layers.31.fc1.weight PASS PASS

@lekurile lekurile merged commit 3798e60 into master Mar 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants