st publish mode only load weight #5116
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for pytorch-fbgemm-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 13, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 13, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
force-pushed
the
export-D85830053
branch
from
01876f2 to
0981d5b
Compare
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 13, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
force-pushed
the
export-D85830053
branch
from
0981d5b to
3225d96
Compare
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 13, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 13, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 13, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 14, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
force-pushed
the
export-D85830053
branch
from
3225d96 to
09360c0
Compare
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 14, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 14, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 14, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
EddyLXJ
force-pushed
the
export-D85830053
branch
from
09360c0 to
e2c6c16
Compare
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 14, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
EddyLXJ
force-pushed
the
export-D85830053
branch
from
e2c6c16 to
c8904bc
Compare
EddyLXJ
added a commit
to EddyLXJ/torchrec
that referenced
this pull request
Nov 14, 2025
Summary: X-link: pytorch/FBGEMM#5116 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
EddyLXJ
added a commit
to EddyLXJ/FBGEMM-1
that referenced
this pull request
Nov 14, 2025
Summary: X-link: meta-pytorch/torchrec#3538 X-link: facebookresearch/FBGEMM#2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053
meta-codesync bot
pushed a commit
to meta-pytorch/torchrec
that referenced
this pull request
Nov 15, 2025
Summary: X-link: pytorch/FBGEMM#5116 Pull Request resolved: #3538 X-link: https://github.com/facebookresearch/FBGEMM/pull/2122 For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host. So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight. For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb Reviewed By: emlin Differential Revision: D85830053 fbshipit-source-id: 0eddbe9e69ea8271e8c77dc0147e87a08f0b3934
Contributor
|
This pull request has been merged in f3d282b. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
X-link: meta-pytorch/torchrec#3538
X-link: https://github.com/facebookresearch/FBGEMM/pull/2122
For silvertorch publish, we don't want to load opt into backend due to limited cpu memory in publish host.
So we need to load the whole row into state dict which loading the checkpoint in st publish, then only save weight into backend, after that backend will only have metaheader + weight.
For the first loading, we need to set dim with metaheader_dim + emb_dim + optimizer_state_dim, otherwise the checkpoint loadding will throw size mismatch error. after the first loading, we only need to get metaheader+weight from backend for state dict, so we can set dim with metaheader_dim + emb
Differential Revision: D85830053