Shuffling scripts for preproc and shuffle during training [dlrm/criteo] #54
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ng [1/n] Summary: **Preproc for dlrm inspired by NVIDIA DLRM Preproc: ** https://catalog.ngc.nvidia.com/orgs/nvidia/resources/dlrm_for_pytorch/advanced (under dataset guidelines) - Re-map sparse ids to contiguous integers (`with this you can have an embedding table of size num_categories x emb_dim`) - Frequency thresholding; if an id shows up less than T times, remap it to a value of 1 (`Fit model on particular GPU`, `Capture all rarely occurring categories into one because otherwise for these categories you would overfit`) full details of benefits of this preprocessing: - NVIDIA/DeepLearningExamples#1062 (comment) Differential Revision: D33998505 fbshipit-source-id: a7a2fb6bcbfbb4ffa347cd3663f3f1a87a56b9aa
Summary: Shuffles the dataset by creating the full dataset from the split .npy files. Outputs the shuffled dataset in split format (labels, sparse, dense) .npy files. Differential Revision: D34007646 fbshipit-source-id: e6c265bf5572619e7b6c9647dc095919c70f76d1
Summary: Pull Request resolved: meta-pytorch/torchrec#10 The original fb dlrm implementation shuffled batches to get their final results. Reviewed By: colin2328 Differential Revision: D34008000 fbshipit-source-id: b008c79841d75590f709150156455d1e9a68805a
facebook-github-bot
added
CLA Signed
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D34008000 |
s4ayub
changed the title
s4ayub
changed the title
Contributor
Author
|
linter warning is unrelated to code changes |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Pull Request resolved: #10
The original fb dlrm implementation shuffled batches to get their final results.
Reviewed By: colin2328
Differential Revision: D34008000