Support tensor betas in Adam and AdamW #134171
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/134171
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit b24cfa9 with merge base ba8568f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 [ghstack-poisoned]
|
@janeyx99 this is ready for review when you get the chance |
mlazos
added
ciflow/trunk
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 [ghstack-poisoned]
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 [ghstack-poisoned]
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 [ghstack-poisoned]
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 [ghstack-poisoned]
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@janeyx99 i actually also need to downcast to bfloat16 for the same reason. I'll update the issue accordingly |
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes #133898 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command |
|
@pytorchbot merge -f "last job stuck" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW. Fixes pytorch#133898 Pull Request resolved: pytorch#134171 Approved by: https://github.com/janeyx99
Adds support for beta1 and beta2 to be wrapped in tensor for Adam and AdamW.
Fixes #133898
Stack from ghstack (oldest at bottom):
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov