[MPS] Fixes for LSTM. #94889
Closed
[MPS] Fixes for LSTM. #94889
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pytorch-bot
bot
added
ciflow/mps
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/94889
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 67edca8: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
|
- Backward pass has to give explicit bias tensor of zeros if none is passed to the op or the bias gradient will not be calculated. - Fixed bias tensor mistakenly getting overwritten to zeros - Fixes crash when lstm op called with has_biases set to false. Change takes into account the changed shape of the input params TensorList depending on the bias flag. - Raise error for projections in LSTM. Currently not supported.
kulinseth
force-pushed
the
lstm_fixes
branch
from
fa2216a
to
3e10b86
Compare
|
@pytorchbot merge -f "All tests are green." |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
drisspg
pushed a commit
to drisspg/pytorch
that referenced
this pull request
Feb 16, 2023
- Backward pass has to give explicit bias tensor of zeros if none is passed to the op or the bias gradient will not be calculated. - Fixed bias tensor mistakenly getting overwritten to zeros - Fixes crash when lstm op called with has_biases set to false. Change takes into account the changed shape of the input params TensorList depending on the bias flag. Fixes #ISSUE_NUMBER Pull Request resolved: pytorch#94889 Approved by: https://github.com/DenisVieriu97
This was referenced Feb 22, 2023
atalman
pushed a commit
that referenced
this pull request
Feb 23, 2023
* [MPS] Fixes for LSTM. (#94889) - Backward pass has to give explicit bias tensor of zeros if none is passed to the op or the bias gradient will not be calculated. - Fixed bias tensor mistakenly getting overwritten to zeros - Fixes crash when lstm op called with has_biases set to false. Change takes into account the changed shape of the input params TensorList depending on the bias flag. Fixes #ISSUE_NUMBER Pull Request resolved: #94889 Approved by: https://github.com/DenisVieriu97 * [MPS] LogSoftmax numerical stability (#95091) Fixes #94043 Calculations are now consistent with numericaly stable formula and CPU: $LogSoftmax(X, \dim) = X - \max(X, \dim) - \log(sum(X - \max(X, \dim), \dim))$ @malfet Pull Request resolved: #95091 Approved by: https://github.com/malfet, https://github.com/kulinseth * [MPS] Cast int64 to int32 for reduction ops (#95231) - give warnings of converting int64 for reduction ops - use cast tensor for reduction sum on trace - unblock trace from running Pull Request resolved: #95231 Approved by: https://github.com/razarmehr * [MPS] Fix Float16 issue with Reduction ops for macOS 12 (#94952) This would fix the issue with `__rdiv__` with float16 Pull Request resolved: #94952 Approved by: https://github.com/kulinseth --------- Co-authored-by: alexdremov <dremov.me@gmail.com> Co-authored-by: Denis Vieriu <dvieriu@apple.com> Co-authored-by: Ramin Azarmehr <razarmehr@apple.com>
pruthvistony
added a commit
to ROCm/pytorch
that referenced
this pull request
May 2, 2023
This reverts commit 54ebf25.
pruthvistony
pushed a commit
to ROCm/pytorch
that referenced
this pull request
May 3, 2023
* [MPS] Fixes for LSTM. (pytorch#94889) - Backward pass has to give explicit bias tensor of zeros if none is passed to the op or the bias gradient will not be calculated. - Fixed bias tensor mistakenly getting overwritten to zeros - Fixes crash when lstm op called with has_biases set to false. Change takes into account the changed shape of the input params TensorList depending on the bias flag. Fixes #ISSUE_NUMBER Pull Request resolved: pytorch#94889 Approved by: https://github.com/DenisVieriu97 * [MPS] LogSoftmax numerical stability (pytorch#95091) Fixes pytorch#94043 Calculations are now consistent with numericaly stable formula and CPU: $LogSoftmax(X, \dim) = X - \max(X, \dim) - \log(sum(X - \max(X, \dim), \dim))$ @malfet Pull Request resolved: pytorch#95091 Approved by: https://github.com/malfet, https://github.com/kulinseth * [MPS] Cast int64 to int32 for reduction ops (pytorch#95231) - give warnings of converting int64 for reduction ops - use cast tensor for reduction sum on trace - unblock trace from running Pull Request resolved: pytorch#95231 Approved by: https://github.com/razarmehr * [MPS] Fix Float16 issue with Reduction ops for macOS 12 (pytorch#94952) This would fix the issue with `__rdiv__` with float16 Pull Request resolved: pytorch#94952 Approved by: https://github.com/kulinseth --------- Co-authored-by: alexdremov <dremov.me@gmail.com> Co-authored-by: Denis Vieriu <dvieriu@apple.com> Co-authored-by: Ramin Azarmehr <razarmehr@apple.com>
jhavukainen
pushed a commit
to kulinseth/pytorch
that referenced
this pull request
Mar 15, 2024
- Backward pass has to give explicit bias tensor of zeros if none is passed to the op or the bias gradient will not be calculated. - Fixed bias tensor mistakenly getting overwritten to zeros - Fixes crash when lstm op called with has_biases set to false. Change takes into account the changed shape of the input params TensorList depending on the bias flag. Fixes #ISSUE_NUMBER Pull Request resolved: pytorch#94889 Approved by: https://github.com/DenisVieriu97
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_NUMBER