Disable check for dropout in MultiheadAttention fast_path #88831
Conversation
Since we already enforce eval mode for the fast_path, we do not need to also check for a falsy dropout value, as a model trained with dropout will have a non-zero dropout during eval mode, even though it won't be applied. Fixes pytorch#88806
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/88831
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit a7396dc: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
|
|
Will sign CLA ASAP. |
|
I didn't think this path was unit test worthy, but if you want I can port over the test from the related issue. |
|
@drisspg please reassign if someone else from BT should review this |
ngimel
added
the
triaged
|
@erichan1 looking at the implementation in pytorch/torch/nn/functional.py Lines 5167 to 5168 in 1ae772a |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
) Since we already enforce eval mode for the fast_path, we do not need to also check for a falsy dropout value, as a model trained with dropout will have a non-zero dropout during eval mode, even though it won't be applied. Fixes pytorch#88806 Pull Request resolved: pytorch#88831 Approved by: https://github.com/drisspg
Since we already enforce eval mode for the fast_path, we do not need to also check for a falsy dropout value, as a model trained with dropout will have a non-zero dropout during eval mode, even though it won't be applied.
Fixes #88806