Fix test_addmm_gelu assertion on Windows CUDA

[aakhundov]

Stack from ghstack (oldest at bottom):

• -> Fix test_addmm_gelu assertion on Windows CUDA #104031

Summary:

This PR fixes the wrong assertion in the test_addmm_gelu happening in the Windows CUDA CI job caused by #103811. The addmm + GELU fusion is likely not happening (or not using the tanh approximation) on Windows. See this comment in the #103811 for the details of the error.

Test Plan:

$ python test/test_linalg.py -k test_addmm_relu -v
test_addmm_relu_cpu_bfloat16 (__main__.TestLinalgCPU.test_addmm_relu_cpu_bfloat16) ... ok
test_addmm_relu_cpu_float32 (__main__.TestLinalgCPU.test_addmm_relu_cpu_float32) ... ok
test_addmm_relu_cpu_float64 (__main__.TestLinalgCPU.test_addmm_relu_cpu_float64) ... ok
test_addmm_relu_cuda_bfloat16 (__main__.TestLinalgCUDA.test_addmm_relu_cuda_bfloat16) ... ok
test_addmm_relu_cuda_float32 (__main__.TestLinalgCUDA.test_addmm_relu_cuda_float32) ... ok
test_addmm_relu_cuda_float64 (__main__.TestLinalgCUDA.test_addmm_relu_cuda_float64) ... ok

----------------------------------------------------------------------
Ran 6 tests in 2.131s

OK

$ python test/test_linalg.py -k test_addmm_gelu -v
test_addmm_gelu_cpu_bfloat16 (__main__.TestLinalgCPU.test_addmm_gelu_cpu_bfloat16) ... ok
test_addmm_gelu_cpu_float32 (__main__.TestLinalgCPU.test_addmm_gelu_cpu_float32) ... ok
test_addmm_gelu_cpu_float64 (__main__.TestLinalgCPU.test_addmm_gelu_cpu_float64) ... ok
test_addmm_gelu_cuda_bfloat16 (__main__.TestLinalgCUDA.test_addmm_gelu_cuda_bfloat16) ... ok
test_addmm_gelu_cuda_float32 (__main__.TestLinalgCUDA.test_addmm_gelu_cuda_float32) ... ok
test_addmm_gelu_cuda_float64 (__main__.TestLinalgCUDA.test_addmm_gelu_cuda_float64) ... ok

----------------------------------------------------------------------
Ran 6 tests in 2.194s

OK

Reviewers: @eellison @huydhn

Subscribers:

Tasks:

Tags:

Differential Revision: D46931688

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/104031

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 5a3d309:

NEW FAILURES - The following jobs have failed:

• ios-12-5-1-x86-64-coreml / build (default, 1, 1, macos-12) (gh)
• win-vs2019-cuda11.8-py3 / test (default, 2, 4, windows.g5.4xlarge.nvidia.gpu) (gh)
• win-vs2019-cuda11.8-py3 / test (default, 3, 4, windows.g5.4xlarge.nvidia.gpu) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[aakhundov]

@aakhundov has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[huydhn]

@pytorchbot rebase

[huydhn]

Jsut FYI, the Windows build failure https://github.com/pytorch/pytorch/actions/runs/5343807634/jobs/9693267625 is a known issue that can be fixed by rebasing to include the fix commit in trunk.

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased gh/aakhundov/4/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/104031)

[malfet]

@pytorchbot merge -f "Lint is green and It simply skips the test"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator

Details for Dev Infra team

Raised by workflow job

[aakhundov]

We've discussed the issue with @eellison and agreed that it would be better to switch to the "tanh" approximation for non-fused GELU in the aten._addmm_activation op, for consistency. As the API is private, this sounds like a more consistent way than patching different cases in the unit test. Will update the PR soon accordingly.

[eellison]

@aakhundov if HUD is in a bad state, maybe it makes sense to land this if it fixes tests, then in a separate pr update ?

[aakhundov]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 1 jobs have failed, first few of them are: periodic / ios-12-5-1-x86-64-coreml / build (default, 1, 1, macos-12)

Details for Dev Infra team

Raised by workflow job

[aakhundov]

@huydhn @malfet @eellison the merge failed again due to the periodic / ios-12-5-1-x86-64-coreml / build (default, 1, 1, macos-12) job, which also seems unrelated. Any hint on how to bypass / fix this? Thanks!

[eellison]

You can merge past failures with merge -f {reason}

[aakhundov]

@pytorchbot merge -f "The failing CI job seems unrelated, merging to fix the HUD."

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here