Run MPS tests #78723

malfet · 2022-06-02T13:31:15Z

This adds a workflow, that is executed on MacOS 12.3+ machines and runs just test_mps.py

facebook-github-bot · 2022-06-02T13:31:22Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/78723
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours
↩️ [fb-only] Re-run with SSH instructions

❌ 3 New Failures

As of commit e70f542 (more details on the Dr. CI page):

Expand to see more

3/3 failures introduced in this PR

🕵️ 3 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages

trunk / linux-bionic-py3.7-clang9-slow / test (slow, 1, 1, linux.2xlarge) (1/3)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-06-03T15:11:54.1880855Z RuntimeError: test_jit_fuser_te failed!

2022-06-03T15:11:53.6384887Z Generated XML report: test-reports/python-unittest/test_jit_fuser_te/TEST-TestNNCOpInfoCPU-20220603144302.xml
2022-06-03T15:11:53.6477879Z Generated XML report: test-reports/python-unittest/test_jit_fuser_te/TEST-TestTEFuserDynamic-20220603144302.xml
2022-06-03T15:11:53.6576609Z Generated XML report: test-reports/python-unittest/test_jit_fuser_te/TEST-TestTEFuserStatic-20220603144302.xml
2022-06-03T15:11:53.6578865Z Generated XML report: test-reports/python-unittest/test_jit_fuser_te/TEST-jit.test_fuser_common.TestFuserCommon-20220603144302.xml
2022-06-03T15:11:53.6581317Z Generated XML report: test-reports/python-unittest/test_jit_fuser_te/TEST-TestLoopnestRandomizationCPU-20220603144302.xml
2022-06-03T15:11:54.1875237Z Traceback (most recent call last):
2022-06-03T15:11:54.1875726Z   File "test/run_test.py", line 1077, in <module>
2022-06-03T15:11:54.1878263Z     main()
2022-06-03T15:11:54.1878458Z   File "test/run_test.py", line 1055, in main
2022-06-03T15:11:54.1880542Z     raise RuntimeError(err_message)
2022-06-03T15:11:54.1880855Z RuntimeError: test_jit_fuser_te failed!
2022-06-03T15:11:54.4704626Z 
2022-06-03T15:11:54.4705128Z real	36m7.130s
2022-06-03T15:11:54.4705337Z user	45m59.067s
2022-06-03T15:11:54.4705509Z sys	1m46.683s
2022-06-03T15:11:54.4705684Z + cleanup
2022-06-03T15:11:54.4705828Z + retcode=1
2022-06-03T15:11:54.4705985Z + set +x
2022-06-03T15:11:54.4741327Z ##[error]Process completed with exit code 1.
2022-06-03T15:11:54.4781445Z ##[group]Run pytorch/pytorch/.github/actions/get-workflow-job-id@master
2022-06-03T15:11:54.4781695Z with:

trunk / win-vs2019-cuda11.3-py3 / test (default, 1, 5, windows.8xlarge.nvidia.gpu) (2/3)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-06-03T16:52:31.7938924Z FAIL [0.000s]: tes...es_logical_xor_cuda_bool (__main__.TestCommonCUDA)

2022-06-03T16:52:31.7936080Z     assert_equal(
2022-06-03T16:52:31.7936606Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_comparison.py", line 1095, in assert_equal
2022-06-03T16:52:31.7937053Z     raise error_metas[0].to_error(msg)
2022-06-03T16:52:31.7937358Z AssertionError: Tensor-likes are not equal!
2022-06-03T16:52:31.7937543Z 
2022-06-03T16:52:31.7937659Z Mismatched elements: 1 / 4 (25.0%)
2022-06-03T16:52:31.7937952Z Greatest absolute difference: 1 at index (1,)
2022-06-03T16:52:31.7938248Z Greatest relative difference: 1.0 at index (1,)
2022-06-03T16:52:31.7938434Z 
2022-06-03T16:52:31.7938561Z ======================================================================
2022-06-03T16:52:31.7938924Z FAIL [0.000s]: test_non_standard_bool_values_logical_xor_cuda_bool (__main__.TestCommonCUDA)
2022-06-03T16:52:31.7939319Z ----------------------------------------------------------------------
2022-06-03T16:52:31.7939651Z Traceback (most recent call last):
2022-06-03T16:52:31.7940175Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 377, in instantiated_test
2022-06-03T16:52:31.7940630Z     result = test(self, **param_kwargs)
2022-06-03T16:52:31.7941136Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 786, in test_wrapper
2022-06-03T16:52:31.7941552Z     return test(*args, **kwargs)
2022-06-03T16:52:31.7941903Z   File "test_ops.py", line 1135, in test_non_standard_bool_values
2022-06-03T16:52:31.7942231Z     self.assertEqual(expect, actual)
2022-06-03T16:52:31.7942729Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_utils.py", line 2250, in assertEqual
2022-06-03T16:52:31.7943120Z     assert_equal(

trunk / macos-11-py3-x86-64 / test (default, 1, 2, macos-12) (3/3)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-06-03T15:16:03.8601850Z RuntimeError: test_ops failed! Received signal: SIGSEGV

2022-06-03T15:16:03.7976560Z   test_non_standard_bool_values_tril_cpu_bool (__main__.TestCommonCPU) ... ok (0.004s)
2022-06-03T15:16:03.8018580Z   test_non_standard_bool_values_triu_cpu_bool (__main__.TestCommonCPU) ... ok (0.004s)
2022-06-03T15:16:03.8103910Z   test_non_standard_bool_values_true_divide_cpu_bool (__main__.TestCommonCPU) ... ok (0.008s)
2022-06-03T15:16:03.8200940Z   test_non_standard_bool_values_unfold_cpu_bool (__main__.TestCommonCPU) ... ok (0.010s)
2022-06-03T15:16:03.8391930Z   test_non_standard_bool_values_unique_consecutive_cpu_bool (__main__.TestCommonCPU) ... expected failure (0.019s)
2022-06-03T15:16:03.8595610Z   test_non_standard_bool_values_unique_cpu_bool (__main__.TestCommonCPU) ... Traceback (most recent call last):
2022-06-03T15:16:03.8597740Z   File "test/run_test.py", line 1077, in <module>
2022-06-03T15:16:03.8598400Z     main()
2022-06-03T15:16:03.8599200Z   File "test/run_test.py", line 1055, in main
2022-06-03T15:16:03.8599930Z     raise RuntimeError(err_message)
2022-06-03T15:16:03.8601850Z RuntimeError: test_ops failed! Received signal: SIGSEGV
2022-06-03T15:16:04.1139230Z + cleanup
2022-06-03T15:16:04.1139740Z + retcode=1
2022-06-03T15:16:04.1140140Z + set +x
2022-06-03T15:16:04.1157400Z ##[error]Process completed with exit code 1.
2022-06-03T15:16:04.1240220Z ##[group]Run pytorch/pytorch/.github/actions/get-workflow-job-id@master
2022-06-03T15:16:04.1240560Z with:
2022-06-03T15:16:04.1241620Z   github-token: ***
2022-06-03T15:16:04.1241870Z env:
2022-06-03T15:16:04.1242040Z   IN_CI: 1
2022-06-03T15:16:04.1242220Z   IS_GHA: 1

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

albanD

Awesome!

kulinseth · 2022-06-03T03:07:04Z

.github/workflows/trunk.yml

      MACOS_SCCACHE_S3_ACCESS_KEY_ID: ${{ secrets.MACOS_SCCACHE_S3_ACCESS_KEY_ID }}
      MACOS_SCCACHE_S3_SECRET_ACCESS_KEY: ${{ secrets.MACOS_SCCACHE_S3_SECRET_ACCESS_KEY }}

+  macos-10-15-py3-arm64-test:


Should we call this macos-11-0-py3-arm64 or macos-12-3-py3-arm64 ?

kulinseth

Looks good.

malfet · 2022-06-03T15:01:38Z

@pytorchbot merge this

pytorch-bot · 2022-06-03T15:01:40Z

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: this

usage: @pytorchbot {merge,revert,rebase,help} ...

Try @pytorchbot help for more info.

malfet · 2022-06-03T15:02:03Z

@pytorchbot merge

pytorchmergebot · 2022-06-03T15:03:17Z

Merge failed due to Refusing to merge as mandatory check(s) linux-docs / build-docs (cpp), win-vs2019-cuda11.3-py3 / build are pending/not yet run for rule superuser
Raised by https://github.com/pytorch/pytorch/actions/runs/2435405680

malfet · 2022-06-03T15:04:56Z

@pytorchbot merge -f

malfet · 2022-06-03T15:35:03Z

And here is the first successful run on trunk: https://hud.pytorch.org/hud/pytorch/pytorch/master/1?name_filter=macos-12.3

kulinseth · 2022-06-03T23:41:45Z

And here is the first successful run on trunk: https://hud.pytorch.org/hud/pytorch/pytorch/master/1?name_filter=macos-12.3

This is awesome , thanks

Summary: This adds a workflow, that is executed on MacOS 12.3+ machines and runs just test_mps.py Pull Request resolved: #78723 Approved by: https://github.com/albanD, https://github.com/kulinseth Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/f7ac389e71e55f84651141c01334dea668b3f90c Reviewed By: b0noI Differential Revision: D36906889 Pulled By: malfet fbshipit-source-id: afe457de2f62bfe476538987bf47ee18e724cafe

This adds a workflow, that is executed on MacOS 12.3+ machines and runs just test_mps.py Pull Request resolved: pytorch#78723 Approved by: https://github.com/albanD, https://github.com/kulinseth

This adds a workflow, that is executed on MacOS 12.3+ machines and runs just test_mps.py Pull Request resolved: #78723 Approved by: https://github.com/albanD, https://github.com/kulinseth (cherry picked from commit f7ac389)

facebook-github-bot added the cla signed label Jun 2, 2022

[WIP] Run MPS tests

242a785

malfet force-pushed the malfet/test-mps branch from 4c14bc4 to 242a785 Compare June 2, 2022 13:36

malfet added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 2, 2022

Connect MPS tests to builds

91b55cf

malfet force-pushed the malfet/test-mps branch from fef71e0 to 91b55cf Compare June 3, 2022 00:45

malfet requested review from albanD and kulinseth June 3, 2022 00:47

malfet changed the title ~~[WIP] Run MPS tests~~ Run MPS tests Jun 3, 2022

malfet marked this pull request as ready for review June 3, 2022 00:48

malfet requested a review from a team as a code owner June 3, 2022 00:48

albanD approved these changes Jun 3, 2022

View reviewed changes

Defaults does not seem to work

2355e6e

kulinseth reviewed Jun 3, 2022

View reviewed changes

kulinseth approved these changes Jun 3, 2022

View reviewed changes

malfet added 3 commits June 2, 2022 21:28

Explicitly checkout pytorch

29099a3

Add cleanup and rename steps

63910df

Do suggested rename

e70f542

malfet force-pushed the malfet/test-mps branch from 344ed08 to e70f542 Compare June 3, 2022 14:19

malfet added the topic: not user facing topic category label Jun 3, 2022

pytorchmergebot added the Merged label Jun 3, 2022

pytorchmergebot closed this in f7ac389 Jun 3, 2022

malfet deleted the malfet/test-mps branch June 3, 2022 15:23

atalman mentioned this pull request Jun 6, 2022

[v.1.12.0] Release Tracker #78005

Closed

Run MPS tests #78723

Run MPS tests #78723

Uh oh!

Conversation

malfet commented Jun 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jun 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

❌ 3 New Failures

🕵️ 3 new failures recognized by patterns

trunk / linux-bionic-py3.7-clang9-slow / test (slow, 1, 1, linux.2xlarge) (1/3)

trunk / win-vs2019-cuda11.3-py3 / test (default, 1, 5, windows.8xlarge.nvidia.gpu) (2/3)

trunk / macos-11-py3-x86-64 / test (default, 1, 2, macos-12) (3/3)

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

kulinseth Jun 3, 2022

Choose a reason for hiding this comment

Uh oh!

kulinseth left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Jun 3, 2022

Uh oh!

pytorch-bot bot commented Jun 3, 2022

Uh oh!

malfet commented Jun 3, 2022

Uh oh!

pytorchmergebot commented Jun 3, 2022

Uh oh!

malfet commented Jun 3, 2022

Uh oh!

malfet commented Jun 3, 2022

Uh oh!

kulinseth commented Jun 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

malfet commented Jun 2, 2022 •

edited

Loading

facebook-github-bot commented Jun 2, 2022 •

edited

Loading