Skip to content

Conversation

r-barnes
Copy link
Contributor

@r-barnes r-barnes commented Jun 8, 2021

Differential Revision: D28974453

r-barnes added 2 commits June 8, 2021 14:12
Differential Revision: D28916206

fbshipit-source-id: 8a1eae9fa9bb9947e696cb23e6a57b1c3f46a113
Differential Revision: D28974453

fbshipit-source-id: eff38bde29c0d2246223e1fde999907b2d992db1
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jun 8, 2021

💊 CI failures summary and remediations

As of commit 0068de2 (more details on the Dr. CI page):


  • 9/10 failures possibly* introduced in this PR
    • 1/9 non-scanned failure(s)
  • 1/10 broken upstream at merge base ea3b2fd on Jun 08 from 1:00pm to 2:45pm

🕵️ 7 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build Windows CI (pytorch-win-vs2019-cpu-py3) / test (1/7)

Step: "Run test scripts" (full log | diagnosis details | 🔁 rerun)

2021-06-08T23:14:26.4715888Z AssertionError: No...orward AD, you should set supports_forward_ad=True
2021-06-08T23:14:26.4706188Z ======================================================================
2021-06-08T23:14:26.4706889Z FAIL [0.005s]: test_forward_mode_AD_outer_cpu_float64 (__main__.TestGradientsCPU)
2021-06-08T23:14:26.4707708Z ----------------------------------------------------------------------
2021-06-08T23:14:26.4708345Z Traceback (most recent call last):
2021-06-08T23:14:26.4709774Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 292, in instantiated_test
2021-06-08T23:14:26.4710797Z     result = test_fn(self, *args)
2021-06-08T23:14:26.4711949Z   File "C:\actions-runner\_work\pytorch\pytorch\build\win_tmp\build\torch\testing\_internal\common_device_type.py", line 266, in test_wrapper
2021-06-08T23:14:26.4712927Z     return test(*args, **kwargs)
2021-06-08T23:14:26.4713640Z   File "test_ops.py", line 228, in test_forward_mode_AD
2021-06-08T23:14:26.4714443Z     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
2021-06-08T23:14:26.4715888Z AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
2021-06-08T23:14:26.4716916Z 
2021-06-08T23:14:28.4036034Z ----------------------------------------------------------------------
2021-06-08T23:14:28.4036559Z Ran 9487 tests in 869.479s
2021-06-08T23:14:28.4036769Z 
2021-06-08T23:14:28.4037098Z FAILED (failures=2, skipped=1807)
2021-06-08T23:14:28.4037340Z 
2021-06-08T23:14:28.4037630Z Generating XML reports...
2021-06-08T23:14:28.4038351Z Generated XML report: test-reports\dist-gloo\test_ops\TEST-TestCommonCPU-20210608225956.xml
2021-06-08T23:14:28.4039435Z Generated XML report: test-reports\dist-gloo\test_ops\TEST-TestGradientsCPU-20210608225956.xml
2021-06-08T23:14:28.4040504Z Generated XML report: test-reports\dist-gloo\test_ops\TEST-TestOpInfoCPU-20210608225956.xml

See CircleCI build pytorch_linux_bionic_py3_6_clang9_noarch_test (2/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jun 09 00:19:46 AssertionError: NotImplementedE...orward AD, you should set supports_forward_ad=True
Jun 09 00:19:46 ======================================================================
Jun 09 00:19:46 FAIL [0.005s]: test_forward_mode_AD_outer_cpu_float64 (__main__.TestGradientsCPU)
Jun 09 00:19:46 ----------------------------------------------------------------------
Jun 09 00:19:46 Traceback (most recent call last):
Jun 09 00:19:46   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Jun 09 00:19:46     result = test_fn(self, *args)
Jun 09 00:19:46   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Jun 09 00:19:46     return test(*args, **kwargs)
Jun 09 00:19:46   File "test_ops.py", line 228, in test_forward_mode_AD
Jun 09 00:19:46     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
Jun 09 00:19:46 AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
Jun 09 00:19:46 
Jun 09 00:19:50 ----------------------------------------------------------------------
Jun 09 00:19:50 Ran 18858 tests in 833.477s
Jun 09 00:19:50 
Jun 09 00:19:50 FAILED (failures=2, skipped=11174)
Jun 09 00:19:50 
Jun 09 00:19:50 Generating XML reports...
Jun 09 00:19:50 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestCommonCPU-20210609000552.xml
Jun 09 00:19:50 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestCommonMETA-20210609000552.xml
Jun 09 00:19:50 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestGradientsCPU-20210609000552.xml

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_test (3/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jun 08 23:04:02 AssertionError: NotImplementedE...orward AD, you should set supports_forward_ad=True
Jun 08 23:04:02 ======================================================================
Jun 08 23:04:02 FAIL [0.004s]: test_forward_mode_AD_outer_cpu_float64 (__main__.TestGradientsCPU)
Jun 08 23:04:02 ----------------------------------------------------------------------
Jun 08 23:04:02 Traceback (most recent call last):
Jun 08 23:04:02   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Jun 08 23:04:02     result = test_fn(self, *args)
Jun 08 23:04:02   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Jun 08 23:04:02     return test(*args, **kwargs)
Jun 08 23:04:02   File "test_ops.py", line 228, in test_forward_mode_AD
Jun 08 23:04:02     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
Jun 08 23:04:02 AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
Jun 08 23:04:02 
Jun 08 23:04:03 ----------------------------------------------------------------------
Jun 08 23:04:03 Ran 9487 tests in 781.868s
Jun 08 23:04:03 
Jun 08 23:04:03 FAILED (failures=2, skipped=1804)
Jun 08 23:04:03 
Jun 08 23:04:03 Generating XML reports...
Jun 08 23:04:03 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestCommonCPU-20210608225100.xml
Jun 08 23:04:03 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestGradientsCPU-20210608225100.xml
Jun 08 23:04:03 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestOpInfoCPU-20210608225100.xml

See CircleCI build pytorch_linux_xenial_py3_clang5_asan_test1 (4/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jun 09 01:41:40 AssertionError: NotImplementedE...orward AD, you should set supports_forward_ad=True
Jun 09 01:41:40 ======================================================================
Jun 09 01:41:40 FAIL [0.022s]: test_forward_mode_AD_outer_cpu_float64 (__main__.TestGradientsCPU)
Jun 09 01:41:40 ----------------------------------------------------------------------
Jun 09 01:41:40 Traceback (most recent call last):
Jun 09 01:41:40   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Jun 09 01:41:40     result = test_fn(self, *args)
Jun 09 01:41:40   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Jun 09 01:41:40     return test(*args, **kwargs)
Jun 09 01:41:40   File "test_ops.py", line 228, in test_forward_mode_AD
Jun 09 01:41:40     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
Jun 09 01:41:40 AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
Jun 09 01:41:40 
Jun 09 01:41:42 ----------------------------------------------------------------------
Jun 09 01:41:42 Ran 9487 tests in 3948.244s
Jun 09 01:41:42 
Jun 09 01:41:42 FAILED (failures=2, skipped=1804)
Jun 09 01:41:42 
Jun 09 01:41:42 Generating XML reports...
Jun 09 01:41:42 Generated XML report: test-reports/python-unittest/test_ops/TEST-TestCommonCPU-20210609003552.xml
Jun 09 01:41:42 Generated XML report: test-reports/python-unittest/test_ops/TEST-TestGradientsCPU-20210609003552.xml
Jun 09 01:41:42 Generated XML report: test-reports/python-unittest/test_ops/TEST-TestOpInfoCPU-20210609003552.xml

See CircleCI build pytorch_linux_bionic_cuda10_2_cudnn7_py3_9_gcc7_test1 (5/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jun 09 02:31:03 AssertionError: NotImplementedE...orward AD, you should set supports_forward_ad=True
Jun 09 02:31:03   File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py", line 1034, in wrapper
Jun 09 02:31:03     method(*args, **kwargs)
Jun 09 02:31:03   File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_utils.py", line 1034, in wrapper
Jun 09 02:31:03     method(*args, **kwargs)
Jun 09 02:31:03   File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Jun 09 02:31:03     result = test_fn(self, *args)
Jun 09 02:31:03   File "/opt/conda/lib/python3.9/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Jun 09 02:31:03     return test(*args, **kwargs)
Jun 09 02:31:03   File "/var/lib/jenkins/workspace/test/test_ops.py", line 228, in test_forward_mode_AD
Jun 09 02:31:03     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
Jun 09 02:31:03 AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
Jun 09 02:31:03 
Jun 09 02:31:03 ----------------------------------------------------------------------
Jun 09 02:31:03 Ran 9565 tests in 2005.400s
Jun 09 02:31:03 
Jun 09 02:31:03 FAILED (failures=2, errors=1, skipped=1820)
Jun 09 02:31:03 
Jun 09 02:31:03 Generating XML reports...
Jun 09 02:31:03 Generated XML report: test-reports/dist-gloo/.var.lib.jenkins.workspace.test.test_ops/TEST-TestCommonCUDA-20210609015738.xml
Jun 09 02:31:04 Generated XML report: test-reports/dist-gloo/.var.lib.jenkins.workspace.test.test_ops/TEST-TestGradientsCUDA-20210609015738.xml
Jun 09 02:31:04 Generated XML report: test-reports/dist-gloo/.var.lib.jenkins.workspace.test.test_ops/TEST-TestOpInfoCUDA-20210609015738.xml

See CircleCI build pytorch_macos_10_13_py3_test (6/7)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Jun 09 00:35:09 AssertionError: NotImplementedE...orward AD, you should set supports_forward_ad=True
Jun 09 00:35:09 ======================================================================
Jun 09 00:35:09 FAIL [0.006s]: test_forward_mode_AD_outer_cpu_float64 (__main__.TestGradientsCPU)
Jun 09 00:35:09 ----------------------------------------------------------------------
Jun 09 00:35:09 Traceback (most recent call last):
Jun 09 00:35:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
Jun 09 00:35:09     result = test_fn(self, *args)
Jun 09 00:35:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 266, in test_wrapper
Jun 09 00:35:09     return test(*args, **kwargs)
Jun 09 00:35:09   File "test_ops.py", line 228, in test_forward_mode_AD
Jun 09 00:35:09     self._grad_test_helper(device, dtype, op, op.get_op(), check_forward_ad=True)
Jun 09 00:35:09 AssertionError: NotImplementedError not raised : Running forward AD for an OP that has does not support it did not raise any error. If your op supports forward AD, you should set supports_forward_ad=True
Jun 09 00:35:09 
Jun 09 00:35:09 ----------------------------------------------------------------------
Jun 09 00:35:09 Ran 9487 tests in 1148.442s
Jun 09 00:35:09 
Jun 09 00:35:09 FAILED (failures=2, skipped=1804)
Jun 09 00:35:09 
Jun 09 00:35:09 Generating XML reports...
Jun 09 00:35:10 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestCommonCPU-20210609001601.xml
Jun 09 00:35:11 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestGradientsCPU-20210609001601.xml
Jun 09 00:35:12 Generated XML report: test-reports/dist-gloo/test_ops/TEST-TestOpInfoCPU-20210609001601.xml

See CircleCI build pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_build (7/7)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 08 21:29:07 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 08 21:29:07 ++++ extract_trap_cmd
Jun 08 21:29:07 ++++ printf '%s\n' ''
Jun 08 21:29:07 +++ printf '%s\n' cleanup
Jun 08 21:29:07 ++ trap -- '
Jun 08 21:29:07 cleanup' EXIT
Jun 08 21:29:07 ++ [[ pytorch-linux-xenial-cuda11.1-cudnn8-py3-gcc7-build != *pytorch-win-* ]]
Jun 08 21:29:07 ++ which sccache
Jun 08 21:29:07 ++ sccache --stop-server
Jun 08 21:29:07 ++ true
Jun 08 21:29:07 ++ rm /var/lib/jenkins/sccache_error.log
Jun 08 21:29:07 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 08 21:29:07 ++ true
Jun 08 21:29:07 ++ [[ -n '' ]]
Jun 08 21:29:07 ++ [[ pytorch-linux-xenial-cuda11.1-cudnn8-py3-gcc7-build == *rocm* ]]
Jun 08 21:29:07 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 08 21:29:07 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 08 21:29:07 ++ RUST_LOG=sccache::server=error
Jun 08 21:29:07 ++ sccache --start-server
Jun 08 21:29:07 sccache: Starting the server...
Jun 08 21:29:07 ++ sccache --zero-stats
Jun 08 21:29:07 Compile requests                      0

1 failure not recognized by patterns:

Job Step Action
GitHub Actions Lint / mypy Run mypy 🔁 rerun

2 jobs timed out:

  • pytorch_macos_10_13_py3_test
  • pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_build

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

ci.pytorch.org: 1 failed


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D28974453

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 3dfb94c.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants