Add reduction string test for ctc_loss. #43888

gchanan · 2020-08-31T18:10:41Z

Stack from ghstack:

Add reduction string test for ctc_loss. #43888 Add reduction string test for ctc_loss.
Actually run backward criterion tests. #43809 Actually run backward criterion tests.
Merge criterion and new criterion tests. #43788 Merge criterion and new criterion tests.
Properly check that reduction strings are valid for l1_loss, smoothl1_loss, and mse_loss. #43527 Properly check that reduction strings are valid for l1_loss, smoothl1_loss, and mse_loss.

[ghstack-poisoned]

ghstack-source-id: 5c2a02e5cb21f314eeb828422ee74291adc2474f Pull Request resolved: #43888

dr-ci · 2020-08-31T18:26:10Z

💊 CI failures summary and remediations

As of commit 432a9da (more details on the Dr. CI page):

14/14 failures possibly* introduced in this PR
- 1/14 non-CircleCI failure(s)

🕵️ 11 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_cuda10_2_cudnn7_py3_ge_config_profiling_test (1/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 19:05:20 ERROR [0.004s]: test_nn_CTCLoss_lengths_tensors (test_jit.TestJitGeneratedModule)

Aug 31 19:05:20 ERROR [0.004s]: test_nn_CTCLoss_lengths_intlists (test_jit.TestJitGeneratedModule) 
Aug 31 19:05:20 ---------------------------------------------------------------------- 
Aug 31 19:05:20 Traceback (most recent call last): 
Aug 31 19:05:20   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 525, in wrapper 
Aug 31 19:05:20     fn(*args, **kwargs) 
Aug 31 19:05:20   File "/var/lib/jenkins/workspace/test/test_jit.py", line 15652, in do_test 
Aug 31 19:05:20     input = input + kwargs['extra_args'] 
Aug 31 19:05:20 TypeError: unsupported operand type(s) for +: 'Tensor' and 'tuple' 
Aug 31 19:05:20  
Aug 31 19:05:20 ====================================================================== 
Aug 31 19:05:20 ERROR [0.004s]: test_nn_CTCLoss_lengths_tensors (test_jit.TestJitGeneratedModule) 
Aug 31 19:05:20 ---------------------------------------------------------------------- 
Aug 31 19:05:20 Traceback (most recent call last): 
Aug 31 19:05:20   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 525, in wrapper 
Aug 31 19:05:20     fn(*args, **kwargs) 
Aug 31 19:05:20   File "/var/lib/jenkins/workspace/test/test_jit.py", line 15652, in do_test 
Aug 31 19:05:20     input = input + kwargs['extra_args'] 
Aug 31 19:05:20 TypeError: unsupported operand type(s) for +: 'Tensor' and 'tuple' 
Aug 31 19:05:20  
Aug 31 19:05:20 ---------------------------------------------------------------------- 
Aug 31 19:05:20 Ran 2688 tests in 225.018s

pytorch_linux_xenial_cuda10_2_cudnn7_py3_ge_config_legacy_test (2/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 19:05:09 ERROR [0.003s]: test_nn_CTCLoss_lengths_tensors (test_jit.TestJitGeneratedModule)

Aug 31 19:05:09 ERROR [0.003s]: test_nn_CTCLoss_lengths_intlists (test_jit.TestJitGeneratedModule) 
Aug 31 19:05:09 ---------------------------------------------------------------------- 
Aug 31 19:05:09 Traceback (most recent call last): 
Aug 31 19:05:09   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 525, in wrapper 
Aug 31 19:05:09     fn(*args, **kwargs) 
Aug 31 19:05:09   File "/var/lib/jenkins/workspace/test/test_jit.py", line 15652, in do_test 
Aug 31 19:05:09     input = input + kwargs['extra_args'] 
Aug 31 19:05:09 TypeError: unsupported operand type(s) for +: 'Tensor' and 'tuple' 
Aug 31 19:05:09  
Aug 31 19:05:09 ====================================================================== 
Aug 31 19:05:09 ERROR [0.003s]: test_nn_CTCLoss_lengths_tensors (test_jit.TestJitGeneratedModule) 
Aug 31 19:05:09 ---------------------------------------------------------------------- 
Aug 31 19:05:09 Traceback (most recent call last): 
Aug 31 19:05:09   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 525, in wrapper 
Aug 31 19:05:09     fn(*args, **kwargs) 
Aug 31 19:05:09   File "/var/lib/jenkins/workspace/test/test_jit.py", line 15652, in do_test 
Aug 31 19:05:09     input = input + kwargs['extra_args'] 
Aug 31 19:05:09 TypeError: unsupported operand type(s) for +: 'Tensor' and 'tuple' 
Aug 31 19:05:09  
Aug 31 19:05:09 ---------------------------------------------------------------------- 
Aug 31 19:05:09 Ran 2688 tests in 286.211s

pytorch_xla_linux_bionic_py3_6_clang9_build (3/11)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (content): Merge conflict in test/test_nn.py 
Removing aten/src/ATen/preprocess_declarations.py 
Removing aten/src/ATen/nn_parse.py 
Removing aten/src/ATen/nn.yaml 
Removing aten/src/ATen/native_parse.py 
Removing aten/src/ATen/gen_backend_select_register.py 
Removing aten/src/ATen/gen.py 
Removing aten/src/ATen/function_wrapper.py 
Removing aten/src/ATen/cwrap_parser.py 
Removing aten/src/ATen/Declarations.cwrap 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_windows_vs2019_py36_cuda10.1_test2 (4/11)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_cpp_api_parity failed!

  warnings.warn(errors.NumbaDeprecationWarning(msg)) 
Traceback (most recent call last): 
  File "test_cpp_api_parity.py", line 34, in <module> 
    (common_nn.new_criterion_tests, common_nn.NewCriterionTest), 
AttributeError: module 'torch.testing._internal.common_nn' has no attribute 'new_criterion_tests' 
Traceback (most recent call last): 
  File "run_test.py", line 734, in <module> 
    main() 
  File "run_test.py", line 717, in main 
    raise RuntimeError(err_message) 
RuntimeError: test_cpp_api_parity failed! 
 
(base) circleci@PACKER-5F0EEC91 C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (5/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 19:29:09 ERROR [1236.268s]: test_MSELoss_prec (__main__.TestNN)

Aug 31 19:29:09     test_case.check_criterion_jacobian(module, input, target) 
Aug 31 19:29:09   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 4502, in check_criterion_jacobian 
Aug 31 19:29:09     self._forward_criterion(criterion, input, target) 
Aug 31 19:29:09   File "test_nn.py", line 334, in _forward_criterion 
Aug 31 19:29:09     output = criterion(input, target, *extra_args) 
Aug 31 19:29:09   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl 
Aug 31 19:29:09     result = self.forward(*input, **kwargs) 
Aug 31 19:29:09 TypeError: forward() missing 2 required positional arguments: 'input_lengths' and 'target_lengths' 
Aug 31 19:29:09  
Aug 31 19:29:09 ====================================================================== 
Aug 31 19:29:09 ERROR [1236.268s]: test_MSELoss_prec (__main__.TestNN) 
Aug 31 19:29:09 ---------------------------------------------------------------------- 
Aug 31 19:29:09 Traceback (most recent call last): 
Aug 31 19:29:09   File "test_nn.py", line 8910, in <lambda> 
Aug 31 19:29:09     add(test_name, lambda self, test=test: test(self)) 
Aug 31 19:29:09   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 4993, in __call__ 
Aug 31 19:29:09     self._do_extra_tests(test_case, module, input, target) 
Aug 31 19:29:09   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 5016, in _do_extra_tests 
Aug 31 19:29:09     gradgradcheck(apply_fn, inputs) 
Aug 31 19:29:09   File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 486, in gradgradcheck 
Aug 31 19:29:09     nondet_tol=nondet_tol, check_undefined_grad=check_undefined_grad)

pytorch_linux_bionic_py3_8_gcc9_coverage_test (6/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 19:53:13 ERROR [3412.901s]: test_MSELoss_prec (__main__.TestNN)

Aug 31 19:53:13     test_case.check_criterion_jacobian(module, input, target) 
Aug 31 19:53:13   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_nn.py", line 4502, in check_criterion_jacobian 
Aug 31 19:53:13     self._forward_criterion(criterion, input, target) 
Aug 31 19:53:13   File "test_nn.py", line 334, in _forward_criterion 
Aug 31 19:53:13     output = criterion(input, target, *extra_args) 
Aug 31 19:53:13   File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl 
Aug 31 19:53:13     result = self.forward(*input, **kwargs) 
Aug 31 19:53:13 TypeError: forward() missing 2 required positional arguments: 'input_lengths' and 'target_lengths' 
Aug 31 19:53:13  
Aug 31 19:53:13 ====================================================================== 
Aug 31 19:53:13 ERROR [3412.901s]: test_MSELoss_prec (__main__.TestNN) 
Aug 31 19:53:13 ---------------------------------------------------------------------- 
Aug 31 19:53:13 Traceback (most recent call last): 
Aug 31 19:53:13   File "test_nn.py", line 8910, in <lambda> 
Aug 31 19:53:13     add(test_name, lambda self, test=test: test(self)) 
Aug 31 19:53:13   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_nn.py", line 4993, in __call__ 
Aug 31 19:53:13     self._do_extra_tests(test_case, module, input, target) 
Aug 31 19:53:13   File "/opt/conda/lib/python3.8/site-packages/torch/testing/_internal/common_nn.py", line 5016, in _do_extra_tests 
Aug 31 19:53:13     gradgradcheck(apply_fn, inputs) 
Aug 31 19:53:13   File "/opt/conda/lib/python3.8/site-packages/torch/autograd/gradcheck.py", line 485, in gradgradcheck 
Aug 31 19:53:13     return gradcheck(new_func, tupled_inputs + tupled_grad_outputs, eps, atol, rtol, raise_exception,

pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (7/11)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (content): Merge conflict in test/test_nn.py 
Removing aten/src/ATen/preprocess_declarations.py 
Removing aten/src/ATen/nn_parse.py 
Removing aten/src/ATen/nn.yaml 
Removing aten/src/ATen/native_parse.py 
Removing aten/src/ATen/gen_backend_select_register.py 
Removing aten/src/ATen/gen.py 
Removing aten/src/ATen/function_wrapper.py 
Removing aten/src/ATen/cwrap_parser.py 
Removing aten/src/ATen/Declarations.cwrap 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_linux_xenial_py3_clang5_asan_test2 (8/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 18:45:21 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/jenkins/workspace/aten/src/ATen/Utils.cpp:11:3 in

Aug 31 18:45:21     #7 0x55bdfdebf7eb in PyEval_EvalCode /tmp/build/80754af9/python_1588903631989/work/Python/ceval.c:731 
Aug 31 18:45:21     #8 0x55bdfdf3fe73 in run_mod /tmp/build/80754af9/python_1588903631989/work/Python/pythonrun.c:1025 
Aug 31 18:45:21     #9 0x55bdfdf3ff0c in PyRun_StringFlags /tmp/build/80754af9/python_1588903631989/work/Python/pythonrun.c:949 
Aug 31 18:45:21     #10 0x55bdfdf3ff6e in PyRun_SimpleStringFlags /tmp/build/80754af9/python_1588903631989/work/Python/pythonrun.c:445 
Aug 31 18:45:21     #11 0x55bdfdf43d72 in run_command /tmp/build/80754af9/python_1588903631989/work/Modules/main.c:301 
Aug 31 18:45:21     #12 0x55bdfdf43d72 in Py_Main /tmp/build/80754af9/python_1588903631989/work/Modules/main.c:749 
Aug 31 18:45:21     #13 0x55bdfde0df2d in main /tmp/build/80754af9/python_1588903631989/work/Programs/python.c:69 
Aug 31 18:45:21     #14 0x7f53c664783f in __libc_start_main /build/glibc-e6zv40/glibc-2.23/csu/../csu/libc-start.c:291 
Aug 31 18:45:21     #15 0x55bdfdeed27e in _start /home/rdonnelly/mc/conda-bld/compilers_linux-64_1534865402226/work/.build/src/glibc-2.12.2/csu/../sysdeps/x86_64/elf/start.S:103 
Aug 31 18:45:21  
Aug 31 18:45:21 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/jenkins/workspace/aten/src/ATen/Utils.cpp:11:3 in  
Aug 31 18:45:21 + retcode=1 
Aug 31 18:45:21 + set -e 
Aug 31 18:45:21 + return 1 
Aug 31 18:45:21 + [[ pytorch-linux-xenial-py3-clang5-asan-test2 == *-NO_AVX-* ]] 
Aug 31 18:45:21 + [[ pytorch-linux-xenial-py3-clang5-asan-test2 == *-NO_AVX2-* ]] 
Aug 31 18:45:21 + '[' -n https://github.com/pytorch/pytorch/pull/43888 ']' 
Aug 31 18:45:21 ++ mktemp 
Aug 31 18:45:21 + DETERMINE_FROM=/tmp/tmp.nDUYDxDahf 
Aug 31 18:45:21 + file_diff_from_base /tmp/tmp.nDUYDxDahf 
Aug 31 18:45:21 + set +e

pytorch_macos_10_13_py3_test (9/11)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Aug 31 11:39:21 RuntimeError: test_cpp_api_parity failed!

Aug 31 11:39:19 Executing ['/Users/distiller/workspace/miniconda3/bin/python', 'test_cpp_api_parity.py', '--verbose', '--verbose', '--verbose', '--verbose'] ... [2020-08-31 11:39:19.459326] 
Aug 31 11:39:21 Traceback (most recent call last): 
Aug 31 11:39:21   File "test_cpp_api_parity.py", line 34, in <module> 
Aug 31 11:39:21     (common_nn.new_criterion_tests, common_nn.NewCriterionTest), 
Aug 31 11:39:21 AttributeError: module 'torch.testing._internal.common_nn' has no attribute 'new_criterion_tests' 
Aug 31 11:39:21 Traceback (most recent call last): 
Aug 31 11:39:21   File "test/run_test.py", line 734, in <module> 
Aug 31 11:39:21     main() 
Aug 31 11:39:21   File "test/run_test.py", line 717, in main 
Aug 31 11:39:21     raise RuntimeError(err_message) 
Aug 31 11:39:21 RuntimeError: test_cpp_api_parity failed! 
Aug 31 11:39:21 + cleanup 
Aug 31 11:39:21 + retcode=1 
Aug 31 11:39:21 + set +x

pytorch_linux_xenial_py3_6_gcc5_4_build (10/11)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result.

CONFLICT (content): Merge conflict in test/test_nn.py 
Removing aten/src/ATen/preprocess_declarations.py 
Removing aten/src/ATen/nn_parse.py 
Removing aten/src/ATen/nn.yaml 
Removing aten/src/ATen/native_parse.py 
Removing aten/src/ATen/gen_backend_select_register.py 
Removing aten/src/ATen/gen.py 
Removing aten/src/ATen/function_wrapper.py 
Removing aten/src/ATen/cwrap_parser.py 
Removing aten/src/ATen/Declarations.cwrap 
Automatic merge failed; fix conflicts and then commit the result.

pytorch_linux_bionic_py3_6_clang9_test (11/11)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 31 20:02:32 ERROR [4222.450s]: test_MSELoss_prec (__main__.TestNN)

Aug 31 20:02:32     test_case.check_criterion_jacobian(module, input, target) 
Aug 31 20:02:32   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 4502, in check_criterion_jacobian 
Aug 31 20:02:32     self._forward_criterion(criterion, input, target) 
Aug 31 20:02:32   File "test_nn.py", line 334, in _forward_criterion 
Aug 31 20:02:32     output = criterion(input, target, *extra_args) 
Aug 31 20:02:32   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl 
Aug 31 20:02:32     result = self.forward(*input, **kwargs) 
Aug 31 20:02:32 TypeError: forward() missing 2 required positional arguments: 'input_lengths' and 'target_lengths' 
Aug 31 20:02:32  
Aug 31 20:02:32 ====================================================================== 
Aug 31 20:02:32 ERROR [4222.450s]: test_MSELoss_prec (__main__.TestNN) 
Aug 31 20:02:32 ---------------------------------------------------------------------- 
Aug 31 20:02:32 Traceback (most recent call last): 
Aug 31 20:02:32   File "test_nn.py", line 8910, in <lambda> 
Aug 31 20:02:32     add(test_name, lambda self, test=test: test(self)) 
Aug 31 20:02:32   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 4993, in __call__ 
Aug 31 20:02:32     self._do_extra_tests(test_case, module, input, target) 
Aug 31 20:02:32   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_nn.py", line 5016, in _do_extra_tests 
Aug 31 20:02:32     gradgradcheck(apply_fn, inputs) 
Aug 31 20:02:32   File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 486, in gradgradcheck 
Aug 31 20:02:32     nondet_tol=nondet_tol, check_undefined_grad=check_undefined_grad)

2 jobs timed out:

pytorch_windows_vs2019_py36_cuda10.1_test1
pytorch_linux_xenial_py3_clang5_asan_test1

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-xenial-rocm3.5.1-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 8 times.

Add reduction string test for ctc_loss.

432a9da

[ghstack-poisoned]

This was referenced Aug 31, 2020

Merge criterion and new criterion tests. #43788

Closed

Actually run backward criterion tests. #43809

Closed

gchanan added a commit that referenced this pull request Aug 31, 2020

Add reduction string test for ctc_loss.

deb403c

ghstack-source-id: 5c2a02e5cb21f314eeb828422ee74291adc2474f Pull Request resolved: #43888

gchanan closed this Sep 1, 2020

facebook-github-bot deleted the gh/gchanan/309/head branch October 2, 2020 14:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add reduction string test for ctc_loss. #43888

Add reduction string test for ctc_loss. #43888

gchanan commented Aug 31, 2020 •

edited

dr-ci bot commented Aug 31, 2020 •

edited

Add reduction string test for ctc_loss. #43888

Add reduction string test for ctc_loss. #43888

Conversation

gchanan commented Aug 31, 2020 • edited

dr-ci bot commented Aug 31, 2020 • edited

💊 CI failures summary and remediations

🕵️ 11 new failures recognized by patterns

pytorch_linux_xenial_cuda10_2_cudnn7_py3_ge_config_profiling_test (1/11)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_ge_config_legacy_test (2/11)

pytorch_xla_linux_bionic_py3_6_clang9_build (3/11)

pytorch_windows_vs2019_py36_cuda10.1_test2 (4/11)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (5/11)

pytorch_linux_bionic_py3_8_gcc9_coverage_test (6/11)

pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (7/11)

pytorch_linux_xenial_py3_clang5_asan_test2 (8/11)

pytorch_macos_10_13_py3_test (9/11)

pytorch_linux_xenial_py3_6_gcc5_4_build (10/11)

pytorch_linux_bionic_py3_6_clang9_test (11/11)

ci.pytorch.org: 1 failed

gchanan commented Aug 31, 2020 •

edited

dr-ci bot commented Aug 31, 2020 •

edited