Remove (most) Python 2 support from CI scripts #35616

dreiss · 2020-03-28T20:46:30Z

Stack from ghstack:

Remove (most) Python 2 support from CI scripts #35616 Remove (most) Python 2 support from CI scripts
Remove _reset_warning_registry #35619 Remove _reset_warning_registry
Remove __future__ imports from many files #35627 Remove future imports from many files
Remove caffe2/python/compatibility #35622 Remove caffe2/python/compatibility
Remove structseq_slice #35625 Remove structseq_slice
Remove (most) Python 2 support from C++ code #35614 Remove (most) Python 2 support from C++ code
Remove a use of exec #35624 Remove a use of exec
Remove a use of checkScript(str) #35623 Remove a use of checkScript(str)
Remove _get_wrapped_func #35621 Remove _get_wrapped_func
Remove _compatible_subtest #35620 Remove _compatible_subtest
Remove _uses_true_division #35618 Remove _uses_true_division
Remove (most) Python 2 support from setup.py #35617 Remove (most) Python 2 support from setup.py

Summary:
Python 2 has reached end-of-life and is no longer supported by PyTorch.
Now we can clean up a lot of cruft that we put in place to support it.
These changes were all done manually, and I skipped anything that seemed
like it would take more than a few seconds, so I think it makes sense to
review it manually as well.

Test Plan:
CI

Differential Revision: D20842882

dr-ci · 2020-03-28T20:47:18Z

💊 CI failures summary and remediations

As of commit 2f7b624 (more details on the Dr. CI page):

8/9 failures possibly* introduced in this PR
- 1/8 non-CircleCI failure(s)
1/9 tentatively recognized as flaky ❄️
- Click here to rerun these jobs

🕵️ 7 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_windows_vs2019_py36_cpu_test2 (1/7)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_overrides failed!

  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 817, in get_overridable_functions 
    if func in get_ignored_functions(): 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 127, in get_ignored_functions 
    torch.nn.functional.division, 
AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Traceback (most recent call last): 
  File "run_test.py", line 688, in <module> 
    main() 
  File "run_test.py", line 681, in main 
    raise RuntimeError(message) 
RuntimeError: test_overrides failed! 
 
(base) circleci@PACKER-5E9D8C4B C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

pytorch_linux_bionic_py3_6_clang9_test (2/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

May 14 04:47:25 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" }

May 14 04:47:24     raise RuntimeError(message) 
May 14 04:47:24 RuntimeError: test_overrides failed! 
May 14 04:47:25  
May 14 04:47:25 real	20m50.952s 
May 14 04:47:25 user	24m28.147s 
May 14 04:47:25 sys	2m11.391s 
May 14 04:47:25 + cleanup 
May 14 04:47:25 + retcode=1 
May 14 04:47:25 + set +x 
May 14 04:47:25 =================== sccache compilation log =================== 
May 14 04:47:25 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
May 14 04:47:25  
May 14 04:47:25 =========== If your build fails, please take a look at the log above for possible reasons =========== 
May 14 04:47:25 Compile requests                 68 
May 14 04:47:25 Compile requests executed        35 
May 14 04:47:25 Cache hits                       27 
May 14 04:47:25 Cache misses                      7 
May 14 04:47:25 Cache timeouts                    0 
May 14 04:47:25 Cache read errors                 0 
May 14 04:47:25 Forced recaches                   0 
May 14 04:47:25 Cache write errors                0

pytorch_windows_vs2019_py36_cuda10.1_on_cpu_test2 (3/7)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_overrides failed!

  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 817, in get_overridable_functions 
    if func in get_ignored_functions(): 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 127, in get_ignored_functions 
    torch.nn.functional.division, 
AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Traceback (most recent call last): 
  File "run_test.py", line 688, in <module> 
    main() 
  File "run_test.py", line 681, in main 
    raise RuntimeError(message) 
RuntimeError: test_overrides failed! 
 
(base) circleci@PACKER-5E9D8C4B C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

pytorch_linux_xenial_py3_6_gcc5_4_test (4/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

May 14 04:53:25 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" }

May 14 04:53:25 Traceback (most recent call last): 
May 14 04:53:25   File "test/run_test.py", line 688, in <module> 
May 14 04:53:25     main() 
May 14 04:53:25   File "test/run_test.py", line 681, in main 
May 14 04:53:25     raise RuntimeError(message) 
May 14 04:53:25 RuntimeError: test_overrides failed! 
May 14 04:53:25 + cleanup 
May 14 04:53:25 + retcode=1 
May 14 04:53:25 + set +x 
May 14 04:53:25 =================== sccache compilation log =================== 
May 14 04:53:25 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/var/lib/jenkins/.cache/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
May 14 04:53:25  
May 14 04:53:25 =========== If your build fails, please take a look at the log above for possible reasons =========== 
May 14 04:53:25 Compile requests                 68 
May 14 04:53:25 Compile requests executed        35 
May 14 04:53:25 Cache hits                       27 
May 14 04:53:25 Cache misses                      7 
May 14 04:53:25 Cache timeouts                    0 
May 14 04:53:25 Cache read errors                 0 
May 14 04:53:25 Forced recaches                   0 
May 14 04:53:25 Cache write errors                0

pytorch_macos_10_13_py3_test (5/7)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

May 13 21:54:10 RuntimeError: test_overrides failed!

May 13 21:54:10   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/_overrides.py", line 817, in get_overridable_functions 
May 13 21:54:10     if func in get_ignored_functions(): 
May 13 21:54:10   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/_overrides.py", line 127, in get_ignored_functions 
May 13 21:54:10     torch.nn.functional.division, 
May 13 21:54:10 AttributeError: module 'torch.nn.functional' has no attribute 'division' 
May 13 21:54:10 Traceback (most recent call last): 
May 13 21:54:10   File "test/run_test.py", line 688, in <module> 
May 13 21:54:10     main() 
May 13 21:54:10   File "test/run_test.py", line 681, in main 
May 13 21:54:10     raise RuntimeError(message) 
May 13 21:54:10 RuntimeError: test_overrides failed! 
May 13 21:54:11 + cleanup 
May 13 21:54:11 + retcode=1 
May 13 21:54:11 + set +x

pytorch_windows_vs2019_py36_cuda10.1_test2 (6/7)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_overrides failed!

  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 817, in get_overridable_functions 
    if func in get_ignored_functions(): 
  File "C:\Users\circleci\project\build\win_tmp\build\torch\_overrides.py", line 127, in get_ignored_functions 
    torch.nn.functional.division, 
AttributeError: module 'torch.nn.functional' has no attribute 'division' 
Traceback (most recent call last): 
  File "run_test.py", line 688, in <module> 
    main() 
  File "run_test.py", line 681, in main 
    raise RuntimeError(message) 
RuntimeError: test_overrides failed! 
 
(base) circleci@PACKER-5E9E037A C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

pytorch_linux_xenial_py3_clang5_asan_test (7/7)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

May 14 04:32:22 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/jenkins/workspace/aten/src/ATen/Utils.cpp:11:3 in

May 14 04:32:22     #7 0x55c93e07874b in PyEval_EvalCode /tmp/build/80754af9/python_1585002248360/work/Python/ceval.c:731 
May 14 04:32:22     #8 0x55c93e0f8633 in run_mod /tmp/build/80754af9/python_1585002248360/work/Python/pythonrun.c:1025 
May 14 04:32:22     #9 0x55c93e0f86cc in PyRun_StringFlags /tmp/build/80754af9/python_1585002248360/work/Python/pythonrun.c:949 
May 14 04:32:22     #10 0x55c93e0f872e in PyRun_SimpleStringFlags /tmp/build/80754af9/python_1585002248360/work/Python/pythonrun.c:445 
May 14 04:32:22     #11 0x55c93e0fc532 in run_command /tmp/build/80754af9/python_1585002248360/work/Modules/main.c:301 
May 14 04:32:22     #12 0x55c93e0fc532 in Py_Main /tmp/build/80754af9/python_1585002248360/work/Modules/main.c:749 
May 14 04:32:22     #13 0x55c93dfc71fd in main /tmp/build/80754af9/python_1585002248360/work/Programs/python.c:69 
May 14 04:32:22     #14 0x7f9adce9982f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291 
May 14 04:32:22     #15 0x55c93e0a5c29 in _start /home/rdonnelly/mc/conda-bld/compilers_linux-64_1534865402226/work/.build/src/glibc-2.12.2/csu/../sysdeps/x86_64/elf/start.S:103 
May 14 04:32:22  
May 14 04:32:22 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /var/lib/jenkins/workspace/aten/src/ATen/Utils.cpp:11:3 in  
May 14 04:32:22 + retcode=1 
May 14 04:32:22 + set -e 
May 14 04:32:22 + return 1 
May 14 04:32:22 + [[ pytorch-linux-xenial-py3-clang5-asan-test == *-NO_AVX-* ]] 
May 14 04:32:22 + [[ pytorch-linux-xenial-py3-clang5-asan-test == *-NO_AVX2-* ]] 
May 14 04:32:22 + '[' -n https://github.com/pytorch/pytorch/pull/35616 ']' 
May 14 04:32:22 ++ mktemp 
May 14 04:32:22 + DETERMINE_FROM=/tmp/tmp.qZco4dbN80 
May 14 04:32:22 + file_diff_from_base /tmp/tmp.qZco4dbN80 
May 14 04:32:22 + set +e

❄️ 1 failure tentatively classified as flaky

but reruns have not yet been triggered to confirm:

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

May 14 05:11:07 ConnectionResetError: [Errno 104] Connection reset by peer

May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 494, in Client 
May 14 05:11:07     deliver_challenge(c, authkey) 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge 
May 14 05:11:07     response = connection.recv_bytes(256)        # reject large message 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes 
May 14 05:11:07     buf = self._recv_bytes(maxlength) 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes 
May 14 05:11:07     buf = self._recv(4) 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv 
May 14 05:11:07     chunk = read(handle, remaining) 
May 14 05:11:07 ConnectionResetError: [Errno 104] Connection reset by peer 
May 14 05:11:07  
May 14 05:11:07 Process ErrorTrackingProcess-120: 
May 14 05:11:07 Traceback (most recent call last): 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap 
May 14 05:11:07     self.run() 
May 14 05:11:07   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 362, in run 
May 14 05:11:07     super(ErrorTrackingProcess, self).run() 
May 14 05:11:07   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run 
May 14 05:11:07     self._target(*self._args, **self._kwargs) 
May 14 05:11:07   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 630, in _test_proper_exit

ci.pytorch.org: 1 failed

Failed: pr/py3.6-clang7-rocmdeb-ubuntu16.04

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 62 times.

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. Now we can clean up a lot of cruft that we put in place to support it. These changes were all done manually, and I skipped anything that seemed like it would take more than a few seconds, so I think it makes sense to review it manually as well. Test Plan: CI ghstack-source-id: 50a5352 Pull Request resolved: pytorch/pytorch#35616

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. Now we can clean up a lot of cruft that we put in place to support it. These changes were all done manually, and I skipped anything that seemed like it would take more than a few seconds, so I think it makes sense to review it manually as well. Test Plan: CI Differential Revision: [D20842882](https://our.internmc.facebook.com/intern/diff/D20842882)

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. Now we can clean up a lot of cruft that we put in place to support it. These changes were all done manually, and I skipped anything that seemed like it would take more than a few seconds, so I think it makes sense to review it manually as well. Test Plan: CI ghstack-source-id: 5bf5b81 Pull Request resolved: #35616

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. Now we can clean up a lot of cruft that we put in place to support it. These changes were all done manually, and I skipped anything that seemed like it would take more than a few seconds, so I think it makes sense to review it manually as well. Test Plan: CI Differential Revision: [D20842882](https://our.internmc.facebook.com/intern/diff/D20842882)

Summary: Python 2 has reached end-of-life and is no longer supported by PyTorch. Now we can clean up a lot of cruft that we put in place to support it. These changes were all done manually, and I skipped anything that seemed like it would take more than a few seconds, so I think it makes sense to review it manually as well. Test Plan: CI ghstack-source-id: a9ce516 Pull Request resolved: #35616

github-actions · 2022-04-12T19:52:32Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

facebook-github-bot added the cla signed label Oct 30, 2020

github-actions bot added the Stale label Apr 12, 2022

github-actions bot closed this May 12, 2022

facebook-github-bot deleted the gh/dreiss/33/head branch June 12, 2022 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Remove (most) Python 2 support from CI scripts #35616

Remove (most) Python 2 support from CI scripts #35616

Uh oh!

dreiss commented Mar 28, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Mar 28, 2020 •

edited

Loading

Uh oh!

github-actions bot commented Apr 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Remove (most) Python 2 support from CI scripts #35616

Remove (most) Python 2 support from CI scripts #35616

Uh oh!

Conversation

dreiss commented Mar 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Mar 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 7 new failures recognized by patterns

pytorch_windows_vs2019_py36_cpu_test2 (1/7)

pytorch_linux_bionic_py3_6_clang9_test (2/7)

pytorch_windows_vs2019_py36_cuda10.1_on_cpu_test2 (3/7)

pytorch_linux_xenial_py3_6_gcc5_4_test (4/7)

pytorch_macos_10_13_py3_test (5/7)

pytorch_windows_vs2019_py36_cuda10.1_test2 (6/7)

pytorch_linux_xenial_py3_clang5_asan_test (7/7)

❄️ 1 failure tentatively classified as flaky

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test (1/1)

ci.pytorch.org: 1 failed

Uh oh!

github-actions bot commented Apr 12, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dreiss commented Mar 28, 2020 •

edited

Loading

dr-ci bot commented Mar 28, 2020 •

edited

Loading