Skip to content

Conversation

@jerryzh168
Copy link
Contributor

@jerryzh168 jerryzh168 commented Feb 13, 2020

Stack from ghstack:

Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@jerryzh168 jerryzh168 requested a review from apaszke as a code owner February 13, 2020 03:23
This was referenced Feb 13, 2020
@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Feb 13, 2020
@dr-ci
Copy link

dr-ci bot commented Feb 13, 2020

💊 CircleCI build failures summary and remediations

As of commit 697edf6:

  • 3/14 broken upstream at merge base ae53f8d since Feb 14

    Please rebase on the viable/strict branch (expand for instructions)

    If your commit is newer than viable/strict, you can try basing on an older, stable commit:

    git fetch origin viable/strict
    git rebase --onto viable/strict $(git merge-base origin/master HEAD)
    

    If your commit is older than viable/strict:

    git fetch origin viable/strict
    git rebase viable/strict
    

    Check out the recency history of this "viable master" tracking branch.

  • 8/14 failures introduced in this PR

  • 3/14 recognized as flaky ❄️

    • Re-run these jobs?

Detailed failure analysis

One may explore the probable reasons each build failed interactively on the Dr. CI website.

🕵️ 8 new failures recognized by patterns

The following build failures do not appear to be due to upstream breakage:

See CircleCI build pytorch_macos_10_13_py3_test (1/8)

Step: "Test" (full log | pattern match details)

Feb 14 15:16:14 RuntimeError: test_jit failed!
Feb 14 15:16:14 Running test_jit ... [2020-02-14 15:16:14.585902] 
Feb 14 15:16:14   File "test_jit.py", line 1535 
Feb 14 15:16:14     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 14 15:16:14     ^ 
Feb 14 15:16:14 IndentationError: unexpected indent 
Feb 14 15:16:14 Traceback (most recent call last): 
Feb 14 15:16:14   File "test/run_test.py", line 486, in <module> 
Feb 14 15:16:14     main() 
Feb 14 15:16:14   File "test/run_test.py", line 479, in main 
Feb 14 15:16:14     raise RuntimeError(message) 
Feb 14 15:16:14 RuntimeError: test_jit failed! 
Feb 14 15:16:14 + cleanup 
Feb 14 15:16:14 + retcode=1 
Feb 14 15:16:14 + set +x 

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_slow_test (2/8)

Step: "Test" (full log | pattern match details)

Feb 14 23:27:54 RuntimeError: test_jit failed!
Feb 14 23:27:54 Running test_jit ... [2020-02-14 23:27:54.332836] 
Feb 14 23:27:54   File "test_jit.py", line 1535 
Feb 14 23:27:54     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 14 23:27:54     ^ 
Feb 14 23:27:54 IndentationError: unexpected indent 
Feb 14 23:27:54 Traceback (most recent call last): 
Feb 14 23:27:54   File "test/run_test.py", line 486, in <module> 
Feb 14 23:27:54     main() 
Feb 14 23:27:54   File "test/run_test.py", line 479, in main 
Feb 14 23:27:54     raise RuntimeError(message) 
Feb 14 23:27:54 RuntimeError: test_jit failed! 
Feb 14 23:27:54 + cleanup 
Feb 14 23:27:54 + retcode=1 
Feb 14 23:27:54 + set +x 
Feb 14 23:27:54 =================== sccache compilation log =================== 
Feb 14 23:27:54 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Feb 14 23:27:54  
Feb 14 23:27:55 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 14 23:27:55 Compile requests               161 
Feb 14 23:27:55 Compile requests executed       57 
Feb 14 23:27:55 Cache hits                      45 

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_nogpu_test (3/8)

Step: "Test" (full log | pattern match details)

Feb 14 23:43:06 RuntimeError: test_jit failed!
Feb 14 23:43:06 Running test_jit ... [2020-02-14 23:43:06.876527] 
Feb 14 23:43:06   File "test_jit.py", line 1535 
Feb 14 23:43:06     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 14 23:43:06     ^ 
Feb 14 23:43:06 IndentationError: unexpected indent 
Feb 14 23:43:06 Traceback (most recent call last): 
Feb 14 23:43:06   File "test/run_test.py", line 486, in <module> 
Feb 14 23:43:06     main() 
Feb 14 23:43:06   File "test/run_test.py", line 479, in main 
Feb 14 23:43:06     raise RuntimeError(message) 
Feb 14 23:43:06 RuntimeError: test_jit failed! 
Feb 14 23:43:07 + cleanup 
Feb 14 23:43:07 + retcode=1 
Feb 14 23:43:07 + set +x 
Feb 14 23:43:07 =================== sccache compilation log =================== 
Feb 14 23:43:07 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Feb 14 23:43:07  
Feb 14 23:43:07 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 14 23:43:07 Compile requests                 49 
Feb 14 23:43:07 Compile requests executed        28 
Feb 14 23:43:07 Cache hits                       21 

See CircleCI build pytorch_windows_test2 (4/8)

Step: "Test" (full log | pattern match details)

RuntimeError: test_jit failed!
Running test_jit ... [2020-02-14 23:45:50.828860] 
  File "test_jit.py", line 1535 
    self.bn = torch.nn.BatchNorm2d(num_features=20) 
    ^ 
IndentationError: unexpected indent 
Traceback (most recent call last): 
  File "run_test.py", line 486, in <module> 
    main() 
  File "run_test.py", line 479, in main 
    raise RuntimeError(message) 
RuntimeError: test_jit failed! 
 
(base) circleci@PACKER-5E29F737 C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1  
+ cleanup
+ retcode=1
+ set +x

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_ge_config_simple_test (5/8)

Step: "Test" (full log | pattern match details)

Feb 14 23:52:29 RuntimeError: test_jit_simple failed!
Feb 14 23:52:29     from test_jit import * 
Feb 14 23:52:29   File "/var/lib/jenkins/workspace/test/test_jit.py", line 1535 
Feb 14 23:52:29     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 14 23:52:29     ^ 
Feb 14 23:52:29 IndentationError: unexpected indent 
Feb 14 23:52:29 Traceback (most recent call last): 
Feb 14 23:52:29   File "test/run_test.py", line 486, in <module> 
Feb 14 23:52:29     main() 
Feb 14 23:52:29   File "test/run_test.py", line 479, in main 
Feb 14 23:52:29     raise RuntimeError(message) 
Feb 14 23:52:29 RuntimeError: test_jit_simple failed! 
Feb 14 23:52:29 + cleanup 
Feb 14 23:52:29 + retcode=1 
Feb 14 23:52:29 + set +x 
Feb 14 23:52:29 =================== sccache compilation log =================== 
Feb 14 23:52:29 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 14 23:52:29 Compile requests                 0 
Feb 14 23:52:29 Compile requests executed        0 
Feb 14 23:52:29 Cache hits                       0 
Feb 14 23:52:29 Cache misses                     0 
Feb 14 23:52:29 Cache timeouts                   0 

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_ge_config_legacy_test (6/8)

Step: "Test" (full log | pattern match details)

Feb 14 23:52:35 RuntimeError: test_jit_legacy failed!
Feb 14 23:52:35     from test_jit import * 
Feb 14 23:52:35   File "/var/lib/jenkins/workspace/test/test_jit.py", line 1535 
Feb 14 23:52:35     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 14 23:52:35     ^ 
Feb 14 23:52:35 IndentationError: unexpected indent 
Feb 14 23:52:35 Traceback (most recent call last): 
Feb 14 23:52:35   File "test/run_test.py", line 486, in <module> 
Feb 14 23:52:35     main() 
Feb 14 23:52:35   File "test/run_test.py", line 479, in main 
Feb 14 23:52:35     raise RuntimeError(message) 
Feb 14 23:52:35 RuntimeError: test_jit_legacy failed! 
Feb 14 23:52:35 + cleanup 
Feb 14 23:52:35 + retcode=1 
Feb 14 23:52:35 + set +x 
Feb 14 23:52:35 =================== sccache compilation log =================== 
Feb 14 23:52:35 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 14 23:52:35 Compile requests                 0 
Feb 14 23:52:35 Compile requests executed        0 
Feb 14 23:52:35 Cache hits                       0 
Feb 14 23:52:35 Cache misses                     0 
Feb 14 23:52:35 Cache timeouts                   0 

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_test (7/8)

Step: "Test" (full log | pattern match details)

Feb 15 00:19:23 RuntimeError: test_jit failed!
Feb 15 00:19:22 Running test_jit ... [2020-02-15 00:19:22.970025] 
Feb 15 00:19:23   File "test_jit.py", line 1535 
Feb 15 00:19:23     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 15 00:19:23     ^ 
Feb 15 00:19:23 IndentationError: unexpected indent 
Feb 15 00:19:23 Traceback (most recent call last): 
Feb 15 00:19:23   File "test/run_test.py", line 486, in <module> 
Feb 15 00:19:23     main() 
Feb 15 00:19:23   File "test/run_test.py", line 479, in main 
Feb 15 00:19:23     raise RuntimeError(message) 
Feb 15 00:19:23 RuntimeError: test_jit failed! 
Feb 15 00:19:23 + cleanup 
Feb 15 00:19:23 + retcode=1 
Feb 15 00:19:23 + set +x 
Feb 15 00:19:23 =================== sccache compilation log =================== 
Feb 15 00:19:23 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 15 00:19:23 Compile requests                42 
Feb 15 00:19:23 Compile requests executed       22 
Feb 15 00:19:23 Cache hits                      21 
Feb 15 00:19:23 Cache misses                     0 
Feb 15 00:19:23 Cache timeouts                   0 

See CircleCI build pytorch_linux_xenial_py3_clang5_asan_test (8/8)

Step: "Test" (full log | pattern match details)

Feb 15 00:23:40 RuntimeError: test_jit failed!
Feb 15 00:23:40 Running test_jit ... [2020-02-15 00:23:40.250084] 
Feb 15 00:23:40   File "test_jit.py", line 1535 
Feb 15 00:23:40     self.bn = torch.nn.BatchNorm2d(num_features=20) 
Feb 15 00:23:40     ^ 
Feb 15 00:23:40 IndentationError: unexpected indent 
Feb 15 00:23:40 Traceback (most recent call last): 
Feb 15 00:23:40   File "test/run_test.py", line 486, in <module> 
Feb 15 00:23:40     main() 
Feb 15 00:23:40   File "test/run_test.py", line 479, in main 
Feb 15 00:23:40     raise RuntimeError(message) 
Feb 15 00:23:40 RuntimeError: test_jit failed! 
Feb 15 00:23:40 + cleanup 
Feb 15 00:23:40 + retcode=1 
Feb 15 00:23:40 + set +x 
Feb 15 00:23:40 =================== sccache compilation log =================== 
Feb 15 00:23:40 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Feb 15 00:23:40 Compile requests                42 
Feb 15 00:23:40 Compile requests executed       22 
Feb 15 00:23:40 Cache hits                      21 
Feb 15 00:23:40 Cache misses                     0 
Feb 15 00:23:40 Cache timeouts                   0 

❄️ 3 failures recognized as flaky

The following build failures have been detected as flaky and may not be your fault:

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_gcc7_test (1/3)

Step: "Test" (full log | pattern match details) ❄️

Feb 15 00:05:12 ConnectionResetError: [Errno 104] Connection reset by peer
Feb 15 00:05:12   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 455, in accept 
Feb 15 00:05:12     deliver_challenge(c, self._authkey) 
Feb 15 00:05:12   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge 
Feb 15 00:05:12     response = connection.recv_bytes(256)        # reject large message 
Feb 15 00:05:12   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes 
Feb 15 00:05:12     buf = self._recv_bytes(maxlength) 
Feb 15 00:05:12   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes 
Feb 15 00:05:12     buf = self._recv(4) 
Feb 15 00:05:12   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv 
Feb 15 00:05:12     chunk = read(handle, remaining) 
Feb 15 00:05:12 ConnectionResetError: [Errno 104] Connection reset by peer 
Feb 15 00:05:12 /opt/conda/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown 
Feb 15 00:05:12   len(cache)) 
Feb 15 00:05:15 Process ErrorTrackingProcess-126: 
Feb 15 00:05:15 Traceback (most recent call last): 
Feb 15 00:05:15   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap 
Feb 15 00:05:15     self.run() 
Feb 15 00:05:15   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 333, in run 
Feb 15 00:05:15     super(ErrorTrackingProcess, self).run() 
Feb 15 00:05:15   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run 
Feb 15 00:05:15     self._target(*self._args, **self._kwargs) 

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_NO_AVX2_test (2/3)

Step: "Test" (full log | pattern match details) ❄️

Feb 15 00:05:18 ConnectionResetError: [Errno 104] Connection reset by peer
Feb 15 00:05:18   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 455, in accept 
Feb 15 00:05:18     deliver_challenge(c, self._authkey) 
Feb 15 00:05:18   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge 
Feb 15 00:05:18     response = connection.recv_bytes(256)        # reject large message 
Feb 15 00:05:18   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes 
Feb 15 00:05:18     buf = self._recv_bytes(maxlength) 
Feb 15 00:05:18   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes 
Feb 15 00:05:18     buf = self._recv(4) 
Feb 15 00:05:18   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv 
Feb 15 00:05:18     chunk = read(handle, remaining) 
Feb 15 00:05:18 ConnectionResetError: [Errno 104] Connection reset by peer 
Feb 15 00:05:18 /opt/conda/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown 
Feb 15 00:05:18   len(cache)) 
Feb 15 00:05:21 Process ErrorTrackingProcess-122: 
Feb 15 00:05:21 Traceback (most recent call last): 
Feb 15 00:05:21   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap 
Feb 15 00:05:21     self.run() 
Feb 15 00:05:21   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 333, in run 
Feb 15 00:05:21     super(ErrorTrackingProcess, self).run() 
Feb 15 00:05:21   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run 
Feb 15 00:05:21     self._target(*self._args, **self._kwargs) 

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_NO_AVX_NO_AVX2_test (3/3)

Step: "Test" (full log | pattern match details) ❄️

Feb 15 00:06:10 ConnectionResetError: [Errno 104] Connection reset by peer
Feb 15 00:06:10   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 455, in accept 
Feb 15 00:06:10     deliver_challenge(c, self._authkey) 
Feb 15 00:06:10   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge 
Feb 15 00:06:10     response = connection.recv_bytes(256)        # reject large message 
Feb 15 00:06:10   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes 
Feb 15 00:06:10     buf = self._recv_bytes(maxlength) 
Feb 15 00:06:10   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes 
Feb 15 00:06:10     buf = self._recv(4) 
Feb 15 00:06:10   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv 
Feb 15 00:06:10     chunk = read(handle, remaining) 
Feb 15 00:06:10 ConnectionResetError: [Errno 104] Connection reset by peer 
Feb 15 00:06:10 /opt/conda/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown 
Feb 15 00:06:10   len(cache)) 
Feb 15 00:06:14 Process ErrorTrackingProcess-122: 
Feb 15 00:06:14 Traceback (most recent call last): 
Feb 15 00:06:14   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap 
Feb 15 00:06:14     self.run() 
Feb 15 00:06:14   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 333, in run 
Feb 15 00:06:14     super(ErrorTrackingProcess, self).run() 
Feb 15 00:06:14   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run 
Feb 15 00:06:14     self._target(*self._args, **self._kwargs) 

🚧 3 upstream failures recognized by patterns:

These builds matched patterns, but were probably caused by upstream breakages:


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 22 times.

Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
Summary:
Move QuantFusion before insertGuards to prevent it from breaking
the quantization patterns

Test Plan:
python test/test_jit.py

Reviewers:
mvz, suo

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@jerryzh168
Copy link
Contributor Author

We changed plans for QuantFusion, we'll be calling it in freezing, so this PR is not needed anymore

@jerryzh168 jerryzh168 closed this Feb 15, 2020
@facebook-github-bot facebook-github-bot deleted the gh/jerryzh168/220/head branch March 16, 2020 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants