Skip to content

[JIT] add unsupported section #31329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

[JIT] add unsupported section #31329

wants to merge 6 commits into from

Conversation

eellison
Copy link
Contributor

Add a section for unsupported ops, and modules. Automatically generate the properties and attributes that aren't bound, and for ops that have semantic mismatches set up tests so the docs stay up to date.

@eellison eellison requested a review from driazati December 16, 2019 21:08
@eellison eellison requested a review from apaszke as a code owner December 16, 2019 21:08
@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Dec 16, 2019
@kostmo
Copy link
Member

kostmo commented Dec 16, 2019 •
edited
Loading

💊 CircleCI build failures summary and remediations

As of commit feba4b4:

  • 1/10 broken upstream at merge base dab5f72 (see grid view)

    You may want to rebase on the viable/strict branch (expand for instructions)

    Since your merge base is older than viable/strict, run these commands:

    git fetch viable/strict
git rebase viable/strict

    Check out the recency history of this "viable master" tracking branch.

  • 9/10 failures introduced in this PR

Detailed failure analysis

One may explore the probable reasons each build failed interactively on the Dr. CI website.

🕵️ 9 new failures recognized by patterns

The following build failures don't appear to be due to upstream breakage:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_ge_config_simple_test (1/9)

Step: "Test" (full log | pattern match details)

Dec 18 01:54:12 RuntimeError: test_jit_simple failed! 
Dec 18 01:54:12 Ran 2300 tests in 89.062s 
Dec 18 01:54:12  
Dec 18 01:54:12 FAILED (errors=1, skipped=77, expected failures=1) 
Dec 18 01:54:12  
Dec 18 01:54:12 Generating XML reports... 
Dec 18 01:54:12 Traceback (most recent call last): 
Dec 18 01:54:12   File "test/run_test.py", line 456, in <module> 
Dec 18 01:54:12     main() 
Dec 18 01:54:12   File "test/run_test.py", line 449, in main 
Dec 18 01:54:12     raise RuntimeError(message) 
Dec 18 01:54:12 RuntimeError: test_jit_simple failed! 
Dec 18 01:54:12 + cleanup 
Dec 18 01:54:12 + retcode=1 
Dec 18 01:54:12 + set +x 
Dec 18 01:54:12 =================== sccache compilation log =================== 
Dec 18 01:54:12 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 01:54:12 Compile requests                56 
Dec 18 01:54:12 Compile requests executed        0 
Dec 18 01:54:12 Cache hits                       0 
Dec 18 01:54:12 Cache misses                     0 
Dec 18 01:54:12 Cache timeouts                   0

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_ge_config_legacy_test (2/9)

Step: "Test" (full log | pattern match details)

Dec 18 01:54:47 RuntimeError: test_jit_legacy failed! 
Dec 18 01:54:46 Ran 2300 tests in 152.756s 
Dec 18 01:54:46  
Dec 18 01:54:46 FAILED (errors=1, skipped=59, expected failures=1) 
Dec 18 01:54:46  
Dec 18 01:54:46 Generating XML reports... 
Dec 18 01:54:47 Traceback (most recent call last): 
Dec 18 01:54:47   File "test/run_test.py", line 456, in <module> 
Dec 18 01:54:47     main() 
Dec 18 01:54:47   File "test/run_test.py", line 449, in main 
Dec 18 01:54:47     raise RuntimeError(message) 
Dec 18 01:54:47 RuntimeError: test_jit_legacy failed! 
Dec 18 01:54:47 + cleanup 
Dec 18 01:54:47 + retcode=1 
Dec 18 01:54:47 + set +x 
Dec 18 01:54:47 =================== sccache compilation log =================== 
Dec 18 01:54:47 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 01:54:47 Compile requests               304 
Dec 18 01:54:47 Compile requests executed        0 
Dec 18 01:54:47 Cache hits                       0 
Dec 18 01:54:47 Cache misses                     0 
Dec 18 01:54:47 Cache timeouts                   0

See CircleCI build pytorch_linux_xenial_py2_7_9_test (3/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:04:53 RuntimeError: test_jit failed! 
Dec 18 02:04:52 Ran 2300 tests in 104.621s 
Dec 18 02:04:52  
Dec 18 02:04:52 FAILED (errors=1, skipped=74, expected failures=1) 
Dec 18 02:04:52  
Dec 18 02:04:52 Generating XML reports... 
Dec 18 02:04:53 Traceback (most recent call last): 
Dec 18 02:04:53   File "test/run_test.py", line 456, in <module> 
Dec 18 02:04:53     main() 
Dec 18 02:04:53   File "test/run_test.py", line 449, in main 
Dec 18 02:04:53     raise RuntimeError(message) 
Dec 18 02:04:53 RuntimeError: test_jit failed! 
Dec 18 02:04:53 + cleanup 
Dec 18 02:04:53 + retcode=1 
Dec 18 02:04:53 + set +x 
Dec 18 02:04:53 =================== sccache compilation log =================== 
Dec 18 02:04:53 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Dec 18 02:04:53  
Dec 18 02:04:53 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:04:53 Compile requests                105 
Dec 18 02:04:53 Compile requests executed        27 
Dec 18 02:04:53 Cache hits                       20

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_test (4/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:14:34 RuntimeError: test_jit failed! 
Dec 18 02:14:34 Ran 2300 tests in 139.490s 
Dec 18 02:14:34  
Dec 18 02:14:34 FAILED (errors=1, skipped=59, expected failures=1) 
Dec 18 02:14:34  
Dec 18 02:14:34 Generating XML reports... 
Dec 18 02:14:34 Traceback (most recent call last): 
Dec 18 02:14:34   File "test/run_test.py", line 456, in <module> 
Dec 18 02:14:34     main() 
Dec 18 02:14:34   File "test/run_test.py", line 449, in main 
Dec 18 02:14:34     raise RuntimeError(message) 
Dec 18 02:14:34 RuntimeError: test_jit failed! 
Dec 18 02:14:34 =================== sccache compilation log =================== 
Dec 18 02:14:34 + cleanup 
Dec 18 02:14:34 + retcode=1 
Dec 18 02:14:34 + set +x 
Dec 18 02:14:34 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Dec 18 02:14:34  
Dec 18 02:14:34 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:14:34 Compile requests                102 
Dec 18 02:14:34 Compile requests executed        25 
Dec 18 02:14:34 Cache hits                       18

See CircleCI build pytorch_linux_xenial_cuda9_cudnn7_py3_nogpu_test (5/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:29:12 RuntimeError: test_jit failed! 
Dec 18 02:29:11 Ran 2300 tests in 120.331s 
Dec 18 02:29:11  
Dec 18 02:29:11 FAILED (errors=1, skipped=59, expected failures=1) 
Dec 18 02:29:11  
Dec 18 02:29:11 Generating XML reports... 
Dec 18 02:29:12 Traceback (most recent call last): 
Dec 18 02:29:12   File "test/run_test.py", line 456, in <module> 
Dec 18 02:29:12     main() 
Dec 18 02:29:12   File "test/run_test.py", line 449, in main 
Dec 18 02:29:12     raise RuntimeError(message) 
Dec 18 02:29:12 RuntimeError: test_jit failed! 
Dec 18 02:29:12 + cleanup 
Dec 18 02:29:12 + retcode=1 
Dec 18 02:29:12 + set +x 
Dec 18 02:29:12 =================== sccache compilation log =================== 
Dec 18 02:29:12 ERROR:sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "/tmp/torch_extensions/test_compilation_error_formatting/main.cpp: In function \'int main()\':\n/tmp/torch_extensions/test_compilation_error_formatting/main.cpp:2:23: error: expected \';\' before \'}\' token\n int main() { return 0 }\n                       ^\n" } 
Dec 18 02:29:12  
Dec 18 02:29:12 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:29:12 Compile requests                102 
Dec 18 02:29:12 Compile requests executed        25 
Dec 18 02:29:12 Cache hits                       18

See CircleCI build pytorch_macos_10_13_py3_test (6/9)

Step: "Test" (full log | pattern match details)

Dec 17 18:34:35 RuntimeError: test_jit failed! 
Dec 17 18:34:34 Ran 2297 tests in 134.038s 
Dec 17 18:34:34  
Dec 17 18:34:34 FAILED (errors=1, skipped=64, expected failures=1) 
Dec 17 18:34:34  
Dec 17 18:34:34 Generating XML reports... 
Dec 17 18:34:35 Traceback (most recent call last): 
Dec 17 18:34:35   File "test/run_test.py", line 456, in <module> 
Dec 17 18:34:35     main() 
Dec 17 18:34:35   File "test/run_test.py", line 449, in main 
Dec 17 18:34:35     raise RuntimeError(message) 
Dec 17 18:34:35 RuntimeError: test_jit failed! 
Dec 17 18:34:35 + cleanup 
Dec 17 18:34:35 + retcode=1 
Dec 17 18:34:35 + set +x

See CircleCI build pytorch_linux_xenial_cuda9_cudnn7_py3_test (7/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:50:48 RuntimeError: test_jit failed! 
Dec 18 02:50:47 Ran 2300 tests in 120.695s 
Dec 18 02:50:47  
Dec 18 02:50:47 FAILED (errors=1, skipped=30, expected failures=1) 
Dec 18 02:50:47  
Dec 18 02:50:47 Generating XML reports... 
Dec 18 02:50:48 Traceback (most recent call last): 
Dec 18 02:50:48   File "test/run_test.py", line 456, in <module> 
Dec 18 02:50:48     main() 
Dec 18 02:50:48   File "test/run_test.py", line 449, in main 
Dec 18 02:50:48     raise RuntimeError(message) 
Dec 18 02:50:48 RuntimeError: test_jit failed! 
Dec 18 02:50:48 + cleanup 
Dec 18 02:50:48 + retcode=1 
Dec 18 02:50:48 + set +x 
Dec 18 02:50:48 =================== sccache compilation log =================== 
Dec 18 02:50:48 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:50:48 Compile requests                 3 
Dec 18 02:50:48 Compile requests executed        0 
Dec 18 02:50:48 Cache hits                       0 
Dec 18 02:50:48 Cache misses                     0 
Dec 18 02:50:48 Cache timeouts                   0

See CircleCI build pytorch_linux_xenial_cuda9_cudnn7_py3_NO_AVX_NO_AVX2_test (8/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:51:01 RuntimeError: test_jit failed! 
Dec 18 02:51:01 Ran 2300 tests in 118.453s 
Dec 18 02:51:01  
Dec 18 02:51:01 FAILED (errors=1, skipped=30, expected failures=1) 
Dec 18 02:51:01  
Dec 18 02:51:01 Generating XML reports... 
Dec 18 02:51:01 Traceback (most recent call last): 
Dec 18 02:51:01   File "test/run_test.py", line 456, in <module> 
Dec 18 02:51:01     main() 
Dec 18 02:51:01   File "test/run_test.py", line 449, in main 
Dec 18 02:51:01     raise RuntimeError(message) 
Dec 18 02:51:01 RuntimeError: test_jit failed! 
Dec 18 02:51:02 + cleanup 
Dec 18 02:51:02 + retcode=1 
Dec 18 02:51:02 + set +x 
Dec 18 02:51:02 =================== sccache compilation log =================== 
Dec 18 02:51:02 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:51:02 Compile requests                 3 
Dec 18 02:51:02 Compile requests executed        0 
Dec 18 02:51:02 Cache hits                       0 
Dec 18 02:51:02 Cache misses                     0 
Dec 18 02:51:02 Cache timeouts                   0

See CircleCI build pytorch_linux_xenial_cuda9_cudnn7_py3_NO_AVX2_test (9/9)

Step: "Test" (full log | pattern match details)

Dec 18 02:51:08 RuntimeError: test_jit failed! 
Dec 18 02:51:08 Ran 2300 tests in 118.773s 
Dec 18 02:51:08  
Dec 18 02:51:08 FAILED (errors=1, skipped=30, expected failures=1) 
Dec 18 02:51:08  
Dec 18 02:51:08 Generating XML reports... 
Dec 18 02:51:08 Traceback (most recent call last): 
Dec 18 02:51:08   File "test/run_test.py", line 456, in <module> 
Dec 18 02:51:08     main() 
Dec 18 02:51:08   File "test/run_test.py", line 449, in main 
Dec 18 02:51:08     raise RuntimeError(message) 
Dec 18 02:51:08 RuntimeError: test_jit failed! 
Dec 18 02:51:09 + cleanup 
Dec 18 02:51:09 + retcode=1 
Dec 18 02:51:09 + set +x 
Dec 18 02:51:09 =================== sccache compilation log =================== 
Dec 18 02:51:09 =========== If your build fails, please take a look at the log above for possible reasons =========== 
Dec 18 02:51:09 Compile requests                 3 
Dec 18 02:51:09 Compile requests executed        0 
Dec 18 02:51:09 Cache hits                       0 
Dec 18 02:51:09 Cache misses                     0 
Dec 18 02:51:09 Cache timeouts                   0

🚧 1 upstream failure recognized by patterns:

These builds matched patterns, but were probably caused by upstream breakages:

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 26 times.

driazati
driazati approved these changes Dec 17, 2019
View reviewed changes
Copy link
Contributor

@driazati driazati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good start but it's missing a lot of stuff from the quip (which itself is missing some other stuff like the unsupported parts of torch.nn.init)

docs/source/jit_unsupported.rst Outdated

TorchScript supports most methods defined on `torch` and `torch.Tensor`, but we do not have full coverage.
Here are specific known ops and categories of ops which have diverging behavior between
Python and TorchScript. Deprecated ops are not listed below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a note that says if you encounter something not listed here that's not supported you should file a GitHub issue

docs/source/jit_unsupported.rst
we suggest using :meth:`torch.jit.trace`.

* :class:`torch.nn.RNN`
* :class:`torch.nn.AdaptiveLogSoftmaxWithLoss`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing torch.nn.modules.normalization.CrossMapLRN2d

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CrossMapLRN2d isn't documented

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CrossMapLRN2d isn't in our documentation

docs/source/jit_unsupported.rst Outdated

TorchScript supports most methods defined on `torch` and `torch.Tensor`, but we do not have full coverage.
Here are specific known ops and categories of ops which have diverging behavior between
Python and TorchScript. Deprecated ops are not listed below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not list deprecated ops? We are detecting them below, we could also tag them with something like * resize (this method is deprecated and won't be supported)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, we can do that

driazati
driazati reviewed Dec 17, 2019
View reviewed changes
torch/jit/unsupported_tensor_ops.py
return x.{op}()
''')

deprecated_apis = set(["volatile", "resize", "reinforce", "new", "name", "map2_", "has_names", "grad_fn", "resize_as"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also where is this list coming from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the undocumented apis & properties. I could try to expose the list here: https://github.com/pytorch/pytorch/blob/master/test/test_docs_coverage.py#L83

@eellison
Copy link
Contributor Author

eellison commented Dec 17, 2019 •
edited
Loading

quip (which itself is missing some other stuff like the unsupported parts of torch.nn.init)

What stuff is missing from the quip exactly? Also torch.nn.init is all deprecated.

@fmassa
Copy link
Member

fmassa commented Dec 17, 2019

FYI only the non-inplace functions in torch.nn.init are are deprecated

facebook-github-bot
facebook-github-bot reviewed Dec 18, 2019
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@eellison merged this pull request in fb30a48.

@eellison eellison mentioned this pull request Dec 19, 2019
Closed
facebook-github-bot pushed a commit that referenced this pull request Dec 19, 2019
@facebook-github-bot
add back in reference to jit_unsupported section (#31486)
779b128 
Summary:
It was added in #31329 and removed in a bad merge in #31138
Pull Request resolved: #31486

Differential Revision: D19181967

Pulled By: eellison

fbshipit-source-id: 7e4b4a9b2042c30ec18f7f737bc4a9a56fac7d92
wuhuikx pushed a commit to wuhuikx/pytorch that referenced this pull request Jan 30, 2020
@facebook-github-bot
add unsupported section (pytorch#31329)
95839c8 
Summary:
Add a section for unsupported ops, and modules. Automatically generate the properties and attributes that aren't bound, and for ops that have semantic mismatches set up tests so the docs stay up to date.
Pull Request resolved: pytorch#31329

Differential Revision: D19164472

Pulled By: eellison

fbshipit-source-id: 46290bb8a64d9de928cfb1eda5ff4558c3799c88
wuhuikx pushed a commit to wuhuikx/pytorch that referenced this pull request Jan 30, 2020
@facebook-github-bot
add back in reference to jit_unsupported section (pytorch#31486)
998271c 
Summary:
It was added in pytorch#31329 and removed in a bad merge in pytorch#31138
Pull Request resolved: pytorch#31486

Differential Revision: D19181967

Pulled By: eellison

fbshipit-source-id: 7e4b4a9b2042c30ec18f7f737bc4a9a56fac7d92
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@apaszke apaszke Awaiting requested review from apaszke

+1 more reviewer

@driazati driazati driazati approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Merged oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@eellison @kostmo @fmassa @facebook-github-bot @driazati @mruberry