Rework compat bindings.

[robieta]

Stack from ghstack:

• Enable callgrind collection for C++ snippets #47865 Enable callgrind collection for C++ snippets
• Add support for timing C++ snippets. #47864 Add support for timing C++ snippets.
• Rework compat bindings. #47863 Rework compat bindings.
• Add option for cpp_extensions to compile standalone executable #47862 Add option for cpp_extensions to compile standalone executable
• Expose CXX_FLAGS through __config__ #47861 Expose CXX_FLAGS through config

Differential Revision: D25199261

[dr-ci]

💊 CI failures summary and remediations

As of commit cf28c05 (more details on the Dr. CI page):

---

• 1/2 failures introduced in this PR
• 1/2 tentatively recognized as flaky ❄️
  • Click here to rerun these jobs

---

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_bazel_build (1/1)

Step: "Bazel Build" (full log | diagnosis details | 🔁 rerun)

Dec 01 18:27:09 FAILED: Build did NOT complete successfully

Dec 01 18:27:09    static char *vparameterStr[7] = { "v", "vv", "", "vv", "v", "vvv", "" }; 
Dec 01 18:27:09                 ^~~~~~~~~~~~~ 
Dec 01 18:27:09 ERROR: missing input file 'external/sleef/src/libm/sleeflibm_header.h.org', owner: '@sleef//:src/libm/sleeflibm_header.h.org' 
Dec 01 18:27:09 ERROR: /var/lib/jenkins/.cache/bazel/_bazel_jenkins/fdf6d09bf4b4f04a71e2a7dfceb40620/external/sleef/BUILD.bazel:178:1: @sleef//:sleef_h: missing input file '@sleef//:src/libm/sleeflibm_header.h.org' 
Dec 01 18:27:09 Target //:torch failed to build 
Dec 01 18:27:09 Use --verbose_failures to see the command lines of failed build steps. 
Dec 01 18:27:09 ERROR: /var/lib/jenkins/.cache/bazel/_bazel_jenkins/fdf6d09bf4b4f04a71e2a7dfceb40620/external/sleef/BUILD.bazel:178:1 1 input file(s) do not exist 
Dec 01 18:27:09 INFO: Elapsed time: 112.353s, Critical Path: 1.40s 
Dec 01 18:27:09 INFO: 196 processes: 196 processwrapper-sandbox. 
Dec 01 18:27:09 FAILED: Build did NOT complete successfully 
Dec 01 18:27:09 FAILED: Build did NOT complete successfully 
Dec 01 18:27:09 + cleanup 
Dec 01 18:27:09 + retcode=1 
Dec 01 18:27:09 + set +x 
Dec 01 18:27:09 =================== sccache compilation log =================== 
Dec 01 18:27:09 ERROR 2020-12-01T18:25:25Z: sccache::server: ["null"] fatal error: Permission denied (os error 13) at path "/dev/.tmpXDOmUG" 
Dec 01 18:27:09  
Dec 01 18:27:09 ERROR 2020-12-01T18:25:25Z: sccache::server: ["null"] 	Permission denied (os error 13) at path "/dev/.tmpXDOmUG" 
Dec 01 18:27:09  
Dec 01 18:27:09 ERROR 2020-12-01T18:25:25Z: sccache::server: ["null"] fatal error: Permission denied (os error 13) at path "/dev/.tmp5JCkDk" 
Dec 01 18:27:09  

---

❄️ 1 failure tentatively classified as flaky

but reruns have not yet been triggered to confirm:

pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun) ❄️

Dec 01 20:37:13 RuntimeError: Process 0 terminated or timed out after 118.06462502479553 seconds

Dec 01 20:37:13 ====================================================================== 
Dec 01 20:37:13 ERROR [118.104s]: test_function_not_on_callee (__main__.TensorPipeRpcTestWithSpawn) 
Dec 01 20:37:13 ---------------------------------------------------------------------- 
Dec 01 20:37:13 Traceback (most recent call last): 
Dec 01 20:37:13   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/common_distributed.py", line 278, in wrapper 
Dec 01 20:37:13     self._join_processes(fn) 
Dec 01 20:37:13   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/common_distributed.py", line 395, in _join_processes 
Dec 01 20:37:13     self._check_return_codes(elapsed_time) 
Dec 01 20:37:13   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/testing/_internal/common_distributed.py", line 436, in _check_return_codes 
Dec 01 20:37:13     raise RuntimeError('Process {} terminated or timed out after {} seconds'.format(i, elapsed_time)) 
Dec 01 20:37:13 RuntimeError: Process 0 terminated or timed out after 118.06462502479553 seconds 
Dec 01 20:37:13  
Dec 01 20:37:13 ---------------------------------------------------------------------- 
Dec 01 20:37:13 Ran 369 tests in 1587.130s 
Dec 01 20:37:13  
Dec 01 20:37:13 FAILED (errors=1, skipped=31) 
Dec 01 20:37:13  
Dec 01 20:37:13 Generating XML reports... 
Dec 01 20:37:13 Generated XML report: test-reports/dist-gloo/TEST-TensorPipeDdpComparisonTestWithSpawn-20201201201046.xml 
Dec 01 20:37:13 Generated XML report: test-reports/dist-gloo/TEST-TensorPipeDdpUnderDistAutogradTestWithSpawn-20201201201046.xml 
Dec 01 20:37:13 Generated XML report: test-reports/dist-gloo/TEST-TensorPipeDistAutogradTestWithSpawn-20201201201046.xml 

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 31 times.

[ezyang]

It would probably be nice to have a Note explaining at a high level what's going on here

[ezyang]

I understand that this is "temporary" stuff but because it interacts nontrivially with some code that isn't in PyTorch itself, it will be harder for other people to figure out how this relates to the bigger picture. A note here will help a lot.

[robieta]

No, it's a valid point. Check out BACK_TESTING_NOTE: and let me know if it seems reasonable.

[ezyang]

Thanks, this note is super helpful. One extra benefit of the note is it also clues the reader in on what kinds of code changes to Timer are permissible, and what are not (changes that add more dependencies on C symbols => you need to add more backtesting support.)

[facebook-github-bot]

@robieta merged this pull request in 17ea112.