Completely remove FutureMessage from RRef Implementations

[mrshenli]

Stack from ghstack:

• Apply clang-format to rpc cpp files #50236 Apply clang-format to rpc cpp files
• Completely remove FutureMessage type #50029 Completely remove FutureMessage type
• Completely Remove FutureMessage from RPC agents #50028 Completely Remove FutureMessage from RPC agents
• Completely Remove FutureMessage from RPC cpp tests #50027 Completely Remove FutureMessage from RPC cpp tests
• Remove FutureMessage from RPC request callback logic #50026 Remove FutureMessage from RPC request callback logic
• Completely remove FutureMessage from FaultyProcessGroupAgent #50025 Completely remove FutureMessage from FaultyProcessGroupAgent
• Remove FutureMessage from sender TensorPipeAgent #50024 Remove FutureMessage from sender TensorPipeAgent
• Remove FutureMessage from sender ProcessGroupAgent #50023 Remove FutureMessage from sender ProcessGroupAgent
• Completely remove FutureMessage from distributed autograd #50020 Completely remove FutureMessage from distributed autograd
• Completely remove FutureMessage from RPC TorchScript implementations #50005 Completely remove FutureMessage from RPC TorchScript implementations
• Completely remove FutureMessage from RRef Implementations #50004 Completely remove FutureMessage from RRef Implementations
• Replace FutureMessage with ivalue::Future in RpcAgent retry logic #49995 Replace FutureMessage with ivalue::Future in RpcAgent retry logic
• Replace FutureMessage with ivalue::Future in RRefContext #49960 Replace FutureMessage with ivalue::Future in RRefContext
• Replace FutureMessage with ivalue::Future in distributed/autograd/utils.* #49927 Replace FutureMessage with ivalue::Future in distributed/autograd/utils.*
• Let RpcAgent::send() return JitFuture #49906 Let RpcAgent::send() return JitFuture

Differential Revision: D25750602

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 748210f (more details on the Dr. CI page):

---

• 3/3 failures possibly* introduced in this PR
  • 1/3 non-CircleCI failure(s)

---

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_bionic_py3_8_gcc9_coverage_test1 (1/2)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 07 01:00:26 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future

Jan 07 01:00:26 At:
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 01:00:26 
Jan 07 01:00:26 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Jan 07 01:00:26 
Jan 07 01:00:26 At:
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 01:00:26 
Jan 07 01:00:26 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Jan 07 01:00:26 
Jan 07 01:00:26 At:
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 01:00:26   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 01:00:26 
Jan 07 01:00:26 [W tensorpipe_agent.cpp:547] RPC agent for worker1 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 01:00:26 [W tensorpipe_agent.cpp:547] RPC agent for worker2 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 01:00:27 ok (2.658s)
Jan 07 01:00:29   test_return_future_remote (__main__.TensorPipeRpcTestWithSpawn) ... [W tensorpipe_agent.cpp:547] RPC agent for worker2 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 01:00:29 [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)

pytorch_windows_vs2019_py36_cuda10.1_test1 (2/2)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

RuntimeError: test_nn failed!

  test_ReflectionPad1d (__main__.TestNN) ... ok (0.108s)
  test_ReflectionPad1d_alert_nondeterministic_cuda (__main__.TestNN) ... ok (0.007s)
  test_ReflectionPad1d_cuda (__main__.TestNN) ... ok (0.020s)
  test_ReflectionPad2d (__main__.TestNN) ... ok (1.118s)
  test_ReflectionPad2d_alert_nondeterministic_cuda (__main__.TestNN) ... ok (0.014s)
  test_ReflectionPad2d_cuda (__main__.TestNN) ... Traceback (most recent call last):
  File "run_test.py", line 910, in <module>
    main()
  File "run_test.py", line 889, in main
    raise RuntimeError(err_message)
RuntimeError: test_nn failed!

(base) circleci@PACKER-5FD865C5 C:\Users\circleci\project\test>if ERRORLEVEL 1 exit /b 1 
+ cleanup
+ retcode=1
+ set +x


Exited with code exit status 1

---

1 job timed out:

• pytorch_linux_bionic_py3_8_gcc9_coverage_test1

---

ci.pytorch.org: 1 failed

• Failed: pr/pytorch-linux-bionic-rocm3.10-py3.6

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 30 times.

[facebook-github-bot]

@mrshenli merged this pull request in 2d5f57c.