Replace FutureMessage with ivalue::Future in RpcAgent retry logic

[mrshenli]

Stack from ghstack:

• Apply clang-format to rpc cpp files #50236 Apply clang-format to rpc cpp files
• Completely remove FutureMessage type #50029 Completely remove FutureMessage type
• Completely Remove FutureMessage from RPC agents #50028 Completely Remove FutureMessage from RPC agents
• Completely Remove FutureMessage from RPC cpp tests #50027 Completely Remove FutureMessage from RPC cpp tests
• Remove FutureMessage from RPC request callback logic #50026 Remove FutureMessage from RPC request callback logic
• Completely remove FutureMessage from FaultyProcessGroupAgent #50025 Completely remove FutureMessage from FaultyProcessGroupAgent
• Remove FutureMessage from sender TensorPipeAgent #50024 Remove FutureMessage from sender TensorPipeAgent
• Remove FutureMessage from sender ProcessGroupAgent #50023 Remove FutureMessage from sender ProcessGroupAgent
• Completely remove FutureMessage from distributed autograd #50020 Completely remove FutureMessage from distributed autograd
• Completely remove FutureMessage from RPC TorchScript implementations #50005 Completely remove FutureMessage from RPC TorchScript implementations
• Completely remove FutureMessage from RRef Implementations #50004 Completely remove FutureMessage from RRef Implementations
• Replace FutureMessage with ivalue::Future in RpcAgent retry logic #49995 Replace FutureMessage with ivalue::Future in RpcAgent retry logic
• Replace FutureMessage with ivalue::Future in RRefContext #49960 Replace FutureMessage with ivalue::Future in RRefContext
• Replace FutureMessage with ivalue::Future in distributed/autograd/utils.* #49927 Replace FutureMessage with ivalue::Future in distributed/autograd/utils.*
• Let RpcAgent::send() return JitFuture #49906 Let RpcAgent::send() return JitFuture

Differential Revision: D25745301

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit fac5a65 (more details on the Dr. CI page):

---

• 3/3 failures possibly* introduced in this PR
  • 2/3 non-CircleCI failure(s)

---

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_bionic_py3_8_gcc9_coverage_test1 (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jan 07 00:59:54 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future

Jan 07 00:59:54 At:
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 00:59:54 
Jan 07 00:59:54 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Jan 07 00:59:54 
Jan 07 00:59:54 At:
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 00:59:54 
Jan 07 00:59:54 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Jan 07 00:59:54 
Jan 07 00:59:54 At:
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Jan 07 00:59:54   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Jan 07 00:59:54 
Jan 07 00:59:54 [W tensorpipe_agent.cpp:547] RPC agent for worker2 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 00:59:54 [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 00:59:55 ok (2.767s)
Jan 07 00:59:56   test_return_future_remote (__main__.TensorPipeRpcTestWithSpawn) ... [W tensorpipe_agent.cpp:547] RPC agent for worker2 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)
Jan 07 00:59:57 [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker0: EOF: end of file (this is expected to happen during shutdown)

---

1 job timed out:

• pytorch_linux_bionic_py3_8_gcc9_coverage_test1

---

ci.pytorch.org: 1 failed

• Failed: pr/caffe2-pytorch-linux-bionic-rocm3.10-py3.6-test

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 68 times.

[mrshenli]

This PR has internal changes

[facebook-github-bot]

@mrshenli merged this pull request in d730c7e.