Add document page for torch.distributed.rpc/autograd

[mrshenli]

distributed_c10d has a nice doc page here, and the source code of that live in here. We should add a similar page for rpc and distributed autograd to describe the API.

@rohan-varma brought up a point that whether we want to add the new docs into the existing distributed.rst file as distributed/distributed_c10d.py, distributed/rpc/ and distributed/autograd all live in the same distributed/ folder, or whether we want to keep them in separate files. I would vote for keeping them separate from the existing one, as the existing one mostly focus on c10d, and the new ones are on rpc/autograd, which are quite different topics. The following items would need more discussion and let's track them here:

1. The title of the existing distributed.rst page says Distributed communication package - torch.distributed, do we want to rename that to Distributed collective communication package - torch.distributed to be more precise?
2. Should rpc and distributed autograd api docs live in the same file?

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @rohan-varma @xush6528

[rohan-varma]

For 2, I think since that rpc and dist autograd interact a lot together and a user would probably use both of them if they were using one, they should live in the same page

[pritamdamania87]

The title of the existing distributed.rst page says Distributed communication package - torch.distributed, do we want to rename that to Distributed collective communication package - torch.distributed to be more precise?

I think we should keep the name, since it's not necessary always collective communication (send/recv). For the new page, maybe we can title it Distributed RPC framework

I agree that we should have a single page for RPC, RRef and dist autograd.

[rohan-varma]

Picking this one up.