Adding an RPC Tutorial

[mrshenli]

No description provided.

[netlify]

Deploy preview for pytorch-tutorials-preview ready!

Built with commit 08acbc3

https://deploy-preview-807--pytorch-tutorials-preview.netlify.com

[rohan-varma]

The tutorials look great! I did a quick pass with some minor comments, and I will try to go through the tutorial + run the code in depth as well.

[mrshenli]

I need to add this to side bar index

[mrshenli]

Hey @jlin27 @SethHWeidman, may I get some early reviews on this tutorial page? Thanks!

[rohan-varma]

General question: There is a model parallel tutorial that is essentially about sharding the model on the same node across many GPUs: https://deploy-preview-807--pytorch-tutorials-preview.netlify.com/intermediate/model_parallel_tutorial.html. Should we update the title or change it accordingly now that we have the ability to do model parallel across multiple nodes?

[rohan-varma]

From running the code for the RL example, I don't think the args are defined? We can add something like this:

parser = argparse.ArgumentParser(
        description="RPC Reinforcement Learning Example",
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
    )

parser.add_argument('--world_size', default=2, help='Number of workers')
parser.add_argument('--log_interval', default=1, help='Log every log_interval episodes')
parser.add_argument('--gamma', default=0.1, help='how much to value future rewards')
parser.add_argument('--seed', default=1, help='random seed for reproducibility')
args = parser.parse_args()

Also, some variables aren't defined:

AGENT_NAME = "agent"
OBSERVER_NAME="obs"
TOTAL_EPISODE_STEP = 100

[mrshenli]

Thanks @rohan-varma!

There is a model parallel tutorial that is essentially about sharding the model on the same node across many GPUs

Yes, let me edit the title and also mention the difference in this tutorial.

From running the code for the RL example, I don't think the args are defined? We can add something like this:

Thanks, those are in the examples, not in the tutorial, let me add them here too. Thank you!