How to run python library module as a script: "python -m"

[JooPoo]

Hi,

I would like to use torch.distributed.launch to launch my script. For example, the following script from github:maskrcnn-benchmark:

python -m torch.distributed.launch --nproc_per_node=4 /path_to_maskrcnn_benchmark/tools/train_net.py --config-file "path/to/config/file.yaml"

script_params can only pass parameters to train_net.py, but how can I pass "-m torch.distributed.launch --nproc_per_node=4" to python?

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 6f52dfdc-ba99-18da-ada5-8e58ed22f243
• Version Independent ID: 3c31cb06-b8af-8426-371e-e7e2610019a0
• Content: azureml.train.dnn.PyTorch class - Azure Machine Learning Python
• Content Source: AzureML/docs-ref-autogen/azureml-train-core/azureml.train.dnn.PyTorch.yml
• Service: machine-learning
• Sub-service: core
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

[Mrxiaoyuer]

Yeah, I have the same problem here. Do you find a walkaround for that @JooPoo ?
Thanks!

[JooPoo]

Not yet. But I have heard that AML is developing the feature of supporting generic executable script.

[yanrez]

@danielsc do you know the way?

[Mrxiaoyuer]

Hi, I finally find a walk-around for it!

The torch.distributed.lauch is just an entry program to load multiple threads, which you can find the source code here: https://github.com/pytorch/pytorch/blob/master/torch/distributed/launch.py

Its key function is to load the args.training_scripts in the line 230-233. https://github.com/pytorch/pytorch/blob/efc5306ad20e8f002fcbc1c85f385c6581f9c213/torch/distributed/launch.py#L232.

Therefore, to use this feature on Azure, you need to

1. Download the lauch.py and set the args.training_scripts as your entry program;
   For example, change line 186 in lauch.py https://github.com/pytorch/pytorch/blob/efc5306ad20e8f002fcbc1c85f385c6581f9c213/torch/distributed/launch.py#L186 to be
   parser.add_argument("--training_script", type=str, default="./tools/train_net.py", help="your entry program.").
2. Then in the Azure program, change the entry_script from your original one to the local lauch.py:

An example:

script_params = {
    '--training_script': './tools/train_net.py',     <--------------- Your original entry program. 
}

est = PyTorch(source_directory="./maskrcnn-benchmark",
                script_params=script_params,
                compute_target=compute_target,
                entry_script = "./tools/lauch.py",    <----------------- Set the Azure entry to be lauch.py. 
                image_registry_details = my_registry,
                custom_docker_image='maskrcnn-benchmark:latest',
                shm_size = '16G',
                use_gpu = True,
                user_managed = True)

In this way, you can achieve a similar function as "python -m torch.distributed.launch script.py".

The problem is that you might need to change the parser in your local lauch.py to pass new parameters. For example, I also add several common Azure parameters in the local lauch.py as following:

parser.add_argument("--input-dir", type=str, default=None,
                    help="The full path of input dir")
parser.add_argument("--dataset", type=str, default='',
                    help="The full path of dataset dir")
parser.add_argument("--output_dir", type=str, default='./outputs',
                    help="The full output input dir")
parser.add_argument("--lr", type=float, default=0.01, help="learning rate", )