How to use distributed trainning?

[haoyan14]

Why when I use distributed training, I get stuck here in "model = torch.nn.parallel.DistributedDataParallel(model.to(local_rank), device_ids=[local_rank], output_device=local_rank, find_unused_parameters=False)"?

[boheumd]

Hello, I did not come across this problem before. Can you provide more information about this issue? If you change to the single gpu training, will it have the similar problem?

[haoyan14]

Sorry,i have solved the problem. The reason for this situation is that I mixed the use of "torchrun" and "-d".