[Question]: Problem when using multi gpu training for navdp

[zlm646]

Question

Hi:
I countered this issue when using two GPUs to train NavDP, and I noticed that find_unused_parameters is already set to True in train.py, do you have any idea?

Unhandled exception: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument find_unused_parameters=True to torch.nn.parallel.DistributedDataParallel, and by making sure all forward function outputs participate in calculating loss.

ps: The training runs fine when only using one GPU.

Thanks.

[superjuices]

I also encountered the same problem, do you have a solution?