RuntimeError: you can only change requires_grad flags of leaf variables.

[Bear-kai]

"item.weight.requires_grad = True" of the forward function of class BlockA, BlockB and BlockC will cause the above error. I found it is caused by the use of DataParallel of the model. So how to train with multiple gpus?

[liuyuisanai]

Hi @Bear-kai , this code is a converted version of the original one and just for inference. We implement the parallel training with an internal library, similar to the distributed data parallel in PyTorch.