the necessity of data_dependent_initialize

[Turlan]

Hi, great work! Thanks for releasing codes. I am confused about the data_dependent_initialize step. Could you explain why we need this extra step before the real training starts？

contrastive-unpaired-translation/train.py

Line 43 in 24e03ce

model.data_dependent_initialize()

[QtacierP]

If you do not set this function, you will get some troubles when you use multiple GPUs. The MLP flag will be initialized and fixed when you use the nn.DataParallel() to replace the naive model (nn.Module). Therefore, we need to initialize the MLP before we parallelize the model

[ibro45]

@QtacierP does that imply that there's no other purpose to it except that it's patching the issue with nn.DataParallel() and that if nn.DataParallel() is not used, it'd be safe to remove it?