如何在pytorch-lightning中固定随机种子，并保证实验结果完全复现

[rentainhe]

相关的官方教程：https://pytorch-lightning.readthedocs.io/en/latest/common/trainer.html#reproducibility

Pytorch-Lightning Reproducibility

如果需要保证结果的完全复现，需要以下两个条件

• 随机种子固定
• 在Trainer中将deterministic设置为True

from pytorch_lightning import Trainer, seed_everything

seed_everything(42, workers=True)
# sets seeds for numpy, torch, python.random and PYTHONHASHSEED.
model = Model()
trainer = Trainer(deterministic=True)

Pytorch Reproducibility

固定随机种子为什么需要设置deterministic？

• torch.backends.cudnn.deterministic如果设置为True的话，意味着每次返回的卷积算法都将是确定的。
• torch.backends.cudnn.benchmark如果设置为True的话，意味着在程序启动阶段，会额外花费时间，为整个网络的每个卷积层搜索最合适实现的卷积算法，进而实现网络的加速

所以在pytorch中如果需要设置需要进行以下设置：

torch.manual_seed(seed) # sets the seed for generating random numbers.
torch.cuda.manual_seed(seed) # Sets the seed for generating random numbers for the current GPU. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.
torch.cuda.manual_seed_all(seed) # Sets the seed for generating random numbers on all GPUs. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False