Data loader will affect the random mechanism even with fixed seed

[zhujiangang]

🐛 Bug

Data loader will affect the random mechanism even with fixed seed, which will make the code unable to reproduce the results.

To Reproduce

Steps to reproduce the behavior:

1. Create a training data loader A
2. Train a pytorch model with the training data loader and monitor the loss
3. Create another data loader B, but don't use it in the training process. Rerun the training code with data loader A, and monitor the loss. The loss will be different with Step 2.

It is a little bit wierd to me, I don't know whether it is a bug or just by design.

[ssnl]

Creation of DataLoader should not advance the default RNG. However, creating an iterator of it will.

[zhujiangang]

Creation of DataLoader should not advance the default RNG. However, creating an iterator of it will.

Thanks for the quick reply. Then did you mean that if the validation data is not loaded by DataLoader but something iterable implemented by myself, the training loss (with only One training DataLoader) will be different from the model that the validation data is loaded by the DataLoader (Two DataLoader, one for training, another for validation)? If so, what is the insight behind such design?

[ngimel]

Please ask questions in discuss.pytorch.org. As @ssnl says, the dataloaders are using random numbers and thus change the state of rng generators, that's expected.