[Question] The logic of data sampler in data parallel.

[kxzxvbk]

Hi, thanks for your attention.

When reading the source code of transformers, I cannot understand the implementation of _get_train_sampler in trainer.py. Why the default data sampler is RandomSampler rather than DistributedSampler? How does the trainer handle the sampler for data parallel?

reference code: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L975

[kxzxvbk]

And I notice that in very earlier versions. There are some if cases to check whether the world size > 1 and use a different sampler. Why this can be removed in the newer version?
This change is introduced in this commit: ebd94b0

[Rocketknight1]

Hi @kxzxvbk, this isn't really a bug, but we have a community support space for asking questions like these and getting answers from other users or Hugging Face engineers! https://huggingface.co/spaces/transformers-community/support

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.