Allow for ACCELERATE_SEED env var (#2126)

* Manual seeds

* None

* Add to docs

* Document

* Use torch seed for simplicity

* Rm from doc

* Better version

docs/source/package_reference/utilities.md

@@ -50,6 +50,12 @@ These are basic dataclasses used throughout 🤗 Accelerate and they can be pass
 
 [[autodoc]] utils.ProjectConfiguration
 
+## Environmental Variables
+
+These are environmental variables that can be enabled for different use cases
+
+* `ACCELERATE_DEBUG_MODE` (`str`): Whether to run accelerate in debug mode. More info available [here](../usage_guides/debug.md).
+
 ## Plugins
 
 These are plugins that can be passed to the [`Accelerator`] object. While they are defined elsewhere in the documentation, 

src/accelerate/data_loader.py

@@ -78,12 +78,15 @@ class SeedableRandomSampler(RandomSampler):
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         self.epoch = 0
+        self.seed = torch.random.initial_seed()
 
     def __iter__(self):
         if self.generator is None:
             self.generator = torch.Generator()
+        else:
+            self.seed = self.generator.initial_seed()
         # Allow `self.epoch` to modify the seed of the generator
-        seed = self.epoch + self.generator.initial_seed()
+        seed = self.epoch + self.seed
         self.generator.manual_seed(seed)
         yield from super().__iter__()
         self.set_epoch(self.epoch + 1)