Doesn't seem to work on Windows

[78Alpha]

Just wondering if this is another Linux only version of Low VRAM DreamBooth. Seems to do what the other implementations do and just quits after 1 step or less. This log was pulled from running the training directly, as accelerate complains that there is no GPU while there in fact is an RTX 3070 present.

I should add that this is Windows Native, no WSL of any kind.

Steps: 0%| | 0/1000 [00:00<?, ?it/s]Traceback (most recent call last): File "H:\Utilities\CUDA\Dreambooth\lora\train_lora_dreambooth.py", line 964, in <module> main(args) File "H:\Utilities\CUDA\Dreambooth\lora\train_lora_dreambooth.py", line 836, in main for step, batch in enumerate(train_dataloader): File "H:\Utilities\CUDA\Dreambooth\dreambooth\lib\site-packages\accelerate\data_loader.py", line 345, in __iter__ dataloader_iter = super().__iter__() File "H:\Utilities\CUDA\Dreambooth\dreambooth\lib\site-packages\torch\utils\data\dataloader.py", line 435, in __iter__ return self._get_iterator() File "H:\Utilities\CUDA\Dreambooth\dreambooth\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator Traceback (most recent call last): return _MultiProcessingDataLoaderIter(self) File "<string>", line 1, in <module> File "H:\Utilities\CUDA\Dreambooth\dreambooth\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in __init__ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main w.start() exitcode = _main(fd, parent_sentinel) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 121, in start File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 126, in _main self._popen = self._Popen(self) self = reduction.pickle.load(from_parent) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 224, in _Popen EOFError: Ran out of input return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 336, in _Popen return Popen(process_obj) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__ reduction.dump(process_obj, to_child) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'main.<locals>.collate_fn'

[2kpr]

I just got the same error as you trying to run it in native Windows.

Seems to be a 'known' issue with python's multiprocessing and windows vs linux, where it works on the latter but not the former:
https://www.google.com/search?q=pickle+multiprocessing+error+python+windows+vs+linux

[78Alpha]

@2kpr yes, native windows. Just wanted to test if this implementation worked natively and it appears not to (out of the box)

[d8ahazard]

Oddly enough, this works fine on windows with accelerate when implemented in my Dreambooth Extension for SD-Webui:

https://github.com/d8ahazard/sd_dreambooth_extension/tree/SD_21

[78Alpha]

Oddly enough, this works fine on windows with accelerate when implemented in my Dreambooth Extension for SD-Webui:

https://github.com/d8ahazard/sd_dreambooth_extension/tree/SD_21

I assumed your version worked, I've kept my eye on it. I was more curious if this implementation worked raw on windows.

[kenjiqq]

You can remove num_workers=1, from the data loader to get past this issue. It will disable the multiprocessing code that isn't working on windows

[2kpr]

You can remove num_workers=1, from the data loader to get past this issue. It will disable the multiprocessing code that isn't working on windows

That worked alright, thanks!

Interesting that the GPU is only consuming about 1/3rd of the normal power/watts when using the LORA example script, and my it/s are lower than the ShivamShrirao/diffusers repo/code I normally use. I can increase the batch size to 4 as it just fits it in 12 GB (my 3080 Ti), will be interesting to see how this LORA pans out in terms of quality.

[cloneofsimo]

Sorry, I didn't test this on window device... Good to see that solutions worked for you. If you get good results please share them!