[BUG]: num_workers>0 hangs on Windows and sometimes MacOS.

[JacksonBurns]

This was originally discovered in #714 - setting num_workers above 0 causes Chemprop to hang on Windows on both GitHub actions and locally, and on MacOS for GitHub actions.

See these two replies to a highly relevant issue on the PyTorch forum - we may need to refactor our calls to train, or just disallow parallel dataloading based on platform:

• https://discuss.pytorch.org/t/errors-when-using-num-workers-0-in-dataloader/97564/5
• https://discuss.pytorch.org/t/errors-when-using-num-workers-0-in-dataloader/97564/4

[davidegraff]

Seems like the default of 8 is a remnant of v1. I don't think it's a bad change to use the torch default, and if we reintroduce caching (#697) then there's really no need to parallelize dataloading. FWIW I think this is due to differences in parallelism implementations across platforms because of python GIL; POSIX uses fork() which is significantly faster to spin up and wind down than spawn() used in Windows/MacOS

[JacksonBurns]

Here's the relevant pytorch docs page as well https://pytorch.org/docs/stable/notes/windows.html#usage-multiprocessing

[donerancl]

This has been addressed

• message that num workers> 0 results in hanging for Windows and MacOS
• default num workers = 0