New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues with Dataloader in getting started tutorial #607
Comments
Summary: Classy's default dataloader currently doesn't work when `num_workers` is set to 0. This is extremely useful for debugging dataloader issues like in facebookresearch#597 and facebookresearch#607. Note that calling `set_dataloader_mp_context(None)` doesn't work since that just sets the mp context to the default value for the environment. If `num_workers` is set to 0, the default call to `Dataloader` now sets `multiprocessing_context` to `None` so that PyTorch doesn't raise an exception. Differential Revision: D23310512 fbshipit-source-id: 8a66a51d7c05c781783f73eda1ee97aa9398e6c9
@lcskrishna can you share some details about the environment you are on? The questionnaire we ask while creating a new bug report would be really helpful! In the mean time, can you try both of the following (independently) -
|
Summary: Pull Request resolved: #608 Classy's default dataloader currently doesn't work when `num_workers` is set to 0. This is extremely useful for debugging dataloader issues like in #597 and #607. Note that calling `set_dataloader_mp_context(None)` doesn't work since that just sets the mp context to the default value for the environment. If `num_workers` is set to 0, the default call to `Dataloader` now sets `multiprocessing_context` to `None` so that PyTorch doesn't raise an exception. Reviewed By: vreis Differential Revision: D23310512 fbshipit-source-id: f4fa6766855446d2c14db7b7054f0e6bc6233bbe
@mannatsingh Thanks for the quick fix for num_workers=0. I am trying to run ClassyVision on ROCm with AMD GPUs.
|
@lcskrishna I see! It'll be great to see the how Classy works on AMD GPUs - it's something we haven't tried ourselves. Looking at pytorch/pytorch#5858 and https://docs.python.org/3/library/multiprocessing.html#multiprocessing.freeze_support, it seems like you are running this on Windows? If that's the case, you are in uncharted territory since Classy was written with only linux support in mind. Classy should work seamlessly even with a "fork" context though on linux. |
@mannatsingh Thanks for the links to the issue and the documentation. I ran the experiments on Linux., and confirming that "fork" option works fine. |
Sounds good. There are environments where spawn and forkserver don't work, but that is a more general issue unrelated to Classy Vision. Closing this out since we have a resolution. |
馃悰 Bug
Getting the following error, when running part (6) of the Getting Started tutorial.
To Reproduce
Steps to reproduce the behavior:
Running part (6) of the getting started tutorial.
Here is the error stack that I got it.
I tried setting the num_workers=0, but i get an issue saying use num_workers>0.
Please let us know a way forward for this issue.
cc: @sunway513
The text was updated successfully, but these errors were encountered: