New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Training Error #7
Comments
Hi, jainnipun I guess that error occurred in this line |
Thanks! Resolved the "fixed_path" issue, but now it's giving CUDA out of memory error: CUDA out of memory. Tried to allocate 314.00 MiB (GPU 0; 15.78 GiB total capacity; 13.97 GiB already allocated; 240.75 MiB free; 14.33 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF What to do? Should I decrease the batch size? |
Yup, reduce the batch size to fit your GPU VRAM first. |
Yes! Training started after I reduced the "batch_size" from 36 to 15. Thanks. |
Hey! I unzipped the images in the suggested path, but still I keep getting the:
FileNotFoundError: [Errno 2] No such file or directory: '/home/data_storage/mimic-cxr/dataset/image_preprocessing/re_512_3ch/Train/s50328096.jpg'
Can you elaborate why this error is coming?
Thanks.
The text was updated successfully, but these errors were encountered: