Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cuda memory problem #12

Closed
jadhu22 opened this issue Mar 7, 2019 · 4 comments
Closed

cuda memory problem #12

jadhu22 opened this issue Mar 7, 2019 · 4 comments

Comments

@jadhu22
Copy link

jadhu22 commented Mar 7, 2019

RuntimeError: CUDA out of memory. Tried to allocate 52.25 MiB (GPU 0; 7.92 GiB total capacity; 6.71 GiB already allocated; 53.94 MiB free; 35.80 MiB cached)
File "main.py", line 247, in
main()
File "main.py", line 230, in main
result, is_best = iterate("val", args, val_loader, model, None, logger, checkpoint['epoch'])
File "main.py", line 105, in iterate
pred = model(batch_data)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 141, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/jadoo/dense_lidar/self-supervised-depth-completion/model.py", line 119, in forward
conv3 = self.conv3(conv2) # batchsize * ? * 176 * 608
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torchvision/models/resnet.py", line 45, in forward
out = self.conv1(x)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/jadoo/pytorch/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 320, in forward
self.padding, self.dilation, self.groups)

@AbnerCSZ
Copy link

AbnerCSZ commented Mar 7, 2019

I can run in 2 or 4 GTX 1080Ti. I think you should use GPU memory more than 16G.

@fangchangma
Copy link
Owner

RuntimeError: CUDA out of memory

As @AbnerCSZ mentioned, it is due to limited GPU memory. You could try with a simpler model or a smaller image size.

@jadhu22
Copy link
Author

jadhu22 commented Mar 7, 2019

@fangchangma, referring to this part of the code in kitti_loader.py:

note: we will take the center crop of the images during augmentation

# that changes the optical centers, but not focal lengths
K[0,2] = K[0,2] - 13 # from width = 1242 to 1216, with a 13-pixel cut on both sides
K[1,2] = K[1,2] - 11.5 # from width = 375 to 352, with a 11.5-pixel cut on both sides

Do I need to make changes here??
Or shall I directly resize the rgb and depth image??

@fangchangma
Copy link
Owner

Do I need to make changes here??

Yes. If you resize the images, the intrinsics should be adjusted accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants