Is it possible to use smaller GPU for inference?

[AmitRozner]

I read about you training 8 images in a batch on P40. Is it possible to use the code with GTX 1080TI (12GB) with smaller batch size?

[lijiannuist]

Hi @AmitRozner
Of course. But Training with smaller batch size may reduce detection performance slightly.

[cqlyiyeshu]

My gpu is V100, 16G, when I run
python demo.py --trained_model weights/WIDERFace_DSFD_RES152.pth --img_root data/worlds-largest-selfie.jpg
It gets error: RuntimeError: CUDA out of memory
Is it possible to use smaller GPU?

[lijiannuist]

Hi @cqlyiyeshu
I think that 16G is enough.
you can try to use less scale in demo.py. especially for 2x.

[jiangziya]

@lijiannuist My gpu is 8G，how can i solve the error "RuntimeError: CUDA out of memory "?
error log is:
traceback (most recent call last):
File "demo.py", line 248, in
test_oneimage()
File "demo.py", line 232, in test_oneimage
det_b = np.row_stack((det_b, infer(net , img , transform , thresh , cuda , bt)))
File "demo.py", line 98, in infer
y = net(x) # forward pass
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/jy/FaceDetection-DSFD/face_ssd.py", line 235, in forward
conv4_3_x = self.layer2(conv3_3_x)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/lib/python2.7/site-packages/torchvision/models/resnet.py", line 85, in forward
out = self.bn3(out)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/lib64/python2.7/site-packages/torch/nn/modules/batchnorm.py", line 66, in forward
exponential_average_factor, self.eps)
File "/usr/lib64/python2.7/site-packages/torch/nn/functional.py", line 1254, in batch_norm
training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: CUDA error: out of memory

[JaywongWang]

Hi @cqlyiyeshu
I think that 16G is enough.
you can try to use less scale in demo.py. especially for 2x.

@lijiannuist Do you mean resizing the input image to a smaller size?

[jiangziya]

Hi @cqlyiyeshu
I think that 16G is enough.
you can try to use less scale in demo.py. especially for 2x.

@lijiannuist Do you mean resizing the input image to a smaller size?

yes，i resize as 100x100

[yihongXU]

Hi, try to add torch.set_grad_enabled(False) inside test_oneimage() function (at the beginning of the function) if your torch version is >= 0.4. It works for me.

[vlad3996]

You can use this https://github.com/vlad3996/FaceDetection-DSFD with original author's checkpoint
or try at least
with torch.no_grad():
to inference

[TekiLi]

My gpu is V100, 16G, when I run
python demo.py --trained_model weights/WIDERFace_DSFD_RES152.pth --img_root data/worlds-largest-selfie.jpg
It gets error: RuntimeError: CUDA out of memory
Is it possible to use smaller GPU?
应该用的是0.4+的torch吧，内存没有释放，用0.3的torch就没问题了

[KarelZhang]

My gpu is V100, 16G, when I run
python demo.py --trained_model weights/WIDERFace_DSFD_RES152.pth --img_root data/worlds-largest-selfie.jpg
It gets error: RuntimeError: CUDA out of memory
Is it possible to use smaller GPU?
应该用的是0.4+的torch吧，内存没有释放，用0.3的torch就没问题了

用0.4+的torch确实会报内存不够，请问有什么方法可以释放内存嘛？我现在只能一张图片一张图片地测。