New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cuda out of memory #4

Closed

qgh1223 opened this issue Feb 26, 2020 · 1 comment

qgh1223 commented Feb 26, 2020

I met cuda out of memory when using 2080 with 11G memory.Is there any way to solve this problem?

Owner

huanglianghua commented Feb 27, 2020

Hi, as a reference, we executed our experiments on 4 TitanX GPUs with 12G memory each, and the training costs around 8~10G memory per GPU.

If the out-of-memory error still occurs, you can use a smaller max_instances in configs/qg_rcnn_r50_fpn.py, e.g., from 8 to 6.

Alternatively, you can also add a --fp16 flag in your training script, which will however slightly reduce the performance by ~1%.

huanglianghua closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment