Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GPU memory out when training with BATCH_SIZE=8 in SECOND_STAGE #542

Closed
dapsjj opened this issue Jun 4, 2020 · 4 comments
Closed

GPU memory out when training with BATCH_SIZE=8 in SECOND_STAGE #542

dapsjj opened this issue Jun 4, 2020 · 4 comments

Comments

@dapsjj
Copy link

dapsjj commented Jun 4, 2020 •
edited

I used BATCH_SIZE=8 in FISRT_STAGE_EPOCHS for training,there is no problem,but when I used BATCH_SIZE=8 in SECOND_STAGE_EPOCHS for training,the error message is memory out.
My GPU is RTX 2070,where is wrong?
@dapsjj dapsjj changed the title Memory out when setting BATCH_SIZE=8 in SECOND_STAGE_EPOCHS Memory out when training with BATCH_SIZE=8 in SECOND_STAGE Jun 4, 2020
@dapsjj dapsjj changed the title Memory out when training with BATCH_SIZE=8 in SECOND_STAGE GPU memory out when training with BATCH_SIZE=8 in SECOND_STAGE Jun 4, 2020
@xinghuokang
Copy link

GPU内存太小了，我用的TitanXp 12G，都没法用batch=8进行第二阶段训练

@dapsjj
Copy link
Author

dapsjj commented Jun 5, 2020

GPU内存太小了，我用的TitanXp 12G，都没法用batch=8进行第二阶段训练

OK，thank you for your reply.

@DongChen06
Copy link

I used batch_size = 6 and 2080 Ti. It shows out of memory!! How to solve it?

@leo-XUKANG
Copy link

@Derekabc set batch_size=2 or 4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dapsjj @DongChen06 @leo-XUKANG @YunYang1994 @xinghuokang