New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于cuda版本的问题 #21

Closed

fn6767 opened this issue Jul 16, 2021 · 9 comments

Labels

fn6767 commented Jul 16, 2021

您好，请问我使用cuda11以上的版本可以训练吗，因为我只有A6000和A100的显卡，它们不兼容cuda11以下的版本。我用cuda11.1和torch1.8或者1.9训练时，都会报double free or corruption (!prev)、RuntimeError: DataLoader worker (pid(s) xxxxx) exited unexpectedly。

Member

wangg12 commented Jul 16, 2021

能用pytorch 1.7.1吗，1.8有问题，1.9我会试试

Member

wangg12 commented Jul 16, 2021

1.8之前我测过确实有问题，pytorch官方有很多类似的issue, 我用1.9跑了下是没问题的，不过我没有A系列的显卡

wangg12 added the help wanted label

Author

fn6767 commented Jul 17, 2021

您用torch1.9跑没问题的cuda是cuda11吗？

Member

wangg12 commented Jul 17, 2021 •

edited

cuda 11.1也可以的

Author

fn6767 commented Jul 17, 2021

我尝试了cuda11.1+torch1.9的配置，会报错malloc(): invalid size (unsorted)

Author

fn6767 commented Jul 17, 2021

或者报malloc(): memory corruption

peppa114 commented Sep 4, 2021

能用pytorch 1.7.1吗，1.8有问题，1.9我会试试
你好，我尝试了pytorch1.7.1+cuda11.0,也是会有RuntimeError: DataLoader worker (pid(s) xxxxx) exited unexpectedly的报错，我用的是RTX3090显卡

Author

fn6767 commented Sep 4, 2021

应该是内存不够，我开始也碰到了这个问题，把batch_size改小或者加大内存

peppa114 commented Sep 4, 2021

应该是内存不够，我开始也碰到了这个问题，把batch_size改小或者加大内存
我的机器内存应该是够的，有125g，之前尝试改了bath_size=12，中间还是会断，目前改成bath_size=2，运行正常，还没有断

wangg12 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment