-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MultiGPU #29
Comments
I haven't explored Tensorflow on multi-GPU currently. |
I add one GPU at the line "os.environ['CUDA_VISIBLE_DEVICES'] = '1,3'" in the main.py and the code can run on these two GPUs. @zhoufengbuaa |
I do not think so. For multiple GPU, you have to compute average gradient and batch normalization. It is very difficult. For easy, just compute average gradient and it will work. See the example of mnist dataset |
@myhooo os.environ['CUDA_VISIBLE_DEVICES'] = '1,3' it is absolutely not ok, the gpu1 and gpu3 are allocated, but only the gpu1 is used for network. |
@zhengyang-wang It is very important to use large batch when semantic segmentation. Multi-gpu is absolutely a good chiose. |
@zhoufengbuaa Thank you for telling me that I am wrong~ ^_^ |
@zhoufengbuaa I'm aware of that. However, there is an easy way as suggested by @John1231983, which is to use accumulated gradients. A similar way is used in the implementation of msc training. You can read my code to figure out how to do it. This approach allows you to use a large batch of larger patches, but it takes longer time to train. |
I thinl gradient is one one problem of multiple gpu. The another is syn. batch norm statistic that is not support in tensorflow now |
How to run your code on multi-GPU? Thank you very much.
The text was updated successfully, but these errors were encountered: