-
Notifications
You must be signed in to change notification settings - Fork 778
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't use one GPU? #47
Comments
We exploit the fact that by default BN is split into multiple GPUs in which mean/std is independently computed. To run on one GPU, you may implement BN split along the N (batch) dimension to mimic this effect (see NaiveBatchNorm in Detectron2). You also need to change the lr (e.g., linearly) if you change the batchsize to fit memory. To run on 2 GPUs, try |
There is an implementation of split (aka sub-batch) BN for video (3d) models that might be helpful to you: This can easily be adapted to the the 2d BN case. |
@rbgirshick Thanks for the link. I actually needed this and am writing a code myself starting from there. BTW, if you know, It would be great if you could answer this question, which is regarding the code snippet. facebookresearch/SlowFast#357 (comment) |
Hello, have you realized this? I also want to implement MOCO on a single GPU. |
@JingLiJJ maybe you can see https://github.com/leftthomas/MoCo |
来信收到,谢谢!
|
Hey! I would really want to run it on a single GPU, but I am not sure if I understand how this implementation works for a single GPU, could you please explain? |
来信收到,谢谢!
|
If I want to use one or two GPUs in one server? What should I do?
The text was updated successfully, but these errors were encountered: