How to do contrastive learning with accumulate_grad_batches? #19132
Unanswered
yipliu
asked this question in
DDP / multi-GPU / multi-node
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have found an excellent solution for contrastive learning with DDP. However, a more difficult scenario is to use
accumulate_grad_batchesandDDPfor contrastive learning.For a clearer discussion, suppose I am training with
accumulate_grad_batches=4with two GPUs.Anyone can help me?
Beta Was this translation helpful? Give feedback.
All reactions