During training, why is optimizer step called after 32 iterations?

[karthik1145]

Referring to this part of code:

if cur_global_batch_size >= cfg.GLOBAL_BATCH_SIZE: # Perform the backward pass. optimizer.zero_grad() loss.backward() # Update the parameters. optimizer.step() else: if cur_iter == 0: optimizer.zero_grad() loss.backward() if (cur_iter + 1) % num_iters == 0: for p in model.parameters(): p.grad /= num_iters optimizer.step() optimizer.zero_grad()

Is it due to batchsize being too small?