Difference between micro_train_batch_size & micro_rollout_batch_size

[pratikkumar018]

Can you please explain the difference between micro_train_batch_size & micro_rollout_batch_size. I tried going through the code and found micro_train_batch_size to be used here.
https://github.com/OpenLLMAI/OpenRLHF/blob/main/openrlhf/trainer/ppo_utils/replay_buffer.py#L152
I am not sure if these two values should be same everytime?

[hijkzzz]

micro_train_batch_size is the training batch size per gpu
micro_rollout_batch_size is the inference batch size per gpu for PPO

[pratikkumar018]

So when you say inference, does it mean batch_size during validation time or does it mean inference batch size for initial_model & reward model (frozen models)?

[wuxibin89]

@pratikkumar018 micro_rollout_batch_size is the size for sequence generation and initial_model/reward_model inference afterwards.

[pratikkumar018]

Thanks a lot