It is recommended to use a dedicated device for vLLM

[BartekKruczek]

Hello,

I'm following your's GRPO multi-GPU approach on SLURM environment. I do understand why we would like to use separate GPU card(s) for vLLM deployment, however I've got errors with NPROC_PER_NODE. I was told that its number should be equal to GPU's count which is incorrect with separate cards for vLLM idea. I can use max 4 GPUs per one node.

AssertionError: Colocate mode requires device_count(4) == num_infer_workers(4). Please check if your device count matches NPROC_PER_NODE setting.

Any idea's why?

[hjh0119]

You can refer to the document at https://swift.readthedocs.io/en/latest/Instruction/GRPO.html to choose between colocate mode or async mode.

[BartekKruczek]

Ok, I see it know. Thank you!

Q: Do you perhaps know which is faster? Let's assume dataset for model (visual one) is more than 10k samples?