v5.0.4.post1
RLHF:
More vllm engines corresponsed to one training duplicate can be started, max_num_seqs*num_vllm_engines repsponces could be generatted at every inferrence request.
Adjusting arguments max_num_seqs and num_vllm_engines to make training speed of max_num_seqs*num_vllm_engines*num_generations samples approximately equals to inferrence speed of one request will benefite a highest performance.