Skip to content

v5.0.4.post1

Choose a tag to compare

@janelu9 janelu9 released this 18 Aug 08:45
· 65 commits to main since this release

RLHF:
More vllm engines corresponsed to one training duplicate can be started, max_num_seqs*num_vllm_engines repsponces could be generatted at every inferrence request.
Adjusting arguments max_num_seqs and num_vllm_engines to make training speed of max_num_seqs*num_vllm_engines*num_generations samples approximately equals to inferrence speed of one request will benefite a highest performance.