We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently, vLLM's vllm.worker.worker.Worker is replaced with openrlhf.trainer.ray.vllm_worker_wrap.WorkerWrap on fly as a monkey patch.
vllm.worker.worker.Worker
openrlhf.trainer.ray.vllm_worker_wrap.WorkerWrap
The monkey patch is avoidable by making init_process_group and update_weight global functions and invoke them via __ray_call__.
init_process_group
update_weight
__ray_call__
__ray_call__ has not been documented yet but it is expected to be documented soon since it is marked as P1 in ray-project/ray#45068.
__ray_call__ is already used in Ray to initialize NCCL group, which is similar to the OpenRLHF use case:
https://github.com/ray-project/ray/blob/4c1519be13087f4ccb47431f9ebc3dc446182775/python/ray/experimental/channel/torch_tensor_nccl_channel.py#L275-L282
The text was updated successfully, but these errors were encountered:
Thanks~
Sorry, something went wrong.
No branches or pull requests
Currently, vLLM's
vllm.worker.worker.Worker
is replaced withopenrlhf.trainer.ray.vllm_worker_wrap.WorkerWrap
on fly as a monkey patch.The monkey patch is avoidable by making
init_process_group
andupdate_weight
global functions and invoke them via__ray_call__
.__ray_call__
has not been documented yet but it is expected to be documented soon since it is marked as P1 in ray-project/ray#45068.__ray_call__
is already used in Ray to initialize NCCL group, which is similar to the OpenRLHF use case:https://github.com/ray-project/ray/blob/4c1519be13087f4ccb47431f9ebc3dc446182775/python/ray/experimental/channel/torch_tensor_nccl_channel.py#L275-L282
The text was updated successfully, but these errors were encountered: