-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support latest Ray 2.10 release #158
Conversation
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. can you please merge master?
serve.run( | ||
deployment, | ||
_blocking=True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we pass keep_serve_terminal
and set it to the new blocking
parameter in serve.run? Better to also rename keep_serve_terminal
to blocking
in our script which will be more consistent and easy to understand.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep current logic for this PR. We printed something after this call.
Use a separate PR to address the parameter, In the new version, user should use Ctrl-C to stop the program after this blocking call.
Do we decide to update to Ray 2.10? it's a breaking change that lower versions will not work. |
Yes, let's upgrade. Can you help create a branch in case if someone want to use Ray 2.9 or earlier versions, they can use that branch? |
Created: https://github.com/intel/llm-on-ray/tree/llm-on-ray_for_ray_2.9 |
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Merged this. thanks! |
* update * fix blocking * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * fix setup and getting started Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * nit Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Add dependencies for tests and update pyproject.toml Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and test workflow Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and fix torch_dist.py Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update OpenAI SDK installation and start ray cluster Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> --------- Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
* update * fix blocking * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * fix setup and getting started Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * nit Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Add dependencies for tests and update pyproject.toml Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and test workflow Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and fix torch_dist.py Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update OpenAI SDK installation and start ray cluster Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> --------- Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
* update * fix blocking * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * fix setup and getting started Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * nit Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Add dependencies for tests and update pyproject.toml Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and test workflow Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and fix torch_dist.py Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update OpenAI SDK installation and start ray cluster Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> --------- Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
* update * fix blocking * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * fix setup and getting started Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * nit Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Add dependencies for tests and update pyproject.toml Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and test workflow Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and fix torch_dist.py Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update OpenAI SDK installation and start ray cluster Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> --------- Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
* update * fix blocking * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * fix setup and getting started Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * update Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * nit Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Add dependencies for tests and update pyproject.toml Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and test workflow Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update dependencies and fix torch_dist.py Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> * Update OpenAI SDK installation and start ray cluster Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com> --------- Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
No description provided.