-
Notifications
You must be signed in to change notification settings - Fork 364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add troubleshooting guide for ray version mismatch #154
Conversation
|
||
## Solution | ||
|
||
In above cases, you need to check if the client ray version is exactly the same with the images version in the ray cluster's configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, after I check, the version check only goes to minor version, matching minor version is enough and image and python patch version difference is tolerant. I will make some complement before push a amend commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. It only check minor version level. Great guidance!
``` | ||
Traceback (most recent call last): | ||
... | ||
RuntimeError: Version mismatch: The cluster was started with: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's another case that even you use same ray version. if the python minor version doesn't match, it would fail as well. I will merge this one as "add trobleshooting for ray
version mismatch" and please give it a try when you get a chance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Co-authored-by: chenyu.jiang <chenyu.jiang@bytedance.com>
Merge branch add_pod_lables of git@gitlab.alipay-inc.com:Arc/kuberay.git into master https://code.alipay.com/Arc/kuberay/pull_requests/4?tab=diff Signed-off-by: 壮酱 <zhengchicheng.zcc@antgroup.com> * add operator config * add vendor * folder name change to kuberay-operator (#143) * Add CRD verb permission in helm (#144) * add crd verb permission in helm * fix ray cluster volume * Update helm-chart/kuberay-operator/README.md Co-authored-by: Oliver Mannion <125105+tekumara@users.noreply.github.com> Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com> Co-authored-by: Oliver Mannion <125105+tekumara@users.noreply.github.com> * Explanation and Best Practice for workers-head Reconnection (#142) * Add Explaination and Best Practice for workers-head Reconnection * Fixed formatting issue * Minor updates * updated to permlink and remove one empty line * minor fix Co-authored-by: Lin Ma <lin.ma1@bytedance.com> * refactor: rename kubray -> kuberay (#145) * docs: fix typo in README.md * refactor: rename kubray -> kuberay * Add nightly docker build workflow (#141) * no --all-tags for nightly build add nightly docker build workflow add nightly docker build workflow remove kubebuilder update to workflow * change docker build tag to repo revision * add more debug info for goimport issue (#151) * add more debug info for bug-150: goimport issue * update goimport failure message * Support enableIngress for RayCluster (#38) * Support enableIngress for RayCluster Add ingress resources in role Copy more configurations from cluster annotation Add ingress example Update ingress version from v1beta1 to v1 * Update to expose dashboard only * Add troubleshooting guide for ray version mismatch (#154) Co-authored-by: chenyu.jiang <chenyu.jiang@bytedance.com> * Enable gofmt and move goimports to linter job (#158) * Enable gofmt and move related work to linter job 1. Introduce gofmt github action 2. Move goimports from build job to lint job 3. Run gofmt -s -w to simplify test codes and fix all lint issues in apiserver and cli projects * Fix lint errors in apiserver * Fix lint errors in cli * Fix goimports error * Setup ci for apiserver (#162) * first release * add vendor * PullRequest: 2 add aci Merge branch aci of git@gitlab.alipay-inc.com:Arc/kuberay.git into master https://code.alipay.com/Arc/kuberay/pull_requests/2 Signed-off-by: 五花 <wuhua.ck@antgroup.com> * add aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * fix aci * test aci * test * test * fix comment * Add KubeRay release guideline (#161) * add flag watchNamespace (#165) Co-authored-by: chen kang <kongchen28@gmail.com> * [Feature]Add subcommand `--version` (#166) * add subcommand --version * fix * lint Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com> * Leader election need lease permission (#169) Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com> * [CLI] check viper error (#172) * check file exist before write * fix Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com> * fix cli typo (#173) Co-authored-by: wuhua.ck <wuhua.ck@alibaba-inc.com> * add vendor * fix * fix
Co-authored-by: chenyu.jiang <chenyu.jiang@bytedance.com>
Why are these changes needed?
A troubleshooting guide for mismatch python version.
Related issue number
#148
Checks