Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Doc] Kubeflow integration #937

Merged
merged 5 commits into from
Mar 28, 2023

Conversation

kevin85421
Copy link
Member

@kevin85421 kevin85421 commented Mar 2, 2023

Why are these changes needed?

Use Kubeflow as a solution of interactive prototyping environment (JupyterLab).

architecture

Related issue number

Closes #725
Part of #502. The issue includes JupyterLab, access control, and authentication. This PR only includes JupyterLab.

Checks

  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • Manual tests
    • This PR is not tested :(

Screen Shot 2023-03-01 at 4 53 30 PM

@kevin85421
Copy link
Member Author

@kevin85421 kevin85421 marked this pull request as ready for review March 2, 2023 00:57
Copy link
Contributor

@architkulkarni architkulkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just added some suggestions. I think the most important one is to include some brief intro/motivation at the top, the rest are minor.

docs/guidance/kubeflow-integration.md Show resolved Hide resolved
```

## Step 2: Install Kubeflow v1.6-branch
* This example installs Kubeflow with the [v1.6-branch](https://github.com/kubeflow/manifests/tree/v1.6-branch).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need the 1.6 branch specifically? Would be good to add a brief comment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubeflow is very sensitive to the Kubernetes version and the Kustomize version, so some instructions in this document may not work with other releases. In addition, some components will be renamed in future Kubeflow releases, and I am not familiar with the Kubeflow community's plan.

```
* This step uses `rayproject/ray:2.2.0-py38-cpu` as its image. Ray is very sensitive to the Python versions and Ray versions between the server (RayCluster) and client (JupyterLab) sides. This image uses:
* Python 3.8.13
* Ray 2.2.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we can use 2.3.0 now, but if it will require a lot more manual testing then it might not be worth it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a huge effort. Ray client is very sensitive to the Python versions and Ray versions. Sometimes I use the same Ray commit in the JupyterLab instance and in the RayCluster's docker image, but it is still not compatible.

docs/guidance/kubeflow-integration.md Outdated Show resolved Hide resolved
kevin85421 and others added 5 commits March 27, 2023 23:14
Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Archit Kulkarni <architkulkarni@users.noreply.github.com>
Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org>
@kevin85421 kevin85421 merged commit 1c648a3 into ray-project:master Mar 28, 2023
Copy link
Contributor

@architkulkarni architkulkarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah forgot to mention, if we're adding a new .md file here, do we need to add it to the table of contents YAML file you showed me earlier? Anyways, we can figure this out when we review the docs as a whole later.

@kevin85421
Copy link
Member Author

Ah forgot to mention, if we're adding a new .md file here, do we need to add it to the table of contents YAML file you showed me earlier? Anyways, we can figure this out when we review the docs as a whole later.

Good point. I forget about it. We can reorganize the website later.

lowang-bh pushed a commit to lowang-bh/kuberay that referenced this pull request Sep 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature][Discussion] Interactive development for Ray on Kubernetes
2 participants