Kubeflow Notebooks 2.0 (Kubeflow Workspaces)

[thesuperzapper]

/kind feature

Here is my proposal for Kubeflow Notebooks 2.0, which can be called "Kubeflow Workspaces".

Overview

My main idea has always been to make the Notebook CRD not just a wrapper around PodSpec, with the goal of abstracting away the Kubernetes resources from end users, while also giving cluster admins the ability to define a set of "templates" that end users can choose from.

• Design Google Doc
• Reference GitHub Repo

Implementation

The high-level overview is to split the Notebook CRD into these two CRDs:

• Workspace (namespaced resource)
  • this is the resource that end-users create via the "Workspace Spawner UI" or kubectl
  • it is NOT a wrapper around PodSpec
• WorkspaceKind (cluster resource)
  • this is the resource that cluster-admins create
  • it specifies the template for a Workspace (e.g. "JupyterLab", "VSCode", "RStudio")
  • initially, we would only support a "podTemplate" kind, which is very similar to the existing Notebook CRD,
    but in the future, we could support other types of templates (e.g. "helmTemplate")

New Components

• a new "Workspace Controller":
  • Will manage and reconcile the new CRDs
  • Will provide webhooks (for validation of CRD patches)
  • Will be written in GoLang with Kubebuilder
• a new "Workspace spawner UI":
  • Will lets users overview the workspaces in a namespace.
  • Will lets users spawn, edit and connect to Workspaces
  • Frontend will be written in JS (Angular OR React)
  • Backend will be written in Python OR Golang

New CRD: Workspace

Here is an example Workspace (a namespaced resource), which uses the WorkspaceKind/jupyter-lab that we define later:

https://github.com/thesuperzapper/kubeflow-notebooks-v2-design/blob/main/crds/workspace.yaml

New CRD: WorkspaceKind

Here is an example WorkspaceKind (a cluster resource), which allows users to run JupyterLab Notebooks in Pods:

https://github.com/thesuperzapper/kubeflow-notebooks-v2-design/blob/main/crds/workspace-kind.yaml

Motivation

The main benefits of this approach are:

• end-users can create Workspaces without needing to know anything about Kubernetes, because the UI literally becomes 3 dropdowns and 2 volume mounts:
  • Choose a WorkspaceKind (e.g. "jupyter-lab", "vs-code", "rstudio")
  • Choose an image from the approved list
  • Choose a pod-config (e.g. "small_cpu", "big_gpu")
  • Create/Mount a home volume (optional)
  • Create/Mount data volumes (optional)
• cluster-admins can update the definitions of WorkspaceKinds without breaking existing Workspaces:
  • they simply add a new image/pod-config option, and redirect the old one to the new one
  • we can even provide a config to make the controller wait for a Workspace to restart before applying the redirect
    (we can also make the Spawner UI display a warning that the Workspace needs to be restarted to get the new config)
• scheduling Pods with GPU resources often requires a number of Pod configs to be correctly aligned (e.g. tolerating a taint, and setting a resource limit), collecting all these configs into a single "pod-config" allows cluster-admins to provide drop-down options, rather than requiring users to understand the structure of their Kubernetes cluster.
• resizing an existing notebook is effectively impossible right now, because all the nice "spawner configs" are only available at the time you create the Notebook, once it's spawned you have a PodSpec.
  • With WorkspaceKinds, we can make an "edit" button on the UI which allows you to pick from the current "pod-configs" and "images" available for the WorkspaceKind.

[thesuperzapper]

@kubeflow/wg-notebooks-leads I would appreciate any feedback or ideas on this proposal.

[thesuperzapper]

@kimwnasptd we (that is, @juliusvonkohout, @tzstoyanov, and me) discussed this in the NB meeting today and are thinking that we can target this for Kubeflow 1.9.

I would love to get your general feedback on this proposal, and if you think it looks ok, then we can discuss some specifics and get a finalized design doc.

I think the general approach would be that we keep "kubeflow notebooks" as a "deprecated" component for 1.9, and add a new component called "kubeflow workspaces", which will have:

1. A new controller for the Workspace and WorkspaceKind CRDS
2. A new UI to manage spawning/editing/pausing/etc them
3. A new "component" section on the docs website

[kimwnasptd]

@thesuperzapper @juliusvonkohout @tzstoyanov I'm good with the way of implementing this as a new CRD, and web app since this will be the least intrusive implementation. Users will be able to pick and choose which one they'll want, until we are fully confident on using Workspaces by default.

What I'd like to confirm before committing to this effort would be what are the criteria for for considering this as the default?
* Have culling?
* Have a web app that allows us to have the same functionality as the current one?

[juliusvonkohout]

@mishraprafful and #6734

[kabartay]

Thanks for this new changes. Looks good.

[thesuperzapper]

Calling anyone who wants to help move this feature forward!

Please reply if you are willing/able to help with any of the following areas:

• Frontend (JS - Angular) (Python - Flask):
  • We will need to add 2 new crud-web-apps, to manage and view the new resources.
    • The Workspace app will behave similarly to the existing Notebooks one, and will mostly be about spawning and managing the state of Workspace instances in each profile namespace.
    • The WorkspaceKind app can start with the goal of only showing the current WorkspaceKinds registered in the cluster, and in the future can allow admins to change them graphically.
• Backend (Golang - Kubebuilder) (Istio):
  • We will need to build a new controller for the Workspace and WorkspaceKind resources.
    • We should use Kubebuilder like we already do in the notebook-controller
    • The Workspace controller will also need to create the Istio VirtualServices/etc to enable access to the notebook web interfaces.
  • We will also probably need to create Webhooks to achieve some of what we are doing, which can also be created using Kubebuilder.
• Docs (Markdown and Images):
  • Docs are just as important as the code! Otherwise, people won't know about the awesome new features!
  • We will need to add a new "component" section of the Kubeflow website for "Kubeflow Workspace".

Once we have a few people willing to help, we can organize a specific call to flesh out a doc with:

• User Stories
• Controller Architecture/Behaviour
• UI Mockups
• Alternatives / Problems

---

@mishraprafful and @guohaoyu110 mentioned that they might be interested in contributing to this, also, there are some others who might be like @punkerpunker @helloericsf @wjhhuizi @wadhah101

[guohaoyu110]

• Backend (Golang - Kubebuilder) (Istio)

I am interested in this part: Backend (Golang - Kubebuilder) (Istio)

[thesuperzapper]

Also, @vvnpan @apo-ger may be interested in helping this effort, given they have raised large PRs on the old Notebooks.

[mishraprafful]

I am interested to take up any of these following these areas:

• Frontend (Python - Flask)
• Backend (Golang - Kubebuilder) (Istio)
• Docs (Markdown and Images)

[SachinVarghese]

I'm happy to collaborate on creating the backend+docs (workspace controller) for this new proposal

[vikas-saxena02]

I am happy to contribute to the documentation if you guys need help with that.

[thesuperzapper]

Hey all, now that Kubeflow 1.8 is out, we are going to start seriously working on Kubeflow Notebooks 2.0!

The first step will be to organize and regular meetings with the people who want to contribute and/or give feedback.

I think the first few meetings will be to finalize the design and requirements. Then next year, we can split everything down into tasks and assign them to specific people, so the meeting just becomes a check-in.

Join the next community call (12 December 2023 @ 8:00am PT) so we can organize the first few meetings.

Once we have organized the meeting we'll send it out on the Kubeflow-discuss mailing list.

Also please join the Kubeflow slack and the #kubeflow-notebooks channel which we can use for discussing ideas.

---

Also just some notes so that I don't forget:

• we need to discuss configurable service account bindings for the pod (rather than just giving default-editor to all notebooks), at very least we should allow the admin to set all of notebooks in a specific profile to viewer only
• we need to discuss if we want to have more granular assignments of access to specific notebooks for specific users (rather than just everyone in a profile can connect to all of the notebooks)

[thesuperzapper]

For those wanting to help with Kubeflow Notebooks 2.0, please see the previous comment:

• Kubeflow Notebooks 2.0 (Kubeflow Workspaces) #7156 (comment).

Please try and get to the next kubeflow community call, so we can organize a meeting time.

Also please join the kubeflow slack (especially if you can't attend the community call), and make yourself known on the #kubeflow-notebooks channel, so we can find a good time that works for the "Notebooks 2.0 catchup" meetings).

@SachinVarghese
@Talador12
@apo-ger
@guohaoyu110
@helloericsf
@juliusvonkohout
@kimwnasptd
@mishraprafful
@punkerpunker
@tzstoyanov
@vikas-saxena02
@vvnpan
@wadhah101
@wjhhuizi

[vikas-saxena02]

@thesuperzapper I am more than happy to be part of the initiative. I will try my best to attend the community call tonight.

[StefanoFioravanzo]

@thesuperzapper I have a high-level observation on the terminology: why did you choose the name "Workspace"? I think the proposed rename to "Workbench" was well received and agreed upon here:

• Rename "Notebook" to "Workbench" #6981

The term "workspace" is generally used to refer to a project or a logical space that contains resources and user data. In Kubeflow, I believe the closest thing to a workspace we have is the namespace.

[umka1332]

I find this proposal very bad, since you are removing options that user have before. Now they will be limited to configurations that were created by admins instead of having the ability to create the thing that they need.
It's like adding complexity and removing flexibility at the same time.
It would be much better to have some presets, from which user can choose, but still be able to specify all needed values by themselves.

[thesuperzapper]

Hey all,

Please attend the Notebooks Working Group Meeting (Thursday, 18 January, 2024 @ 8:00am PT) if you want to help with Kubeflow Notebooks 2.0!

This will be the first meeting, and our goal will be to start an architecture doc and discuss ideas.

---

I think the overall steps to get Notebooks 2.0 shipped are:

• Step 1: organize a "Kubeflow Notebooks 2.0" catchup meeting.
• Step 2: finalize the design (by discussing requirements and implementation)
• Step 3: split the design up into tasks, and allocate people
• Step 4: continue regular catch-ups (to monitor progress)
• Step 5: ship as part of Kubeflow 1.9

---

@SachinVarghese
@Talador12
@apo-ger
@guohaoyu110
@helloericsf
@juliusvonkohout
@kimwnasptd
@mishraprafful
@punkerpunker
@tzstoyanov
@vikas-saxena02
@vvnpan
@wadhah101
@wjhhuizi
etc.

[thesuperzapper]

Hey all,

We have started writing up the design document for Kubeflow Notebooks 2.0.
Please review it and feel free to make comments/suggestions.

The next meeting is Thursday 25th January @ 8 AM PT / Thursday 25th January @ 4 PM PT
Where we will continue finalizing the document and to start assigning tasks to contributors.

PS: we will use the Notebooks WG meeting, to get the invite, either join kubeflow-discuss (if you use Google Calendar) or manually add the "Kubeflow Community Calendar" to yours

[juliusvonkohout]

@thesuperzapper Thursday 25th January @ 8 AM PT is the manifests wg meeting. are you sure you do not mean Febuary 1 ?

[vikas-saxena02]

Hi guys, It would be a public holiday/long weekend in Australia (Australia Day) and India (Republic Day). Can we do it next week? Cheers, Vikas

…

On Fri, 19 Jan 2024, 9:15 pm Julius von Kohout, ***@***.***> wrote: @thesuperzapper <https://github.com/thesuperzapper> Thursday 25th January @ 8 AM PT is the manifests wg meeting. are you sure you do not mean Febuary 1 ? — Reply to this email directly, view it on GitHub <#7156 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVSEDXX44LNCL4YOSTSREFDYPJBT5AVCNFSM6AAAAAAYY6GY72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBQGEZDINJZGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[thesuperzapper]

@juliusvonkohout I did not see the Manifests WG meeting at that time!

---

Currently, the Notebooks WG Meeting only runs every two weeks, we need to make it weekly while we work on Kubeflow Notebooks 2.0.

I think we can add a new bi-weekly 4:00PM PT meeting starting next week, which would be aimed at US + APAC attendees (but impossible for EU attendees). It gives everyone a chance to attend and does not overlap with the existing Manifests WG meeting.

I have made a PR that will update the community calendar to reflect this:

• Update Notebooks WG meetings community#689

[thesuperzapper]

@vikas-saxena02 that would mean the next meeting is on an Indian/Australian holiday, but I think it's fine if fewer people attend for that specific meeting, we need to keep the momentum up for the Notebooks 2.0 planning.

[thesuperzapper]

So, for anyone who wants to attend next week, expect the meeting next week to be on:

Thursday 25th Janurary @ 4:00 PM PT.

---

Either way, everyone please comment/contribute to the design document whenever you have time:

• Notebooks 2.0 - Design Document

[WYGIN]

@thesuperzapper I am willing to contribute to this feature out side the GSoC, willing to know if it is fine. Thank You.

[thesuperzapper]

@WYGIN what areas do you have experience in that could help us?

[WYGIN]

@WYGIN what areas do you have experience in that could help us?

around 9 months of experience on containerization and GoLang and more than 2 years learning curve on js(react, mainly building backend services) and not quite familiar with python(but always wanted to get in).

This feature strongly aligns with my interests and might be the best PR to work around to explore kubeflow and to challenge myself, Happy to hand it over to GSoC mentee at anytime if needed. Thank You.

You can also point me to any core feature(hopefully large/challenging issue and focuses on one of the core features on kubeflow) for me to work around.
cc: @thesuperzapper

[rockyzhou-zeyu]

@thesuperzapper I am a cs undergrad from CMU. I am really interested in Kubeflow Notebooks 2.0. I have experiences of Golang Development and good amount of understanding of Kubernetes at School. The design doc of Kubeflow Notebooks 2.0 is just inspiring to me. I am willing to contribute for the Backend as a GSoC contributor, specifically in Workspace Controller and Validation Webhooks parts.

[thesuperzapper]

For those watching, we are nearly ready to start commiting code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

• Workspace
• [WorkspaceKind](https://github.com/thesuperzapper/kubeflow-notebooks-v2-design/blob/main/crd s/workspace-kind.yaml)

We have also made good progress on the design document:

• Design Google Doc

We only need to finalize the UI mock ups for the new spawner, which we plan to share shortly.

Once we have a final design, we can start by creating folders in the repo for the new controller and front-end (with only mock code and CI/CD build pipelines), after that we can start making PRs to build out each component!

[thesuperzapper]

For those watching, we are nearly ready to start commiting code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

• Workspace
• WorkspaceKind

We have also made good progress on the design document:

• Design Google Doc

We only need to finalize the UI mock ups for the new spawner, which we plan to share shortly.

Once we have a final design, we can start by creating folders in the repo for the new controller and front-end (with only mock code and CI/CD build pipelines), after that we can start making PRs to build out each component!

[WYGIN]

For those watching, we are nearly ready to start committing code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

@thesuperzapper Have you finalized whether to use GoLang or Python? Also, have you decided between using React or Angular?

[thesuperzapper]

@thesuperzapper Have you finalized whether to use GoLang or Python? Also, have you decided between using React or Angular?

@WYGIN, for the front end, we have a number of people more familiar with React who are planning to contribute, so unless that changes, we probably will go with React.

It's still not clear which is going to be better (Go or Python) for the back-end of the front-end (not the controller, which will be in Go + Kubebuilder). .

What we choose will largely depend on who's available to work on it. However, because it's much easier to interact with Kubernetes via Go, and Go is a lot more scalable, I do have a slight preference for Go, if both options have contributors available.

[juliusvonkohout]

@thesuperzapper

i saw https://github.com/thesuperzapper/kubeflow-notebooks-v2-design/blob/667275bdbf62e6f8a3af73ea302724f8430cad22/crds/workspace-kind.yaml#L83

but i think as discussed in the meetings and documents something additional like maxSeconds: 2*86400 or so is useful in a lot of enterprise environments where we want to prevent Workspaces being abused for long-running jobs and terminate them independent of the activity after 48 hours or so.

[thesuperzapper]

Just tagging @apo-ger based on the discussion in #6927 (comment), we might consider adding a third activity probe based on "istio metrics" (e.g. the last time the Pod got an HTTP request).

Note, this will not be in the initial release of Notebooks 2.0 (unless you want to contribute it @apo-ger), but might be in a future one, once we agree on the implementation details.

[krishnakaushik195]

Hi, this is kaushik
there is a small idea from my side for the future development
how about adding the integrated LLM or any multimodel like AI

[krishnakaushik195]

hlo team
i want to contribute to the flask + react and LLM
so can some one say how do i start the contributing to contribute

[thesuperzapper]

@krishnakaushik195 we are currently developing Notebooks 2.0 in the notebooks-v2 branch of the kubeflow/notebooks repo.

Please be aware, it's NOT ready for usage yet (we are still actively developing it). If you want to help, you should attend the Notebooks Working Group (WG) meetings know what we are actively working on:

• Kubeflow Community Calander
• Notebooks WG - Meeting Notes

Get invited to the Notebooks WG meeting (on Google Calander) by joining the kubeflow-discuss Google Group. (NOTE: this will also invite you to many other community events, which you can decline or attend)

[krishnakaushik195]

Yeah sir got it
Thank you