fixes 'create a notebook controller that can replace jupyterhub and uses k8 native auth'

[kkasravi]

fixes #1769

This PR will launch a jupyter notebook in a manner similar to jupyterhub but has a
few noteworthy improvements:

• a Notebook CRD defines the API where the Notebook.spec.template.spec (PodTemplateSpec) replaces c.KubeSpawner.<attributes>
• no separate authentication layer is imposed. Uses k8 authentication
• will launch a jupyter notebook in a user's own namespace if paired with the profiles component

See the README.md for details.

---

This change is 

[kkasravi]

[WIP] see description

[kkasravi]

/hold

[jlzhao27]

hey @kkasravi, this is the metacontroller docs right? https://metacontroller.app/

Might be worthwhile to call out in the README that the only requirements are kubectl access and an installed metacontroller, I think you mention it in the design part but it is a runtime requirement also.

[jlewi]

Makes sense, however I'll need to add an httpserver to present the ui (similar to jupyterhub)

I'm not sure I understood this comment. What does this have to do with the ksonnet package organization of the new Jupyter CRD?

[kkasravi]

hey @kkasravi, this is the metacontroller docs right? https://metacontroller.app/

Might be worthwhile to call out in the README that the only requirements are kubectl access and an installed metacontroller, I think you mention it in the design part but it is a runtime requirement also.

Sure, I'll update the README.

[kkasravi]

Makes sense, however I'll need to add an httpserver to present the ui (similar to jupyterhub)

I'm not sure I understood this comment. What does this have to do with the ksonnet package organization of the new Jupyter CRD?

My misunderstanding I think ...

Were you saying that I should do I git mv kubeflow/notebooks/* kubeflow/jupyter and have 2 prototypes under kubeflow/jupyter/prototypes/ or completely replace the current kubeflow/jupyter with what's under kubeflow/notebooks?

[jlewi]

Were you saying that I should do I git mv kubeflow/notebooks/* kubeflow/jupyter and have 2 prototypes under kubeflow/jupyter/prototypes/ or completely replace the current kubeflow/jupyter with what's under kubeflow/notebooks?

I was thinking multiple prototypes in the kubeflow/jupyter package. Is there any downside to doing this? I was thinking this would just avoid additional churn later on as I think kubeflw/jupyter is the package where we'd like them to live long term.

[kkasravi]

Were you saying that I should do I git mv kubeflow/notebooks/* kubeflow/jupyter and have 2 prototypes under kubeflow/jupyter/prototypes/ or completely replace the current kubeflow/jupyter with what's under kubeflow/notebooks?

I was thinking multiple prototypes in the kubeflow/jupyter package. Is there any downside to doing this? I was thinking this would just avoid additional churn later on as I think kubeflw/jupyter is the package where we'd like them to live long term.

I agree. No downside since they both in theory could be deployed. I'll try and get this out asap.

[jlewi]

Thanks @kkasravi !

[jlewi]

@kkasravi Think we can get this merged this week?

[kkasravi]

yes, I'll push changes today which should be most of what's required.

[kkasravi]

/retest

[jlewi]

Thanks @kkasravi

2 comments but I think these are best handled in follow on discussion and PRs as necessary.

1. Why deployment not statefulset?

The main advantage of a statefulset is that the pod name is stable. This is mostly useful from a debugging perspective; i.e. we can tell users to look for a pod with a predictable name as opposed to filtering by labels or some other mechanism.

Since the type of resource is internal to the CRD we could always fix it later.

1. Other question is about naming. I think notebook might refer to the ipynb file. Whereas the server is jupyter. So having the resource named "notebooks.kubeflow.org" might be confusing. It might lead users to think that the resource is tied to a single notebook.

The text on the website leaves me a bit confused; http://jupyter.org
They say "Jupyter Notebook".

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jlewi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jlewi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment