Make it easy to get started with Kubeflow

[jlewi]

We'd like to have a super simple getting started experience for people who want to kick the tires on Kubeflow.

In #90, ksonnet was highlighted as a potential barrier.

As a strawman I think a good getting started experience might be

kubectl run --image=gcr.io/kubeflow/bootstrap:v1

In which case the bootstrap program would automatically deploy a stock Kubeflow deployment on the cluster. This would prevent the user from having to know anything about ksonnet until they need to more complex things like customize the deployment.

We could enrich this by having bootstrap startup a webapp that could provide a guided walk through to make it easy for the user to set basic options.

[elmiko]

i like the idea of a bootstrap container to do all the setup, i'm also a fan of enabling user growth by exposing the details. when this had been brought up in #23 , you had recommended the ks show command as a possible alternative for getting at the kubernetes manifests. i wonder if we could also demonstrate something like this in the quick start guide (eg "Here's how to get the kubernetes manifests"), or it could even be a quick export from the bootstrap webapp.

this way we could keep the ksonnet files as the primary source of truth for installation, give the users some quick commands to get at the kubernetes manifests and not need to maintain a separate set of files/instructions. i realize it may still require the user to have ksonnet (barring some sort of bootstrap webapp), but at least we could do a little "teaching to fish".

[jlewi]

I like the idea of the webapp spitting out the ksonnet app either by giving you a tarball that you can download or pushing to a Git repo

[foxish]

I think this might be a point to consider a kubeflow-operator as a concept. It could be configurable and also allowing sufficient flexibility, while abstracting away the underlying implementation, and offer high level knobs dealing with the individual sub components.

…

On Jan 9, 2018 10:17 AM, "Jeremy Lewi" ***@***.***> wrote: I like the idea of the webapp spitting out the ksonnet app either by giving you a tarball that you can download or pushing to a Git repo — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#105 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA3U5z6HKbWW9Wp4edpNT3wLUCKDDSZ1ks5tI60dgaJpZM4RYLuN> .

[elmiko]

@jlewi by "ksonnet app", which app were you referring to?

[sub-mod]

Can we have the manifest file available along with ksonnet code on github?

[erikerlandson]

I like the idea of a kubeflow command for dumping manifests. That ought to eliminate the issues of developer burden raised in #90, since it becomes self serve

[erikerlandson]

Also +1 for kubeflow operator, although I remain skeptical that there is just one uber-CRD for this

[jlewi]

I'd like to keep this issue pretty narrowly scoped to providing a quick start that hides ksonnet. Lets open up other issues as needed to think about maintaining a Kubeflow deployment throughout its lifetime.

I think if we keep the scope narrow, we can keep the issue focused on something that is tractable and could be accomplished in a couple days.

I think a CRD is way more complicated and I share @erikerlandson's skepticism that a single uber-CRD may not work.

Can we have the manifest file available along with ksonnet code on github?
I think it would be fine for bootstrap.py to spit out YAML manifests in addition to ksonnet code.

@jlewi by "ksonnet app", which app were you referring to?

If you follow the quick start you do

ks init ${APP_NAME}
cd ${APP_NAME}
...

So by ksonnet app I meant all the files in the ${APP_NAME} directory which would be created by boostrap.

[sub-mod]

@jlewi i agree.I don;t want to change the scope of this issue.But the issue #90 is closed.Do you agree whether the manifest files need to be published on github?Do we open #90 to discuss there?

[jlewi]

What problem are you trying to solve? The problem in #90 was that downloading/using ksonnet was a barrier to getting started. I claim that the proposal in this issue is a better solution to that problem then publishing manifests.

[zomglings]

1. Manifests: Do we need a kubeflow command for doing this as @erikerlandson suggested? Why not just use ks show? Is the problem not wanting to install ksonnet (which I am totally sympathetic to)?

2. bootstrap container: It would be good to start defining our requirements for this. Is it just for quick start in the sense of "introduction to kubeflow" or is it intended to get a barebones deployment up and running (so that a user would use it every time they wanted to start a new kubeflow project). The nice thing about doing it this way is that we can parametrize it with apps, volumes, secrets, models, etc.

[jlewi]

bootstrap container: It would be good to start defining our requirements for this. Is it just for quick start in the sense of "introduction to kubeflow" or is it intended to get a barebones deployment up and running (so that a user would use it every time they wanted to start a new kubeflow project). The nice thing about doing it this way is that we can parametrize it with apps, volumes, secrets, models, etc.

My preference would be to let whoever is interested in picking this up drive this. I think this is an area where we can iterate quickly.

[sfabel]

I think that using ksonnet is not a hurdle; it's just a lack of documentation, IMHO. Simplify too many things and you'll end up with something trivial and useless.

Here's how I think it could be done: point newcomers to something like my tutorial, so they get to use kubeflow and experience it. Then, let's create material that shows how to create your own custom job. This could involve adding to the Tensorflow documentation as well. I am planning to evolve the documentation from our side to show the developer workflow e2e, so people can get started quickly with leveraging Kubeflow without it getting in their way.

[bhack]

For newcomers could be useful to expose examples with Estimator high level API. See kubeflow/training-operator#61

[elmiko]

Manifests: Do we need a kubeflow command for doing this as @erikerlandson suggested? Why not just use ks show? Is the problem not wanting to install ksonnet (which I am totally sympathetic to)?

i think it would be nice for newcomers to have easy access to the kubernetes manifests. i get that installing and using ksonnet may not be the most difficult task, but even if we don't end up checking the manifests into the repo it seems like an easy wrapper to have the tool we are discussing have the ability to emit the manifests.

i'm not sure this addresses @sub-mod 's concerns, but my issue is providing easy access to those manifests without requiring additional tooling beyond kubeflow. given that we are discussing have a deployment mechanism that obviates the need for ksonnet, it seems reasonable to wrap the ks show (or similar) functionality. i also realize this conversation may be better placed for the resulting design discussion for the tooling, but it would be nice to see how much interest there is in having these manifests.

[jlewi]

@elsonrodriguez I think you said you had a container with ksonnet that could run and install kubeflow. Anything that might be useful for this issue?

ksonnet is also in our notebooks images so arguably we could just use that.

[elsonrodriguez]

@jlewi Yeah, I built a container for the end-to-end kubeflow example. Here are the bits, let me know what you think can be reused:

Dockerfile: https://github.com/elsonrodriguez/examples/blob/7a2f2b3a6d0f15ef5a4155cc2632f857804d80aa/e2e/Dockerfile.ksonnet
Entrypoint: https://github.com/elsonrodriguez/examples/blob/e2e/e2e/ksonnet-entrypoint.sh
Usage: https://github.com/elsonrodriguez/examples/blob/0241f4177ecbad200bb2560e73891284572f64f1/e2e/tfargo.yaml#L321-L338
Built Image: elsonrodriguez/ksonnet:0.8.0-test10

The main issue was that ksonnet expects a kubeconfig, and doesn't do inclusterconfig ( ksonnet/ksonnet#332), so I utilized this script to generate a kubeconfig from a service account. This means that the serviceaccount that is provided to the ksonnet container has to have permissions to read serviceaccounts, so not very secure. Alternately a kubeconfig can be provided as a secret.

[jlewi]

@kkasravi Had a comment about a script he has for deploying Kubeflow. I think that script is related to this issue.

[jlewi]

Some info about what sig apps is thinking
SIG apps repo - https://github.com/kubernetes-sigs/application
KEP - kubernetes/community#1629

[mhausenblas]

Another option might be Parameterizer which is WIP and we'll present soon to SIG Apps …

[jlewi]

/assign jlewi
I'm going to take an initial stab at this

Parameterizer looks interesting but I think the experience I want is kubectl run....

If you have to deploy a CRD and then create a manifest for that CRD and then the options in that manifest start to explode, its not clear to me that's a fundamentally different experience from what exists today with ksonnet.

[jlewi]

I think #595 is a good starting point here are some follow up improvements to the bootstraper in #594

1. Check if the namespace exists and if not create it
2. Check if user is a cluster admin and therefore has permission to create the roles as part of Kubeflow
   deployment and if not create a cluster role

• Basically deal with this RBAC issue

We would also like to detect if we are using GKE and run the following if we are

1. Create a static IP address to use with IAP
2. Create GCP service accounts and store them as secrets in the cluster

• We should probably create two service accounts
• An admin service account that can be used for things like the IAP CRD
• A user service account to be used with TFJobs and the like.

1. We should configure JupyterHub to attach the GCP service account for the user to pods

[jlewi]

Closing this issue since we have #595 which is a good start. I will open more specific issues.