Support install kubeflow by Deployment Manager #757
Comments
|
Can you leverage our statefulset for that? bootstrap/examples/kubeflow-toolbox.yaml |
|
We want to use deployment manager to setup non K8s resources; e.g. GKE, static ips, buckets, service accounts etc... The goal is to provide declarative configs that make it easy to get started with Kubeflow on GCP. We'd like this infrastructure to connect with the bootstrapper e.g. by launching the stateful set to run the bootstrapper to initialize the app. |
|
Here's an example of using Deployment Manager to create a daemonset. I suspect a similar approach could be used to
|
|
@jlewi this has similarities with the bootstrapper modifications proposal (i'm addressing your comments under there separately). Would using the deployment manager couple kubeflow to GCP or would there be other deployment managers for other cloud-providers? Having an initial infrastructure to create resources required for the bootstrapper to run is what I was suggesting in the proposal but using a CRD and 2 controllers with different RBACs. The idea was to enable self-serve for data scientists so they are able deploy kubeflow w/o requiring cluster level RBACs, rather they could have rolebindings specific to the namespace. I think this does the same thing by using the deployment manager API rather than a CRD. |
|
No this would not couple Kubeflow to GCP. We will never do that. The goal here is to provide recommendations about how to setup Kubeflow on GCP (#240 and #241). As part of that, we want to recommend using deployment manager mostly to setup non GCP resources (e.g. DNS, buckets, IAM roles, secrets etc...). The reason I'm advocating for using deployment manager (as opposed to say a shell script for running gcloud), is because I want to follow the K8s pattern of managing infrastructure declaratively. The intent here is that each Cloud Provider or K8s distribution should provide best practices/recommendations for their platform (issue #240). |
|
For comparison, at Kubecon @sfabel of Canonical demo'd using Juju to provision resources and then deploy Kubeflow. |
…to run Kubeflow. * The config just creates the GKE cluster we still have a lot of work to do to get a complete solution for deploying Kubeflow. * The goal is to provide a complete declarative solution that makes it easy to get started with Kubeflow. The idea is the configs should provide sensible defaults (e.g. n1-standard-8) so people can just copy it to create a good initial setup. * For more advanced setup users can customize the config. For example, they might configure VPC networks as part of their deployment. * The current thinking is that deployment manager should create the K8s cluster and other resources (service accounts, namespaces, etc...) to run the bootstrapper. * Deployment manager could then kick off the bootstrapper which would create a ksonnet app to deploy Kubeflow and then deploy Kubeflow. * Create an E2E test to provide a smoke test of the configs. Related to: kubeflow#757 Use Deployment manager kubeflow#241 Recommended setup for GKE - The idea is to encode the recommended setup as a deployment manager config.
…to run Kubeflow (#787) * Initial set of deployment manager configs for creating a GKE cluster to run Kubeflow. * The config just creates the GKE cluster we still have a lot of work to do to get a complete solution for deploying Kubeflow. * The goal is to provide a complete declarative solution that makes it easy to get started with Kubeflow. The idea is the configs should provide sensible defaults (e.g. n1-standard-8) so people can just copy it to create a good initial setup. * For more advanced setup users can customize the config. For example, they might configure VPC networks as part of their deployment. * The current thinking is that deployment manager should create the K8s cluster and other resources (service accounts, namespaces, etc...) to run the bootstrapper. * Deployment manager could then kick off the bootstrapper which would create a ksonnet app to deploy Kubeflow and then deploy Kubeflow. * Create an E2E test to provide a smoke test of the configs. Related to: #757 Use Deployment manager #241 Recommended setup for GKE - The idea is to encode the recommended setup as a deployment manager config. * Autoformat jsonnet. * Fix the parameters. * Fix lint.
|
We can close this once #875 is committed. |
…to run Kubeflow (kubeflow#787) * Initial set of deployment manager configs for creating a GKE cluster to run Kubeflow. * The config just creates the GKE cluster we still have a lot of work to do to get a complete solution for deploying Kubeflow. * The goal is to provide a complete declarative solution that makes it easy to get started with Kubeflow. The idea is the configs should provide sensible defaults (e.g. n1-standard-8) so people can just copy it to create a good initial setup. * For more advanced setup users can customize the config. For example, they might configure VPC networks as part of their deployment. * The current thinking is that deployment manager should create the K8s cluster and other resources (service accounts, namespaces, etc...) to run the bootstrapper. * Deployment manager could then kick off the bootstrapper which would create a ksonnet app to deploy Kubeflow and then deploy Kubeflow. * Create an E2E test to provide a smoke test of the configs. Related to: kubeflow#757 Use Deployment manager kubeflow#241 Recommended setup for GKE - The idea is to encode the recommended setup as a deployment manager config. * Autoformat jsonnet. * Fix the parameters. * Fix lint.
To provide better user experience on GCP,
we should provide installing kubeflow through Deployment Manager.
Installation should be as simple as:
The text was updated successfully, but these errors were encountered: