Skip to content
Permalink
Browse files

Editorial pass on Kubeflow for AWS content.

  • Loading branch information...
nrdlngr authored and sarahmaddox committed Apr 23, 2019
1 parent f26dc00 commit 2a63f2e0dc443c53edc2014db3a90b87636525be
@@ -4,27 +4,27 @@ description = "Add TLS and authentication with your custom domain"
weight = 90
+++

This section shows the how to add TLS support and create user pool to authenticate users with your custom domain.
This section shows the how to add TLS support and create a user pool to authenticate users with your custom domain.


## Traffic Flow
External Traffic → [ Ingress → Istio-ingress-gateway → ambassador ]
External Traffic → [ Ingress → Istio ingress gateway → ambassador ]

When you generate and apply kubernetes resources, an ingress is created to manage external traffic to Kubernetes services. AWS ALB Ingress Controller will provision an Application Load balancer for that ingress. By default, TLS and authentication are not enabled support yet.
When you generate and apply kubernetes resources, an ingress is created to manage external traffic to Kubernetes services. The AWS ALB Ingress Controller will provision an Application Load balancer for that ingress. By default, TLS and authentication are not enabled at creation time.

Kubeflow community plans to move from [Ambassador](https://www.getambassador.io/) to [Istio](https://istio.io/) to manage internal traffic, see [issue](https://github.com/kubeflow/kubeflow/issues/2261). At current stage, [Ambassador](https://www.getambassador.io/) still plays the role of API gateway. TLS, authentication and authorization either can be done at ALB layer or Istio layer for aws, we plan to have istio here and forward traffic from ingress to istio gateway and then to ambassador at this moment. Once we clear direction with community, we will enable TLS and authentication by default.
The Kubeflow community plans to move from [Ambassador](https://www.getambassador.io/) to [Istio](https://istio.io/) to manage internal traffic, see [issue](https://github.com/kubeflow/kubeflow/issues/2261). Currently, [Ambassador](https://www.getambassador.io/) still plays the role of an API gateway. TLS, authentication, and authorization either can be done at the ALB or Istio layer for the AWS platform, and we plan to have Istio forward ingress traffic to the Istio gateway and then on to Ambassador when this happens. Once receive a clear direction from the community, we will enable TLS and authentication by default.


## Enable TLS and Authentication

Right now, certificate for ALB domain is not supported. Instead, you need to prepare a custom domain. You can register your domain in Route53 or any domain providers like [Godaddy](https://www.godaddy.com/).
Right now, certificates for ALB public DNS names are not supported. Instead, you must prepare a custom domain. You can register your domain in Route53 or any domain provider such as [GoDaddy.com](https://www.godaddy.com/).

[AWS Certificate Manager](https://aws.amazon.com/certificate-manager/) is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources.

To get TLS support from ALB Ingress Controller, you need to follow [tutorial](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html) to request a certificate in AWS Certificate Manager. After validation success, you will get a certificate ARN.
To get TLS support from the ALB Ingress Controller, you need to follow [tutorial](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html) to request a certificate in AWS Certificate Manager. After successful validation, you will get a certificate ARN to use with the ALB Ingress Controller.

<img src="/docs/images/aws/cognito-certarn.png"
alt="Cognito Cert Arn"
alt="Cognito Certificate ARN"
class="mt-3 mb-3 border border-info rounded">

[AWS Cognito](https://aws.amazon.com/cognito/) lets you add user sign-up, sign-in, and access control to your web and mobile apps quickly and easily. Amazon Cognito scales to millions of users and supports sign-in with social identity providers, such as Facebook, Google, and Amazon, and enterprise identity providers via SAML 2.0.
@@ -37,7 +37,7 @@ To get TLS support from ALB Ingress Controller, you need to follow [tutorial](ht
alt="Cognito Domain Name"
class="mt-3 mb-3 border border-info rounded">

In order to authenticate and manage users for kubeflow, let's create a user pool. You can follow instructions here. Once a user pool created, we have UserPoolId, Cognito Domain name and Cognito Pool Arn.
In order to authenticate and manage users for Kubeflow, let's create a user pool. You can follow these instructions here. Once a user pool created, we will have a `UserPoolId`, a Cognito Domain name, and a Cognito Pool Arn.

Before you apply k8s, you can go into ${KUBEFLOW_SRC}/${KFAPP}/ks_app,

@@ -49,18 +49,18 @@ ks param set istio-ingress enableCognito true
ks param set istio-ingress certArn arn:aws:acm:us-west-2:xxx:certificate/xxxe4031c
```

After your finish TLS and Authentication configuration, then you can run `${KUBEFLOW_SRC}/${KFAPP}/scripts/kfctl.sh apply k8s`.
After you finish the TLS and Authentication configuration, then you can run `${KUBEFLOW_SRC}/${KFAPP}/scripts/kfctl.sh apply k8s`.

After your ingress dns is ready, you need to create a `CNAME` in your DNS records.
After your ingress DNS is ready, you need to create a `CNAME` in your DNS records.

<img src="/docs/images/aws/custom-domain-cname.png"
alt="Custom Domain CNAME"
class="mt-3 mb-3 border border-info rounded">

Then you can visit `https://www.shanjiaxin.com`, which is a custom domain we use in this case, it will redirect you to authentication page. We added a user `kubeflow-test-user` in the cognito setting and we can use this user to login service.
Then you can visit `https://www.shanjiaxin.com`, which is a custom domain we use in this case, it will redirect you to an authentication page. We added a user `kubeflow-test-user` in the cognito setting and we can use this user for the login service.

<img src="/docs/images/aws/authentication.png"
alt="Coginito Authentication popup"
alt="Cognito Authentication pop-up"
class="mt-3 mb-3 border border-info rounded">

<img src="/docs/images/aws/kubeflow-main-page.png"
@@ -4,28 +4,28 @@ description = "Tailoring a AWS deployment of Kubeflow"
weight = 20
+++

This guide describes how to customize your deployment of Kubeflow on EKS.
Some of the steps can be done before `apply platform`, some of them can be done before `apply k8s`. Please check following sections for details. If you don't understand development phase. Please check [deploy](/docs/aws/deploy) for details.
This guide describes how to customize your deployment of Kubeflow on Amazon EKS.
Some of the steps can be done before you run the `apply platform` command, and some of them can be done before you run the `apply k8s` command. Please see the following sections for details. If you don't understand the deployment process, please see [deploy](/docs/aws/deploy) for details.


## Customizing Kubeflow

Here're all configuration options for kfctl for platform aws.
Here are the optional configuration parameters for `kfctl` on the AWS platform.

| Options | Description | Required |
|---|---|---|
| awsClusterName | Name of new cluster or existing eks cluster | YES |
| awsRegion | Region EKS cluster launch in | YES |
| awsNodegroupRoleNames | EKS node groups role names | YES for existing cluster/ No for new cluster |
| `awsClusterName` | Name of your new or existing Amazon EKS cluster | YES |
| `awsRegion` | The AWS Region to launch in | YES |
| `awsNodegroupRoleNames` | The IAM role names for your worker nodes | YES for existing clusters / No for new clusters |


### Customize EKS cluster
### Customize your Amazon EKS cluster

Before you run `${KUBEFLOW_SRC}/scripts/kfctl.sh apply platform`, you can edit cluster configuration file to change cluster specs before cluster creation.
Before you run `${KUBEFLOW_SRC}/scripts/kfctl.sh apply platform`, you can edit the cluster configuration file to change cluster specification before you create the cluster.

Cluster config is located in `${KUBEFLOW_SRC}/${KFAPP}/aws_config/cluster_config.yaml`. Please check [eksctl](https://eksctl.io/) for configuration details.
Cluster configuration is stored in `${KUBEFLOW_SRC}/${KFAPP}/aws_config/cluster_config.yaml`. Please see [eksctl](https://eksctl.io/) for configuration details.

For example, this is a cluster manifest with one node group which has 2 p2.xlarge instance. You can easily enable SSH and configure public key. All worker nodes will be in single availability zone.
For example, the following is a cluster manifest with one node group which has 2 `p2.xlarge` instances. You can easily enable SSH and configure a public key. All worker nodes will be in single Availability Zone.

```yaml
apiVersion: eksctl.io/v1alpha4
@@ -39,7 +39,7 @@ metadata:
#availabilityZones: ["us-west-2b", "us-west-2c", "us-west-2d"]
# NodeGroup holds all configuration attributes that are specific to a nodegroup
# You can have several node group in your cluster.
# You can have several node groups in your cluster.
nodeGroups:
- name: eks-gpu
instanceType: p2.xlarge
@@ -77,10 +77,10 @@ nodeGroups:
```

### Customize Private Access
Please check [section](/docs/aws/private-access)
Please see [section](/docs/aws/private-access)

### Customize Logging
Please check [section](/docs/aws/logging)
Please see [section](/docs/aws/logging)

### Customize Authentication
Please check [section](/docs/aws/authentication)
Please see [section](/docs/aws/authentication)
@@ -1,5 +1,5 @@
+++
title = "Deployment"
description = "Instructions for deploying Kubeflow with the shell"
description = "Instructions for deploying Kubeflow on AWS"
weight = 4
+++
@@ -3,23 +3,23 @@ title = "Init cluster setup for existing cluster"
weight = 6
+++

## Before start
## Before you start

This is one step of [tutoral](/docs/aws/deploy/install-kubeflow), please make sure you have previous setup done.
This is one step of [installing Kubeflow](/docs/aws/deploy/install-kubeflow), please make sure you have completed the prerequisite steps there before proceeding.

### Deploy Kubeflow on existing EKS Cluster
### Deploy Kubeflow on existing Amazon EKS Cluster

If you would like to deploy Kubeflow on existing EKS cluster, the only difference setup is when you init platform setup. Since you manage your own cluster resources, you need to provide `AWS_CLUSTER_NAME` and `AWS_NODE_GROUP_ROLE_NAMES`.
If you would like to deploy Kubeflow on existing Amazon EKS cluster, the only difference in setup is when you initialize the platform setup. Since you manage your own cluster resources, you need to provide `AWS_CLUSTER_NAME` and `AWS_NODE_GROUP_ROLE_NAMES`.


1. Get your cluster name and node group roles ready
1. Retrieve your Amazon EKS cluster name and the IAM role name for your worker nodes. Set these values to the following environment variables.

```
export AWS_CLUSTER_NAME=<YOUR EKS CLUSTER NAME>
export AWS_NODE_GROUP_ROLE_NAMES=<YOUR NODE GROUP ROLE NAMES>
```

> Note: To get your EKS cluster node groups, you can check IAM setting or running following commands. We assume you use `eksctl` to create cluster. If you use other provision tools to create node groups, please find the roles by yourself.
> Note: To get your Amazon EKS worker node IAM role name, you can check IAM setting by running the following commands. This command assumes that you used `eksctl` to create your cluster. If you use other provisioning tools to create your worker node groups, please find the role that is associated with your worker nodes in the Amazon EC2 console.
```
aws iam list-roles \
@@ -31,9 +31,9 @@ If you would like to deploy Kubeflow on existing EKS cluster, the only differenc
eksctl-kubeflow-example-nodegroup-ng-185-NodeInstanceRole-1DDJJXQBG9EM6
```

If you have multiple node groups, you will see corresponding number of node group roles. In that case, please use comma , between roles for string concat.
If you have multiple node groups, you will see corresponding number of node group roles. In that case, please provide the role names as a comma-separated list.

1. Init setups
1. Initial setup

```
cd ${KUBEFLOW_SRC}

0 comments on commit 2a63f2e

Please sign in to comment.
You can’t perform that action at this time.