Skip to content

Latest commit

 

History

History
219 lines (137 loc) · 9.42 KB

kubernetes-on-aws.md

File metadata and controls

219 lines (137 loc) · 9.42 KB
Raw

Kubernetes on AWS

Deploy a fully-functional Kubernetes cluster using AWS CloudFormation. Your cluster will be configured to use AWS features to enhance Kubernetes. For example, Kubernetes may automatically provision an Elastic Load Balancer for each Kubernetes Service. After completing this guide, a deployer will be able to interact with the Kubernetes API from their workstation using the kubectl CLI tool.

Choose one of the following paths to get started:

  1. Use the kube-aws CLI tool to automate cluster deployment. See the kube-aws Quickstart section below.

  2. Launch Stack & Configuration Guide

    Click the following Launch Stack button, then use the Configuration Guide later in this document to decide how to set the CloudFormation template parameters:

    Launch Stack

kube-aws Quickstart

Download

The latest releases of kube-aws are published through GitHub. Find the latest release and download the appropriate release artifact (i.e. not "Source Code"), then extract the kube-aws binary from the downloaded <file> like so:

tar -xf <file> kube-aws

Configure AWS Credentials

Configure your local workstation with AWS credentials using one of the following methods:

  1. Environment Variables

    Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to the value of your your AWS access and secret keys, repsectively:

    export AWS_ACCESS_KEY_ID=AKID1234567890
export AWS_SECRET_ACCESS_KEY=MY-SECRET-KEY

  2. Config File

    Write your credentials into the file ~/.aws/credentials using the following template:

    [default]
aws_access_key_id = AKID1234567890
aws_secret_access_key = MY-SECRET-KEY

Configure Cluster

Create a local cluster.yaml config file, using the example available on GitHub as a starting point.

curl --silent --location https://raw.githubusercontent.com/coreos/coreos-kubernetes/master/multi-node/aws/cluster.yaml.example > cluster.yaml

Edit the file, setting any necessary options according to the Configuration Guide. Note that some options are required, and have no preconfigured default.

Deploy

Run kube-aws up, which will first establish the TLS certificate infrastructure necessary for your cluster, then create all necessary AWS resources. Once the AWS resources are created, Kubernetes will start up automatically.

NOTE: the TLS keys and certificates generated by kube-aws should not be used to deploy a production Kubernetes cluster. Each component certificate is only valid for 90 days, while the CA is valid for 365 days. If deploying a production Kubernetes cluster, consider establishing PKI independently of this tool first.

Navigate to the DNS registrar hosting the zone for the provided external DNS name and ensure a single A record exists, routing the value of externalDNSName defined in cluster.yaml to the externally-accessible IP of the controller instance. You may use kube-aws describe to get this value after cluster creation, if necessary.

A kubectl config file will be written to ./clusters/<cluster-name>/kubeconfig, which can be used to interact with your Kubernetes cluster like so:

kubectl --kubeconfig=clusters/<cluster-name>/kubeconfig get nodes

Destroy

When you are done with your cluster, simply run kube-aws destroy and all cluster components will be destroyed. If you created any Kubernetes Services of type LoadBalancer, you must delete these first, as the CloudFormation cannot be fully destroyed if any externally-managed resources still exist.

How it Works

Before creating any AWS resources, kube-aws begins by initializing the TLS infrastructure needed to securely operate Kubernetes. This includes the certificate authority, signed server certificates for the Kubernetes API server and workers, and a signed client certificate for administrative use. The API server certificate will be valid for the value of externalDNSName, as well as a the DNS names used to route Kubernetes API requests inside the cluster.

kube-aws does not manage a DNS zone for the cluster. This means that the deployer is responsible for ensuring the routability of the external DNS name to the public IP of the controller instance.

After generating the necessary TLS infrastructure, kube-aws creates a new CloudFormation, pointing to a CloudFormation template managed by this project. The CloudFormation template encompasses everything necessary to deploy the cluster.

After some time, the cluster will come up and start serving API requests. You can view the current state of the cluster, including the publicly-routable IP address, using kube-aws describe. kube-aws generates a kubeconfig file pre-configured with the admin TLS certificates that can be used with kubectl to interact with the cluster.

When the cluster is taken down, kube-aws simply destroys the CloudFormation and the local workspace.

Configuration Guide

If using kube-aws, the TLS certificates & keys will automatically be configured. Other values are set using the cluster config, an example of which is available on GitHub.

kube-aws Cluster Config

The following parameters can be set in the cluster config file used when calling kube-aws up. The TLS infrastructure (CA, certs & keys) are automatically generated and configured.

clusterName

Choose a unique name for the Kubernetes cluster. This will be used to identify all AWS resources that make the cluster, and must not conflict with other Kubernetes clusters deployed into the same AWS account.

region

Deploy the CloudFormation stack into a specific region.

availabilityZone

Place all resources in a specific AWS availability zone. If not set, a zone will be chosen for you in the region being used to deploy the CloudFormation stack.

keyName

Decide what SSH keypair to authorize across all instances in the cluster. The value of this field is the name of a keypair already loaded into the AWS account in use.

workerCount

This configures the number of worker instances to deploy into the cluster. This may be increased or decreased in the future.

artifactURL

The coreos-kubernetes artifacts are available at a default location, but the location may be overridden for development.

externalDNSName

This DNS name is assumed to be routable from outside the cluster to the Kubernetes API. It is automatically added as a SAN to the kube-apiserver TLS certificate.

CloudFormation Template Parameters

The folowing parameters can be set when directly launching a CloudFormation stack through the AWS web console.

ClusterName

Choose a unique name for the Kubernetes cluster. This will be used to identify all AWS resources that make the cluster, and must not conflict with other Kubernetes clusters deployed into the same AWS account.

AvailabilityZone

Place all resources in a specific AWS availability zone. If not set, a zone will be chosen for you in the region being used to deploy the CloudFormation stack.

KeyName

Decide what SSH keypair to authorize across all instances in the cluster. The value of this field is the name of a keypair already loaded into the AWS account in use.

ControllerInstanceType, WorkerInstanceType

These fields are the names of the EC2 instance types to use for the controller and worker instances, respectively. It is recommended that instances have 3gb+ of RAM. More resources allocated to the worker instances directly increases the scheduleable capacity of the Kubernetes cluster.

WorkerCount

This configures the number of worker instances to deploy into the cluster. This may be increased or decreased in the future.

ReleaseChannel

This configures which CoreOS Linux release channel to use. The only supported channel right now is "alpha".

ArtifactURL

The coreos-kubernetes artifacts are available at a default location, but the location may be overridden for development.

Certs & Keys

It is the responsibility of the deployer to establish the TLS infrastructure needed to secure the Kubernetes cluster. All keys and certs must be PEM-formatted and base64-encoded.

APIServerCert, APIServerKey

The certificate and key granted to the kube-apiserver. This certificate will be presented to external clients of the Kubernetes cluster, so it should be valid for external DNS names, if necessary.

Additionally, the certificate must have the following SANs:

  • 10.0.0.50
  • 10.3.0.1
  • kubernetes
  • kubernetes.default
  • kubernetes.default.svc
  • kubernetes.default.svc.cluster.local
WorkerCert, WorkerKey

The certificate and key granted to the kubelets on worker instances. The certificate is shared across all workers, so it must be valid for all worker hostnames. This is achievable with the SAN *.*.cluster.internal, or *.ec2.internal if using the us-east-1 AWS region.

CACert

The certificate authority's TLS certificate used to sign other certificates in the cluster.