Skip to content

Commit

Permalink
Added terraform script to create eks.
Browse files Browse the repository at this point in the history
see docs/cloud.md for setting up AKS/EKS clusters, and install Antrea over them.
  • Loading branch information
Su Wang authored and antoninbas committed Mar 23, 2020
1 parent 03bf183 commit e7d8502
Show file tree
Hide file tree
Showing 7 changed files with 448 additions and 0 deletions.
55 changes: 55 additions & 0 deletions docs/cloud.md
@@ -0,0 +1,55 @@
# Deploying Antrea on a cloud provider

Antrea may run in networkPolicyOnly mode in some cloud managed clusters. This document describes
steps to create EKS using terraform.

## Common Prerequisites
1. To run EKS cluster, install and configure AWS cli(either version 1 or 2), see
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html, and
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
2. Install aws-iam-authenticator, see
https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html
3. Install terraform, see https://learn.hashicorp.com/terraform/getting-started/install.html
4. You must already have ssh key-pair created. This key pair will be used to access worker Node via
ssh.
```bash
ls ~/.ssh/
id_rsa id_rsa.pub
```


## Create an EKS cluster via terraform
Ensures that you have permission to create EKS cluster, and have already
created EKS cluster role as well as worker Node profile.
```bash
export TF_VAR_eks_cluster_iam_role_name=YOUR_EKS_ROLE
export TF_VAR_eks_iam_instance_profile_name=YOUR_EKS_WORKER_NODE_PROFILE
export TF_VAR_eks_key_pair_name=YOUR_KEY_PAIR_TO_ACCESS_WORKER_NODE
```
Where
- TF_VAR_eks_cluster_iam_role_name may be created by following these
[instructions](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html#create-service-role)
- TF_VAR_eks_iam_instance_profile_name may be created by following these
[instructions](https://docs.aws.amazon.com/eks/latest/userguide/worker_node_IAM_role.html#create-worker-node-role)
- TF_VAR_eks_key_pair_name is the aws key pair name you have configured by following these
[instructions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws),
using ssh-pair created in Prerequisites item 4



Create EKS cluster
```bash
./hack/terraform-eks.sh create
```
Interact with EKS cluster
```
./hack/terraform-eks.sh kubectl ... // issue kubectl commands to EKS cluster
./hack/terraform-eks.sh load ... // load local built images to EKS cluster
./hack/terraform-eks.sh destroy // destroy EKS cluster
```
and worker Node can be accessed with ssh via their external IPs.

Apply Antrea to EKS cluster
```bash
./hack/generate-manifest.sh --encap-mode networkPolicyOnly | ~/terraform/eks kubectl apply -f -
```
19 changes: 19 additions & 0 deletions hack/install-cloud-tools.sh
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

# Copyright 2020 Antrea Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -eo pipefail

mkdir -p $HOME/terraform && cp ./hack/terraform-eks.sh $HOME/terraform/eks && cp -R ./hack/terraform $HOME/terraform
150 changes: 150 additions & 0 deletions hack/terraform-eks.sh
@@ -0,0 +1,150 @@
#!/usr/bin/env bash

# Copyright 2020 Antrea Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# NOTE: terraform enviroment variables need to be set
AWS_CLI="aws"
TERRAFORM="terraform"
KUBECTL="kubectl"
AWS_IAM="aws-iam-authenticator"
RUN_PATH="$HOME/tmp/terraform-eks"
if [[ $(uname) == "Darwin" ]]; then
READLINK="greadlink"
else
READLINK="readlink"
fi
CMD_ARRAY=($AWS_CLI $TERRAFORM $KUBECTL $AWS_IAM pv bzip2 jq $READLINK)
ENV_ARRAY=(TF_VAR_eks_cluster_iam_role_name TF_VAR_eks_iam_instance_profile_name TF_VAR_eks_key_pair_name)

_usage="Usage: $0 :
[create] : create eks cluster
[destroy] : destroy eks cluster
[kubectl] : access eks
[load] : load container image from local machine to eks cluster
[--help|h]"

function print_help {
echo "Try '$0 --help' for more information."
}

function print_usage {
echo "$_usage"
}

function validate {
for cmd in ${CMD_ARRAY[@]}; do
command -v $cmd > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo $cmd not found, please install first
exit 1
fi
done

for env in ${ENV_ARRAY[@]}; do
printenv $env > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo required enviroment variable $env is not set.
exit 1
fi
done

# check for aws permissions
$AWS_CLI eks list-clusters > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo aws user has no permission to eks
exit 1
fi

$AWS_CLI iam list-roles > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo aws user has no permission to iam
exit 1
fi
}

function apply {
validate
CONFIG_PATH=$(dirname $($READLINK -f "$0"))/terraform/eks
mkdir -p $RUN_PATH
if [ $? -ne 0 ]; then
echo $RUN_PATH cannot be created
exit 1
fi
cp $CONFIG_PATH/*.tf $RUN_PATH
cd $RUN_PATH
$TERRAFORM init
if [ $? -ne 0 ]; then
exit 1
fi
echo "yes" | $TERRAFORM apply
if [[ $? -ne 0 ]]; then
echo "eks creation failed"
exit 1
fi
echo eks cluster created.
$TERRAFORM output kubectl_config > $RUN_PATH/kubeconfig
echo run eks kubectl ... to acess it.
}

function destroy {
validate
if [ ! -d $RUN_PATH ]; then
exit 0
fi
cd $RUN_PATH
echo "yes" | $TERRAFORM destroy
}

function load {
validate
image=$1
nodes=$(KUBECONFIG=$RUN_PATH/kubeconfig kubectl get nodes -o json | jq -r '.items[] | .status | .addresses[] | select(.type == "ExternalIP") |.address')
for node in $(echo $nodes); do
echo load $image to $node
docker save $image | bzip2 |pv| ssh -oStrictHostKeyChecking=no ec2-user@$node 'docker load'
done
}

while [[ $# -gt 0 ]]
do
key="$1"

case $key in
create)
apply
exit 0
;;
destroy)
destroy
exit 0
;;
kubectl)
KUBECONFIG=$RUN_PATH/kubeconfig kubectl ${@:2}
exit $?
;;
load)
load $2
exit $?
;;
-h|--help)
print_usage
exit 0
;;
*) # unknown option
echo "Unknown option $1"
exit 1
;;
esac
done
1 change: 1 addition & 0 deletions hack/terraform/eks/README.md
@@ -0,0 +1 @@
imported from https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/basic with some changes.
125 changes: 125 additions & 0 deletions hack/terraform/eks/main.tf
@@ -0,0 +1,125 @@
terraform {
required_version = ">= 0.12.0"
}

provider "aws" {
version = ">= 2.28.1"
region = var.region
}

provider "random" {
version = "~> 2.1"
}

provider "local" {
version = "~> 1.2"
}

provider "null" {
version = "~> 2.1"
}

provider "template" {
version = "~> 2.1"
}

data "aws_eks_cluster" "cluster" {
name = module.eks.cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
name = module.eks.cluster_id
}

provider "kubernetes" {
load_config_file = false
host = data.aws_eks_cluster.cluster.endpoint
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
token = data.aws_eks_cluster_auth.cluster.token
# eks module not yet compatible with newer kubenetes provider 1.11.
version = "1.10"
}

data "aws_availability_zones" "available" {
}

locals {
cluster_name = "test-eks-${random_string.suffix.result}"
}

resource "random_string" "suffix" {
length = 8
special = false
}

resource "aws_security_group" "all_worker_mgmt" {
name_prefix = "all_worker_management"
vpc_id = module.vpc.vpc_id

ingress {
from_port = var.eks_open_port_begin
to_port = var.eks_open_port_end
protocol = "tcp"

cidr_blocks = [
"0.0.0.0/0",
"172.16.0.0/12",
"192.168.0.0/16",
]
}
}

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "2.21.0"

name = "test-vpc"
cidr = "10.0.0.0/16"
azs = data.aws_availability_zones.available.names
public_subnets = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]
enable_nat_gateway = false
enable_dns_hostnames = true

tags = {
"kubernetes.io/cluster/${local.cluster_name}" = "shared"
}

public_subnet_tags = {
"kubernetes.io/cluster/${local.cluster_name}" = "shared"
"kubernetes.io/role/elb" = "1"
}
}

module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = local.cluster_name
subnets = module.vpc.public_subnets

tags = {
Environment = "test"
GithubRepo = "terraform-aws-eks"
GithubOrg = "terraform-aws-modules"
}

vpc_id = module.vpc.vpc_id

worker_groups = [
{
name = "worker-group"
instance_type = var.eks_worker_type
additional_userdata = "echo foo bar"
asg_desired_capacity = var.eks_worker_count
public_ip = true
iam_instance_profile_name = var.eks_iam_instance_profile_name
key_name = var.eks_key_pair_name
},
]

worker_additional_security_group_ids = [aws_security_group.all_worker_mgmt.id]
map_roles = var.map_roles
map_users = var.map_users
map_accounts = var.map_accounts
manage_cluster_iam_resources = false
cluster_iam_role_name = var.eks_cluster_iam_role_name
manage_worker_iam_resources = false
}
25 changes: 25 additions & 0 deletions hack/terraform/eks/outputs.tf
@@ -0,0 +1,25 @@
output "cluster_endpoint" {
description = "Endpoint for EKS control plane."
value = module.eks.cluster_endpoint
}

output "cluster_security_group_id" {
description = "Security group ids attached to the cluster control plane."
value = module.eks.cluster_security_group_id
}

output "kubectl_config" {
description = "kubectl config as generated by the module."
value = module.eks.kubeconfig
}

output "config_map_aws_auth" {
description = "A kubernetes configuration to authenticate to this EKS cluster."
value = module.eks.config_map_aws_auth
}

output "region" {
description = "AWS region."
value = var.region
}

0 comments on commit e7d8502

Please sign in to comment.