forked from antrea-io/antrea
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added terraform script to create eks.
see docs/cloud.md for setting up AKS/EKS clusters, and install Antrea over them.
- Loading branch information
1 parent
03bf183
commit e7d8502
Showing
7 changed files
with
448 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# Deploying Antrea on a cloud provider | ||
|
||
Antrea may run in networkPolicyOnly mode in some cloud managed clusters. This document describes | ||
steps to create EKS using terraform. | ||
|
||
## Common Prerequisites | ||
1. To run EKS cluster, install and configure AWS cli(either version 1 or 2), see | ||
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html, and | ||
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html | ||
2. Install aws-iam-authenticator, see | ||
https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html | ||
3. Install terraform, see https://learn.hashicorp.com/terraform/getting-started/install.html | ||
4. You must already have ssh key-pair created. This key pair will be used to access worker Node via | ||
ssh. | ||
```bash | ||
ls ~/.ssh/ | ||
id_rsa id_rsa.pub | ||
``` | ||
|
||
|
||
## Create an EKS cluster via terraform | ||
Ensures that you have permission to create EKS cluster, and have already | ||
created EKS cluster role as well as worker Node profile. | ||
```bash | ||
export TF_VAR_eks_cluster_iam_role_name=YOUR_EKS_ROLE | ||
export TF_VAR_eks_iam_instance_profile_name=YOUR_EKS_WORKER_NODE_PROFILE | ||
export TF_VAR_eks_key_pair_name=YOUR_KEY_PAIR_TO_ACCESS_WORKER_NODE | ||
``` | ||
Where | ||
- TF_VAR_eks_cluster_iam_role_name may be created by following these | ||
[instructions](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html#create-service-role) | ||
- TF_VAR_eks_iam_instance_profile_name may be created by following these | ||
[instructions](https://docs.aws.amazon.com/eks/latest/userguide/worker_node_IAM_role.html#create-worker-node-role) | ||
- TF_VAR_eks_key_pair_name is the aws key pair name you have configured by following these | ||
[instructions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws), | ||
using ssh-pair created in Prerequisites item 4 | ||
|
||
|
||
|
||
Create EKS cluster | ||
```bash | ||
./hack/terraform-eks.sh create | ||
``` | ||
Interact with EKS cluster | ||
``` | ||
./hack/terraform-eks.sh kubectl ... // issue kubectl commands to EKS cluster | ||
./hack/terraform-eks.sh load ... // load local built images to EKS cluster | ||
./hack/terraform-eks.sh destroy // destroy EKS cluster | ||
``` | ||
and worker Node can be accessed with ssh via their external IPs. | ||
|
||
Apply Antrea to EKS cluster | ||
```bash | ||
./hack/generate-manifest.sh --encap-mode networkPolicyOnly | ~/terraform/eks kubectl apply -f - | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Copyright 2020 Antrea Authors | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
set -eo pipefail | ||
|
||
mkdir -p $HOME/terraform && cp ./hack/terraform-eks.sh $HOME/terraform/eks && cp -R ./hack/terraform $HOME/terraform |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,150 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Copyright 2020 Antrea Authors | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# NOTE: terraform enviroment variables need to be set | ||
AWS_CLI="aws" | ||
TERRAFORM="terraform" | ||
KUBECTL="kubectl" | ||
AWS_IAM="aws-iam-authenticator" | ||
RUN_PATH="$HOME/tmp/terraform-eks" | ||
if [[ $(uname) == "Darwin" ]]; then | ||
READLINK="greadlink" | ||
else | ||
READLINK="readlink" | ||
fi | ||
CMD_ARRAY=($AWS_CLI $TERRAFORM $KUBECTL $AWS_IAM pv bzip2 jq $READLINK) | ||
ENV_ARRAY=(TF_VAR_eks_cluster_iam_role_name TF_VAR_eks_iam_instance_profile_name TF_VAR_eks_key_pair_name) | ||
|
||
_usage="Usage: $0 : | ||
[create] : create eks cluster | ||
[destroy] : destroy eks cluster | ||
[kubectl] : access eks | ||
[load] : load container image from local machine to eks cluster | ||
[--help|h]" | ||
|
||
function print_help { | ||
echo "Try '$0 --help' for more information." | ||
} | ||
|
||
function print_usage { | ||
echo "$_usage" | ||
} | ||
|
||
function validate { | ||
for cmd in ${CMD_ARRAY[@]}; do | ||
command -v $cmd > /dev/null 2>&1 | ||
if [ $? -ne 0 ]; then | ||
echo $cmd not found, please install first | ||
exit 1 | ||
fi | ||
done | ||
|
||
for env in ${ENV_ARRAY[@]}; do | ||
printenv $env > /dev/null 2>&1 | ||
if [ $? -ne 0 ]; then | ||
echo required enviroment variable $env is not set. | ||
exit 1 | ||
fi | ||
done | ||
|
||
# check for aws permissions | ||
$AWS_CLI eks list-clusters > /dev/null 2>&1 | ||
if [ $? -ne 0 ]; then | ||
echo aws user has no permission to eks | ||
exit 1 | ||
fi | ||
|
||
$AWS_CLI iam list-roles > /dev/null 2>&1 | ||
if [ $? -ne 0 ]; then | ||
echo aws user has no permission to iam | ||
exit 1 | ||
fi | ||
} | ||
|
||
function apply { | ||
validate | ||
CONFIG_PATH=$(dirname $($READLINK -f "$0"))/terraform/eks | ||
mkdir -p $RUN_PATH | ||
if [ $? -ne 0 ]; then | ||
echo $RUN_PATH cannot be created | ||
exit 1 | ||
fi | ||
cp $CONFIG_PATH/*.tf $RUN_PATH | ||
cd $RUN_PATH | ||
$TERRAFORM init | ||
if [ $? -ne 0 ]; then | ||
exit 1 | ||
fi | ||
echo "yes" | $TERRAFORM apply | ||
if [[ $? -ne 0 ]]; then | ||
echo "eks creation failed" | ||
exit 1 | ||
fi | ||
echo eks cluster created. | ||
$TERRAFORM output kubectl_config > $RUN_PATH/kubeconfig | ||
echo run eks kubectl ... to acess it. | ||
} | ||
|
||
function destroy { | ||
validate | ||
if [ ! -d $RUN_PATH ]; then | ||
exit 0 | ||
fi | ||
cd $RUN_PATH | ||
echo "yes" | $TERRAFORM destroy | ||
} | ||
|
||
function load { | ||
validate | ||
image=$1 | ||
nodes=$(KUBECONFIG=$RUN_PATH/kubeconfig kubectl get nodes -o json | jq -r '.items[] | .status | .addresses[] | select(.type == "ExternalIP") |.address') | ||
for node in $(echo $nodes); do | ||
echo load $image to $node | ||
docker save $image | bzip2 |pv| ssh -oStrictHostKeyChecking=no ec2-user@$node 'docker load' | ||
done | ||
} | ||
|
||
while [[ $# -gt 0 ]] | ||
do | ||
key="$1" | ||
|
||
case $key in | ||
create) | ||
apply | ||
exit 0 | ||
;; | ||
destroy) | ||
destroy | ||
exit 0 | ||
;; | ||
kubectl) | ||
KUBECONFIG=$RUN_PATH/kubeconfig kubectl ${@:2} | ||
exit $? | ||
;; | ||
load) | ||
load $2 | ||
exit $? | ||
;; | ||
-h|--help) | ||
print_usage | ||
exit 0 | ||
;; | ||
*) # unknown option | ||
echo "Unknown option $1" | ||
exit 1 | ||
;; | ||
esac | ||
done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
imported from https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/basic with some changes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
terraform { | ||
required_version = ">= 0.12.0" | ||
} | ||
|
||
provider "aws" { | ||
version = ">= 2.28.1" | ||
region = var.region | ||
} | ||
|
||
provider "random" { | ||
version = "~> 2.1" | ||
} | ||
|
||
provider "local" { | ||
version = "~> 1.2" | ||
} | ||
|
||
provider "null" { | ||
version = "~> 2.1" | ||
} | ||
|
||
provider "template" { | ||
version = "~> 2.1" | ||
} | ||
|
||
data "aws_eks_cluster" "cluster" { | ||
name = module.eks.cluster_id | ||
} | ||
|
||
data "aws_eks_cluster_auth" "cluster" { | ||
name = module.eks.cluster_id | ||
} | ||
|
||
provider "kubernetes" { | ||
load_config_file = false | ||
host = data.aws_eks_cluster.cluster.endpoint | ||
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) | ||
token = data.aws_eks_cluster_auth.cluster.token | ||
# eks module not yet compatible with newer kubenetes provider 1.11. | ||
version = "1.10" | ||
} | ||
|
||
data "aws_availability_zones" "available" { | ||
} | ||
|
||
locals { | ||
cluster_name = "test-eks-${random_string.suffix.result}" | ||
} | ||
|
||
resource "random_string" "suffix" { | ||
length = 8 | ||
special = false | ||
} | ||
|
||
resource "aws_security_group" "all_worker_mgmt" { | ||
name_prefix = "all_worker_management" | ||
vpc_id = module.vpc.vpc_id | ||
|
||
ingress { | ||
from_port = var.eks_open_port_begin | ||
to_port = var.eks_open_port_end | ||
protocol = "tcp" | ||
|
||
cidr_blocks = [ | ||
"0.0.0.0/0", | ||
"172.16.0.0/12", | ||
"192.168.0.0/16", | ||
] | ||
} | ||
} | ||
|
||
module "vpc" { | ||
source = "terraform-aws-modules/vpc/aws" | ||
version = "2.21.0" | ||
|
||
name = "test-vpc" | ||
cidr = "10.0.0.0/16" | ||
azs = data.aws_availability_zones.available.names | ||
public_subnets = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"] | ||
enable_nat_gateway = false | ||
enable_dns_hostnames = true | ||
|
||
tags = { | ||
"kubernetes.io/cluster/${local.cluster_name}" = "shared" | ||
} | ||
|
||
public_subnet_tags = { | ||
"kubernetes.io/cluster/${local.cluster_name}" = "shared" | ||
"kubernetes.io/role/elb" = "1" | ||
} | ||
} | ||
|
||
module "eks" { | ||
source = "terraform-aws-modules/eks/aws" | ||
cluster_name = local.cluster_name | ||
subnets = module.vpc.public_subnets | ||
|
||
tags = { | ||
Environment = "test" | ||
GithubRepo = "terraform-aws-eks" | ||
GithubOrg = "terraform-aws-modules" | ||
} | ||
|
||
vpc_id = module.vpc.vpc_id | ||
|
||
worker_groups = [ | ||
{ | ||
name = "worker-group" | ||
instance_type = var.eks_worker_type | ||
additional_userdata = "echo foo bar" | ||
asg_desired_capacity = var.eks_worker_count | ||
public_ip = true | ||
iam_instance_profile_name = var.eks_iam_instance_profile_name | ||
key_name = var.eks_key_pair_name | ||
}, | ||
] | ||
|
||
worker_additional_security_group_ids = [aws_security_group.all_worker_mgmt.id] | ||
map_roles = var.map_roles | ||
map_users = var.map_users | ||
map_accounts = var.map_accounts | ||
manage_cluster_iam_resources = false | ||
cluster_iam_role_name = var.eks_cluster_iam_role_name | ||
manage_worker_iam_resources = false | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
output "cluster_endpoint" { | ||
description = "Endpoint for EKS control plane." | ||
value = module.eks.cluster_endpoint | ||
} | ||
|
||
output "cluster_security_group_id" { | ||
description = "Security group ids attached to the cluster control plane." | ||
value = module.eks.cluster_security_group_id | ||
} | ||
|
||
output "kubectl_config" { | ||
description = "kubectl config as generated by the module." | ||
value = module.eks.kubeconfig | ||
} | ||
|
||
output "config_map_aws_auth" { | ||
description = "A kubernetes configuration to authenticate to this EKS cluster." | ||
value = module.eks.config_map_aws_auth | ||
} | ||
|
||
output "region" { | ||
description = "AWS region." | ||
value = var.region | ||
} | ||
|
Oops, something went wrong.