A Digital Transformation framework for scaling containerized microservices with Kubernetes and AWS EKS
The fleet of Kubernetes worker nodes scales up and down in response to demand.
The code is based in part on commit 61bee0b7858bbcd3d4276f186cc4cc7bf298ac11 from the ModusCreateOrg/devops-infra-demo repository.
To run the demo end to end, you will need:
Optionally, you can use Jenkins to orchestrate creation of AWS resources in conjunction with GitHub branches and pull requests.
You will also need to set a few environment variables. The method of doing so will vary from platform to platform.
AWS_PROFILE AWS_DEFAULT_PROFILE AWS_DEFAULT_REGION AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY
A sample file is provided as a template to customize:
cp env.sh.sample env.sh vim env.sh . env.sh
The AWS profile IAM user should have full control of EC2 in the account you are using.
Jenkinsfile is provided that will allow Jenkins to execute Terraform. In order for Jenkins to do this, it needs to have AWS credentials set up, preferably through an IAM role, granting full control of EC2 and VPC resources in that account. Terraform needs this to create a VPC and EC2 resources. This could be pared down further through some careful logging and role work.
An ECR repository to store the images.
The command line utility 'jq'.
curl -o /usr/local/bin/aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/aws-iam-authenticator chmod 755 /usr/local/bin/aws-iam-authenticator
If 'aws-iam-authenticator' isn't installed, prep.sh will install it from the AWS repository.
curl -o /usr/local/bin/kubectl https://amazon-eks.s3-us-west-2.amazonaws.com/1.11.5/2018-12-06/bin/linux/amd64/kubectl chmod 755 /usr/local/bin/kubectl
If 'kubectl' isn't installed, prep.sh will install it from the AWS repository.
brew install kubernetes-helm
tiller: Run these commands to install the tiller cli:
cd /tmp sudo helm init
This Terraform setup stores its state in Amazon S3 and uses DynamoDB for locking. There is a bit of setup required to bootstrap that configuration. You can use this repository to use Terraform to do that bootstrap process. The
backend.tfvars file in that repo should be modified as follows to work with this project:
(Replace us-east-1 and XXXXXXXXXXXX with the AWS region and your account ID)
bucket = "tf-state.k8s-eks-scaling-demo.us-east-1.XXXXXXXXXXXX" dynamodb_table = "TerraformStatelock-k8s-eks-scaling-demo" key = "terraform.tfstate" profile = "terraform" region = "us-east-1"
You'll also need to modify the list of operators who can modify the object in the S3 bucket. Put in the IAM user names of the user into the
setup/variables.tf file in that project. If your Jenkins instance uses an IAM role to grant access, give it a similar set of permissions to those granted on in the bucket policy to IAM users.
These commands will then set up cloud resources using terraform:
cd terraform terraform init terraform get # Example with values from our environment (replace with values from your environment) # terraform plan -var domain=modus.app -out tf.plan terraform plan -out tf.plan -var 'domain=example.net' terraform apply tf.plan # check to see if everything worked - use the same variables here as above terraform destroy -var 'domain=example.net'
This assumes that you already have a Route 53 domain in your AWS account created.
You need to either edit variables.tf to match your domain and AWS zone or specify these values as command line
At any time you can enter "./bin/build.sh help" for the available commands.
./bin/build.sh stand-up-demo Connect a browser to test these endpoints: http://eks-demo-webapp.moduscreate.com http://eks-demo-spin.moduscreate.com http://eks-demo-spin.moduscreate.com/api/spin
./bin/build.sh proxy-dashboard Follow the onscreen instruction, connect a browser to the dashboard, and login with the token.
./bin/build.sh run-jmeter-www webapp
./bin/build.sh run-jmeter-www spin
Useful commands: ./bin/build.sh kubeconfig (run this first to update the config in your home directory) kubectl top node kubectl top pod kubectl get hpa kubectl get deployments kubectl get rs kubectl get pods kubectl describe deployments kubectl scale deployment.v1.apps/k8s-dev-spin --replicas=10 kubectl get deployment metrics-server -n kube-system kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" kubectl run -i --tty load-generator --image=busybox /bin/sh kubectl -n metrics logs -l app=metrics-server
- Show the PODs and the Nodes increasing in the dashboard.
- Explain the request rates and response times in the jmeter windows
- The ECR repositories are not currently created by Terraform. Depending on the goals of the demo they could be managed by Terraform.
- Run './bin/build.sh help' for help on building applications.
- For EKS to report CPU usage to the metrics server, the 'kubectl run' command needs a cpu limit applied: EG: "--limits=cpu=200m,memory=512Mi"
Modus Create is a digital product consultancy. We use a distributed team of the best talent in the world to offer a full suite of digital product design-build services; ranging from consumer facing apps, to digital migration, to agile development training, and business transformation.
This project is part of Modus Labs.
This project is MIT licensed.
The content in
application is adapted from Dimension by https://html5up.net/ and is licensed under a Creative Commons Attribution 3.0 License See its README.md and LICENSE.md files for more details.