Amazon EKS scaling with KEDA and Karpenter

EKS with KEDA HPA & Karpenter cluster autoscaler

This repository contains the necessary files and instructions to deploy and configure KEDA (Kubernetes-based Event Driven Autoscaling) and Karpenter (Kubernetes Node Autoscaler) on an Amazon Elastic Kubernetes Service (EKS) cluster.

KEDA enables autoscaling of Kubernetes pods based on the number of events in event sources such as Azure Service Bus, RabbitMQ, Kafka, and more. Karpenter is a Kubernetes node autoscaler that scales the number of nodes in your cluster based on resource usage.

---

Sample Usecase

Full Demo is available @Youtube

Prerequisites

Before you begin, ensure that you have the following prerequisites:

• An active AWS account.
• Kubernetes command-line tool (kubectl) installed.
• eksctl
• AWS CLI
• kubectl
• helm
• K9s

Installation

To install KEDA and Karpenter on your AWS EKS cluster, follow these steps:

1. Clone this repository to your local machine or download it as a ZIP file.

   git clone https://github.com/aws-samples/amazon-eks-scaling-with-keda-and-karpenter.git

2. Navigate to the repository's directory.

   cd amazon-eks-scaling-with-keda-and-karpenter

3. Update environmentVariables.sh file in /deployment

Variable Name	Description
AWS_REGION	The AWS region.
ACCOUNT_ID	The AWS account ID.
TEMPOUT	Temporary output file. This used to temp. store CFN for karpenter
DYNAMODB_TABLE	The name of the DynamoDB table.
CLUSTER_NAME	The name of the EKS cluster.
KARPENTER_VERSION	The version of Karpenter.
NAMESPACE	The Kubernetes namespace for KEDA.
SERVICE_ACCOUNT	The Kubernetes service account for KEDA.
IAM_KEDA_ROLE	The IAM role for KEDA.
IAM_KEDA_SQS_POLICY	The IAM policy for KEDA to access SQS.
IAM_KEDA_DYNAMO_POLICY	The IAM policy for KEDA to access DynamoDB.
SQS_QUEUE_NAME	The name of the SQS queue.
SQS_QUEUE_URL	The URL of the SQS queue.
SQS_TARGET_DEPLOYMENT	The target deployment for KEDA to scale based on SQS messages.
SQS_TARGET_NAMESPACE	The target namespace for the deployment that KEDA scales based on SQS messages.

4. To strat deployment run

   sh ./deployment/_main.sh

5. You will be asked to verfiy the account in context

6. Select your deployment option

🚨🚨 Experiencing Issues 🚨🚨

If you are experiencing error while deploying Karpenter, it can be due to Karpenter repository currently supports limited version and you may be havign old version in environmentVariables.sh. Please follow steps below:

• Please check latest karpenter version from https://karpenter.sh -> Getting Started. environmentVariables should match with supported version

• Check file createkarpenter.sh under deployment-->karpenter line 80 - 90 and compare with karpneter getting started helm https://karpenter.sh/

Mock Testing 💻

I have used iTerm2 on Mac with K9s to connect with cluster and run scripts

Architecture

1. Once Cluster, Karpenter & KEDA is deployed

2. Open two more terminal and connect to cluster using K9s (check Prerequisites)

3. In first terminal with K9s, navigate to keda-test namespace pod view

4. In seconds terminal with k9s, navigate to Nodes

5. Open three or more terminals, copy content from deployment/environmentVariables.sh and run it on all three terminal

6. In the terminal run below command to test Karpenter status. If you see error first fix it before moving furthe

kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

7. Run keda-mock-sqs-post.py script on all four terminals

   cd app/keda
   python3 -m venv env
   source env/bin/activate
   pip install boto3
   cd  {your path}/amazon-eks-scaling-with-keda-and-karpenter
   python3 ./app/keda/keda-mock-sqs-post.py

8. Pod sclaing using KEDA

9. Node scaling using Karpenter

🚨 Cleanup 🚨

1. Navigate to the repository's root directory.

   cd amazon-eks-scaling-with-keda-and-karpenter

2. To strat cleanup

   sh ./cleanup.sh

This will delelte all the AWS services and workloads created for the solution.

Configuration

The repository contains the necessary configuration files for deploying KEDA and Karpenter. You can modify these files to suit your specific requirements. Here are some important files to note:

• /deploy/keda: Contains the deployment files for KEDA components.
• /deploy/karpenter: Contains the deployment files for Karpenter components.

Feel free to modify these files according to your needs.

Acknowledgements

• KEDA - Kubernetes-based Event Driven Autoscaling
• Karpenter - Kubernetes Node Autoscaler

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file..