Skip to content

WhisperNet/eks-with-tf

BranchesTags

Repository files navigation

EKS Cluster with Terraform

A complete Infrastructure as Code (IaC) example for deploying an Amazon EKS (Elastic Kubernetes Service) cluster using Terraform. This repository serves as a learning reference for DevOps practices.

Table of Contents

Overview

This project provisions a production-ready EKS cluster on AWS with:

Component Description
VPC Custom VPC with public/private subnets across 3 AZs
EKS Managed Kubernetes cluster (v1.33)
Node Groups EKS-managed node group with auto-scaling (1-2 nodes)
Networking NAT Gateway, Internet Gateway, proper subnet tagging
Add-ons CoreDNS, kube-proxy, VPC-CNI pre-installed

Why This Setup?

  • Security: Worker nodes in private subnets, only accessible via NAT
  • Scalability: Auto-scaling node groups with configurable min/max
  • Cost-Effective: Single NAT gateway for development environments
  • Best Practices: Uses official Terraform AWS modules

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              AWS Cloud (us-east-1)                          │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                         VPC: 10.10.0.0/16                             │  │
│  │                                                                       │  │
│  │   ┌─────────────────┐                                                 │  │
│  │   │ Internet Gateway│                                                 │  │
│  │   └────────┬────────┘                                                 │  │
│  │            │                                                          │  │
│  │   ┌────────┴────────────────────────────────────────────────────┐     │  │
│  │   │                    Public Subnets                           │     │  │
│  │   │  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐        │     │  │
│  │   │  │ 10.10.4.0/24 │ │ 10.10.5.0/24 │ │ 10.10.6.0/24 │        │     │  │
│  │   │  │    (AZ-a)    │ │    (AZ-b)    │ │    (AZ-c)    │        │     │  │
│  │   │  └──────────────┘ └──────────────┘ └──────────────┘        │     │  │
│  │   │         │              │              │                     │     │  │
│  │   │         └──────────────┼──────────────┘                     │     │  │
│  │   │                        │                                    │     │  │
│  │   │                 ┌──────┴──────┐                             │     │  │
│  │   │                 │ NAT Gateway │                             │     │  │
│  │   │                 └──────┬──────┘                             │     │  │
│  │   └────────────────────────┼────────────────────────────────────┘     │  │
│  │                            │                                          │  │
│  │   ┌────────────────────────┴────────────────────────────────────┐     │  │
│  │   │                    Private Subnets                          │     │  │
│  │   │  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐        │     │  │
│  │   │  │ 10.10.1.0/24 │ │ 10.10.2.0/24 │ │ 10.10.3.0/24 │        │     │  │
│  │   │  │    (AZ-a)    │ │    (AZ-b)    │ │    (AZ-c)    │        │     │  │
│  │   │  └──────┬───────┘ └──────┬───────┘ └──────┬───────┘        │     │  │
│  │   │         │                │                │                 │     │  │
│  │   │  ┌──────┴────────────────┴────────────────┴──────┐         │     │  │
│  │   │  │           EKS Managed Node Group              │         │     │  │
│  │   │  │  ┌─────────┐  ┌─────────┐                     │         │     │  │
│  │   │  │  │  Node   │  │  Node   │   (t2.small)        │         │     │  │
│  │   │  │  │   1     │  │   2     │   min:1, max:2      │         │     │  │
│  │   │  │  └─────────┘  └─────────┘                     │         │     │  │
│  │   │  └───────────────────────────────────────────────┘         │     │  │
│  │   └─────────────────────────────────────────────────────────────┘     │  │
│  │                                                                       │  │
│  │   ┌───────────────────────────────────────────────────────────────┐   │  │
│  │   │                    EKS Control Plane                          │   │  │
│  │   │    ┌─────────┐    ┌───────────┐    ┌──────────┐              │   │  │
│  │   │    │ CoreDNS │    │ kube-proxy│    │ VPC-CNI  │              │   │  │
│  │   │    └─────────┘    └───────────┘    └──────────┘              │   │  │
│  │   │                  (AWS Managed)                                │   │  │
│  │   └───────────────────────────────────────────────────────────────┘   │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────────┘

Prerequisites

Before deploying, ensure you have:

  • Terraform >= 1.14.x
  • AWS CLI configured with appropriate credentials
  • kubectl for cluster interaction
  • AWS IAM user/role with permissions for EKS, VPC, EC2, and IAM

Verify installations:

terraform version   # Should show >= 1.14.x
aws --version       # Should show AWS CLI
kubectl version     # Should show kubectl client
aws sts get-caller-identity   # Verify AWS credentials

Project Structure

.
├── providers.tf      # Terraform & AWS provider configuration
├── vpc.tf            # VPC, subnets, NAT gateway, and networking
├── eks-cluster.tf    # EKS cluster and node group configuration
├── terraform.tfstate # State file (auto-generated, do not edit)
└── README.md         # Documentation

Configuration Deep Dive

1. Providers (providers.tf)

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "6.26.0"
    }
  }
}

What it does:

  • Declares Terraform's required AWS provider
  • Pins the AWS provider to version 6.26.0 for reproducibility

Why version pinning matters:

  • Prevents unexpected breaking changes
  • Ensures consistent deployments across team members
  • Critical for production environments

2. VPC (vpc.tf)

This file creates the networking foundation for EKS.

Provider & Variables

provider "aws" {
  region = "us-east-1"
}

variable vpc_cidr_block {
    default = "10.10.0.0/16"
}
variable private_subnet_cidr_blocks {
    default = ["10.10.1.0/24","10.10.2.0/24","10.10.3.0/24"]
}
variable public_subnet_cidr_blocks {
    default = ["10.10.4.0/24","10.10.5.0/24","10.10.6.0/24"]
}

VPC Module

module "myapp-vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "6.5.1"

  name = "my-tf-eks-vpc"
  cidr = var.vpc_cidr_block

  # Subnets spread across availability zones
  private_subnets = var.private_subnet_cidr_blocks
  public_subnets  = var.public_subnet_cidr_blocks
  azs = data.aws_availability_zones.azs.names

  # NAT Configuration
  enable_nat_gateway   = true
  single_nat_gateway   = true    # Cost optimization for dev
  enable_dns_hostnames = true

  # Kubernetes-specific tags (required for ELB auto-discovery)
  tags = {
    "kubernetes.io/cluster/my-tf-eks-cluster" = "shared"
  }

  public_subnet_tags = {
    "kubernetes.io/cluster/my-tf-eks-cluster" = "shared"
    "kubernetes.io/role/elb" = 1   # External load balancers
  }

  private_subnet_tags = {
    "kubernetes.io/cluster/my-tf-eks-cluster" = "shared"
    "kubernetes.io/role/internal-elb" = 1   # Internal load balancers
  }
}

Key Concepts:

Configuration Purpose
single_nat_gateway = true Uses 1 NAT instead of 3 (saves ~$64/month in dev)
enable_dns_hostnames Required for EKS nodes to communicate
kubernetes.io/role/elb Tells K8s where to place public ALB/NLB
kubernetes.io/role/internal-elb Tells K8s where to place internal ALB/NLB

Subnet Layout:

VPC CIDR: 10.10.0.0/16 (65,536 IPs)
│
├── Private Subnets (for EKS worker nodes)
│   ├── 10.10.1.0/24 → AZ-a (254 IPs)
│   ├── 10.10.2.0/24 → AZ-b (254 IPs)
│   └── 10.10.3.0/24 → AZ-c (254 IPs)
│
└── Public Subnets (for load balancers, bastion hosts)
    ├── 10.10.4.0/24 → AZ-a (254 IPs)
    ├── 10.10.5.0/24 → AZ-b (254 IPs)
    └── 10.10.6.0/24 → AZ-c (254 IPs)

3. EKS Cluster (eks-cluster.tf)

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "21.10.1"

  name               = "my-tf-eks-cluster"
  kubernetes_version = "1.33"

  # Network configuration
  vpc_id     = module.myapp-vpc.vpc_id
  subnet_ids = module.myapp-vpc.private_subnets

  # Access configuration
  endpoint_public_access                   = true
  enable_cluster_creator_admin_permissions = true

  # EKS Add-ons
  addons = {
    coredns    = {}
    kube-proxy = {}
    vpc-cni    = {
      before_compute = true   # Install VPC-CNI before nodes join
    }
  }

  # Managed Node Group
  eks_managed_node_groups = {
    dev = {
      min_size       = 1
      max_size       = 2
      desired_size   = 1
      instance_types = ["t2.small"]
    }
  }

  tags = {
    environment = "development"
    application = "myapp"
  }
}

EKS Add-ons Explained:

Add-on Purpose
CoreDNS Provides DNS resolution within the cluster (service discovery)
kube-proxy Maintains network rules for pod-to-pod communication
VPC-CNI AWS-native networking plugin for pod IP assignment

Why before_compute = true for VPC-CNI?

  • Ensures the CNI plugin is ready before nodes attempt to join
  • Prevents pod scheduling failures during cluster bootstrap

Node Group Configuration:

┌─────────────────────────────────────────┐
│        EKS Managed Node Group: dev      │
├─────────────────────────────────────────┤
│  Instance Type: t2.small                │
│  vCPU: 1    Memory: 2 GiB               │
├─────────────────────────────────────────┤
│  Scaling:                               │
│    Min:     1 node                      │
│    Max:     2 nodes                     │
│    Desired: 1 node                      │
├─────────────────────────────────────────┤
│  Location: Private subnets only         │
└─────────────────────────────────────────┘

Network Architecture

Traffic Flow

External Traffic                         Internal Traffic
      │                                        │
      ▼                                        ▼
┌─────────────┐                      ┌─────────────────┐
│   Internet  │                      │   VPC Internal  │
└──────┬──────┘                      └────────┬────────┘
       │                                      │
       ▼                                      │
┌──────────────┐                              │
│    IGW       │                              │
└──────┬───────┘                              │
       │                                      │
       ▼                                      │
┌─────────────────────┐                       │
│   Public Subnets    │◄──────────────────────┘
│  (External ALB/NLB) │
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│    NAT Gateway      │
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│  Private Subnets    │
│  (EKS Worker Nodes) │
│  (Internal ALB/NLB) │
└─────────────────────┘

Why Private Subnets for Worker Nodes?

  1. Security: Nodes aren't directly accessible from the internet
  2. Compliance: Meets security requirements for many industries
  3. Best Practice: Follows AWS Well-Architected Framework

Deployment

Step 1: Initialize Terraform

terraform init

This downloads required providers and modules.

Step 2: Review the Plan

terraform plan

Review the resources that will be created (~50+ resources).

Step 3: Apply Configuration

terraform apply

Type yes when prompted. Deployment takes approximately 15-20 minutes.

Expected resources created:

  • 1 VPC
  • 6 Subnets (3 public, 3 private)
  • 1 Internet Gateway
  • 1 NAT Gateway
  • Route tables and associations
  • Security groups
  • EKS Cluster
  • EKS Managed Node Group
  • IAM Roles and Policies
  • EKS Add-ons

Connecting to the Cluster

After deployment, configure kubectl:

# Update kubeconfig
aws eks update-kubeconfig --name my-tf-eks-cluster --region us-east-1

# Verify connection
kubectl get nodes

# Check system pods
kubectl get pods -n kube-system

Expected output:

NAME                              STATUS   ROLES    AGE   VERSION
ip-10-10-1-xxx.ec2.internal       Ready    <none>   5m    v1.33.x

Cost Considerations

Resource Approximate Monthly Cost
EKS Control Plane ~$73
NAT Gateway ~$32 + data transfer
t2.small instances (1-2) ~$17-34
Total (Dev) ~$122-139/month

Cost Optimization Tips:

  1. Use single_nat_gateway = true (already configured) for dev
  2. Consider Spot instances for non-production workloads
  3. Scale down to 0 nodes when not in use
  4. Use t3.small instead of t2.small for better performance/cost

Clean Up

To avoid ongoing charges, destroy all resources:

terraform destroy

Type yes when prompted. This removes all created infrastructure.

⚠️ Warning: This will delete the EKS cluster and all workloads running on it.

Learning Resources

Terraform

Amazon EKS

Kubernetes

Next Steps

After deploying this cluster, consider exploring:

  1. Ingress Controllers - Deploy NGINX or AWS ALB Ingress Controller
  2. Monitoring - Set up Prometheus and Grafana
  3. Logging - Configure Fluent Bit to CloudWatch
  4. Secrets Management - Integrate AWS Secrets Manager or HashiCorp Vault
  5. GitOps - Implement ArgoCD or Flux for continuous deployment
  6. Service Mesh - Explore AWS App Mesh or Istio

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages