Skip to content

Latest commit

 

History

History

11-Kubernetes-Resources-via-Terraform

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
02-k8sresources-terraform-manifests
02-k8sresources-terraform-manifests
 
 
README.md
README.md
 
 

README.md

title description
Kubernetes Resources using Terraform
Create Kubernetes Resources using Terraform Kubernetes Provider

Step-01: Introduction

  1. Kubernetes Terraform Provider
  2. Kubernetes Resources using Terraform
    1. Kubernetes Deployment Resource
    2. Kubernetes LoadBalancer Service Resource
    3. Kubernetes NodePort Service Resource
  3. Terraform Remote State Datasource Concept
  4. Terraform State sharing across multiple projects which uses local backend
  5. Terraform Backends Concept

Step-02: Review the EKS Cluster Resouces

  • Folder: 08-AWS-EKS-Cluster-Basics/01-ekscluster-terraform-manifests
  • No changes from previous section

Step-03: c1-versions.tf

# Terraform Settings Block
terraform {
  required_version = ">= 1.0.0"
  required_providers {
    aws = {
      source = "hashicorp/aws"
      #version = "~> 3.70"
      version = ">= 4.65"
    }
    kubernetes = {
      source = "hashicorp/kubernetes"
      #version = "~> 2.7"
      version = ">= 2.20"
    }    
  }
}

Step-04: c2-remote-state-datasource.tf

  • Terraform Remote State Datasource
  • Folder: 02-k8sresources-terraform-manifests
  • Important Note: We will use the Terraform State file terraform.tfstate file from ekscluster Terraform project to get the EKS Resources information
# Terraform Remote State Datasource
data "terraform_remote_state" "eks" {
  backend = "local"

  config = {
    path = "../../08-AWS-EKS-Cluster-Basics/01-ekscluster-terraform-manifests/terraform.tfstate"
  }
}

Step-05: c3-providers.tf

  • Terraform Kubernetes Provider
  • Folder: 02-k8sresources-terraform-manifests
  • Define AWS Provider and Kubernetes Provider
  • Also define the Terraform Datasources required to access required data.
# Terraform AWS Provider Block
provider "aws" {
  region = "us-east-1"
}

data "aws_eks_cluster" "cluster" {
  name = data.terraform_remote_state.eks.outputs.cluster_id
}

data "aws_eks_cluster_auth" "cluster" {
  name = data.terraform_remote_state.eks.outputs.cluster_id
}

# Terraform Kubernetes Provider
provider "kubernetes" {
  host                   = data.terraform_remote_state.eks.outputs.cluster_endpoint
  cluster_ca_certificate = base64decode(data.terraform_remote_state.eks.outputs.cluster_certificate_authority_data)
  token                  = data.aws_eks_cluster_auth.cluster.token
}

Step-06: c4-kubernetes-deployment.tf

# Kubernetes Deployment Manifest
resource "kubernetes_deployment_v1" "myapp1" {
  metadata {
    name = "myapp1-deployment"
    labels = {
      app = "myapp1"
    }
  }

  spec {
    replicas = 2
    selector {
      match_labels = {
        app = "myapp1"
      }      
    }
    template {
      metadata {
        labels = {
          app = "myapp1"
        }
      }
      spec {
        container {
          image = "stacksimplify/kubenginx:1.0.0"
          name  = "myapp1-container"

          port {
            container_port = 80
          }
        }
      }
    }
  }
}

Step-07: c5-kubernetes-loadbalancer-service.tf

# Kubernetes Service Manifest (Type: Load Balancer)
resource "kubernetes_service_v1" "lb_service" {
  metadata {
    name = "myapp1-lb-service-clb"
  }
  spec {
    selector = {
      #app = kubernetes_deployment_v1.myapp1.spec.0.template.0.metadata[0].labels.app
      app = kubernetes_deployment_v1.myapp1.spec.0.selector.0.match_labels.app
    }
    port {
      port        = 80
      target_port = 80
    }

    type = "LoadBalancer"
  }
}

Step-08: c6-kubernetes-nodeport-service.tf

  • Folder: 02-k8sresources-terraform-manifests
# Kubernetes Service Manifest (Type: Node Port Service)
resource "kubernetes_service_v1" "np_service" {
  metadata {
    name = "myapp1-nodeport-service"
  }
  spec {
    selector = {
      app = kubernetes_deployment_v1.myapp1.spec.0.selector.0.match_labels.app      
    }
    port {
      port        = 80
      target_port = 80
      node_port = 31280
    }

    type = "NodePort"
  }
}

Step-09: c7-kubernetes-loadbalancer-service-nlb.tf

# Kubernetes Service Manifest (Type: Load Balancer)
resource "kubernetes_service_v1" "lb_service_nlb" {
  metadata {
    name = "myapp1-lb-service-nlb"
    annotations = {
      "service.beta.kubernetes.io/aws-load-balancer-type" = "nlb"    # To create Network Load Balancer
    }    
  }
  spec {
    selector = {
      app = kubernetes_deployment_v1.myapp1.spec.0.selector.0.match_labels.app      
    }
    port {
      port        = 80
      target_port = 80
    }

    type = "LoadBalancer"
  }
}

Step-10: Create Kubernetes Resources: Execute Terraform Commands

# Change Directroy
cd 11-Kubernetes-Resources-via-Terraform/02-k8sresources-terraform-manifests

# Terraform Initialize
terraform init

# Terraform Validate
terraform validate

# Terraform plan
terraform plan

# Terraform Apply
terraform apply -auto-approve

Step-11: Verify Kubernetes Resources

# List Nodes
kubectl get nodes -o wide

# List Pods
kubectl get pods -o wide
Observation: 
1. Both app pod should be in Public Node Group 

# List Services
kubectl get svc
kubectl get svc -o wide
Observation:
1. We should see both Load Balancer Service and NodePort service created

# Access Sample Application on Browser
http://<CLB-DNS-NAME>
http://<NLB-DNS-NAME>
http://abb2f2b480148414f824ed3cd843bdf0-805914492.us-east-1.elb.amazonaws.com

Step-12: Verify Kubernetes Resources via AWS Management console

  1. Go to Services -> EC2 -> Load Balancing -> Load Balancers
  2. Verify Tabs
    • Description: Make a note of LB DNS Name
    • Instances
    • Health Checks
    • Listeners
    • Monitoring

Step-13: Node Port Service Port - Update/Verify Node Security Group

  • Important Note: This is not a recommended option to update the Node Security group to open ports to internet, but just for learning and testing we are doing this.
  • Go to Services -> Instances -> Find Private Node Group Instance -> Click on Security Tab
  • Find the Security Group with name eks-remoteAccess-
  • Go to the Security Group (Example Name: sg-027936abd2a182f76 - eks-remoteAccess-d6beab70-4407-dbc7-9d1f-80721415bd90)
  • Add an additional Inbound Rule
    • Type: Custom TCP
    • Protocol: TCP
    • Port range: 31280
    • Source: Anywhere (0.0.0.0/0)
    • Description: NodePort Rule
  • Click on Save rules

Step-14: Verify by accessing the Sample Application using NodePort Service

# List Nodes
kubectl get nodes -o wide
Observation: Make a note of the Node External IP

# List Services
kubectl get svc
Observation: Make a note of the NodePort service port "myapp1-nodeport-service" which looks as "80:31280/TCP"

# Access the Sample Application in Browser
http://<EXTERNAL-IP-OF-NODE>:<NODE-PORT>
http://54.165.248.51:31280

Step-15: Remove Inbound Rule added

  • Go to Services -> Instances -> Find Private Node Group Instance -> Click on Security Tab
  • Find the Security Group with name eks-remoteAccess-
  • Go to the Security Group (Example Name: sg-027936abd2a182f76 - eks-remoteAccess-d6beab70-4407-dbc7-9d1f-80721415bd90)
  • Remove the NodePort Rule which we added.

Step-16: Clean-Up

# Delete Kubernetes  Resources
cd 02-k8sresources-terraform-manifests
terraform apply -destroy -auto-approve
rm -rf .terraform* terraform.tfstate*

# Delete EKS Cluster
cd 08-AWS-EKS-Cluster-Basics/01-ekscluster-terraform-manifests/
terraform apply -destroy -auto-approve
rm -rf .terraform* terraform.tfstate*