Kubernetes Deployment of Notejam application on AWS EKS with Terraform

Notejam flask application with postgres DB on RDS. Kubernetes Deployment on AWS EKS by provisioning with Terraform. Github Actions to build docker images to be pushed to AWS ECR.

Architecture:

Prerequisities:

• Docker & Docker-compose
• AWS client Setup
• Terraform
• Kubectl
• Eksctl
• Helm

Development Environment with docker-compose

# update dotenv file under notejam-flask with your credentials and run docker-compose
mv notejam-flask/dotenv notejam-flask/.env
docker-compose --env-file notejam-flask/.env up -d

• notejam application can be visited at port 5000
• You can run tests in docker container:

docker exec -it notejam_web sh
python tests.py

Provisioning AWS EKS with Terraform

Steps

1. Update terraform.tfvars with your own config variables and ssh-key. When ready, initiate terraform:

terraform -chdir=terraform init

2. Make all the scripts executable:

find . -maxdepth 1 -type f -name "*.sh" -exec chmod +x {} \;

3. Create AWS Secret for DB Credentials: ./1_create_db_creds.sh
4. Create Route 53 DNS Zone if not exists: ./2_route53.sh
5. Provision AWS with terraform:

terraform -chdir=terraform apply

• There is a bastion server provisioned as well to login to EKS worker nodes through private network. Login with your ssh-key that you've created:

ssh -i "ssh-key" ubuntu@$(terraform -chdir=terraform output -raw bastion_public_ip)

6. Kubectl config setup: ./3_EKS_setup.sh

• Check that you are able to connect to k8s cluster:

kubectl get nodes

• Check your access to RDS

# Note the db_endpoint and port
terraform -chdir=terraform output db_endpoint

# Run a temporary busybox pod
kubectl run -it --rm busybox --image=busybox --restart=Never -- sh

# Check if you can connect to db_endpoint instance with telnet
telnet db_endpoint port

7. Login to AWS ECR service:

aws ecr get-login-password --region $(terraform -chdir=terraform output -raw aws_region) \
| docker login --username AWS --password-stdin $(terraform -chdir=terraform output -raw registry_backend)

8. Associate EKS cluster with OpenID Connect Provider (Guide): ./4_OIDC_associate.sh
   
   
9. Install cert-manager (Guide) ./5_cert_manager.sh

• make sure that pods are in Running state: kubectl -n cert-manager get pods

10. Install Load Balancer Controller (Guide): ./6_LB_controller.sh
    
    
11. Create cert-manager cluster issuer (Guide and Link): ./7_cluster_issuer.sh
    
    
12. Create Namespaces: kubectl apply -f k8s/namespaces.yaml
    
    
13. Create Secrets for Postgres Credentials (Alternatively, a more robust and secure solution would be to use AWS Secrets Manager service → Guide): ./8_k8s_secrets.sh

• This will be created under 'staging' namespace since I will be deploying the application under this namespace. This should be modified to be used in a CI/CD system.

14. Github actions will build a notejam image and push to AWS ECR after each git push to master branch.

• Setting up the github actions scripts are based on this guide

---

Kubernetes deployment under staging namespace

• Run deploy_k8s_alb.sh to create a deployment and expose it with a k8s service and ALB.
• Import the Key and Cert to ACM and add FQDN to Route53

# Run below script to push the key and cert to ACM in ingress file:
./9_sync_tls_resources.sh

Run below to update DNS record to point to ALB:
./10_route53_recordset.sh