Skip to content

Upgrade EKS to Kubernetes 1.29 #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
averyjennings wants to merge 1 commit into
base: main
Choose a base branch
from
+2 −2
Open

Upgrade EKS to Kubernetes 1.29 #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion modules/eks/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ variable "cluster_name" {
variable "kubernetes_version" {
description = "Kubernetes version"
type = string
default = "1.28"
default = "1.29"
}

variable "vpc_id" {
Expand Down
2 changes: 1 addition & 1 deletion prod/eks/terragrunt.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ dependency "vpc" {

inputs = {
cluster_name = "prod-eks-cluster"
kubernetes_version = "1.28"
kubernetes_version = "1.29"
Copy link

@coderabbitai coderabbitai bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Verify no workloads use deprecated Kubernetes APIs before upgrading.

Based on external tools context: Kubernetes 1.29 includes breaking changes - the deprecated flowcontrol.apiserver.k8s.io/v1beta2 API version of FlowSchema and PriorityLevelConfiguration is no longer served. Any existing manifests or client software using this deprecated API must be updated before the upgrade, or the upgrade will break those workloads.

Run the following script to check for usage of deprecated APIs in the cluster:

#!/bin/bash
# Description: Check for deprecated API usage in Kubernetes manifests and cluster resources
# Note: This requires kubectl access to the prod cluster

echo "=== Checking for deprecated flowcontrol API usage in manifests ==="
# Search for deprecated API in local manifests/helm charts
rg -n "flowcontrol\.apiserver\.k8s\.io/v1beta2" --type=yaml -C3

echo ""
echo "=== Checking cluster for deprecated API usage (requires kubectl) ==="
echo "Run these commands with cluster access:"
echo "kubectl get flowschemas.v1beta2.flowcontrol.apiserver.k8s.io 2>&1"
echo "kubectl get prioritylevelconfigurations.v1beta2.flowcontrol.apiserver.k8s.io 2>&1"

Based on learnings from external tools: The deprecated API must be migrated before the upgrade can proceed safely.

🤖 Prompt for AI Agents 
In prod/eks/terragrunt.hcl around line 19, the kubernetes_version bump to 1.29
will drop flowcontrol.apiserver.k8s.io/v1beta2 support; run the provided
detection commands against local manifests and the cluster to find any usages,
then update any FlowSchema and PriorityLevelConfiguration manifests and Helm
charts to the v1 API (and corresponding field changes) or remove them if
obsolete, update client libraries or operators that emit v1beta2, apply and
validate the converted resources in a staging cluster, and only then proceed
with the K8s version upgrade.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🔍 Additional research query:

Search Linear for EKS Version Management Policy and kubernetes version 1.29 upgrade requirements

💡 Result:

No relevant context found from external knowledge sources.

🔍 Additional research query:

Search Notion for EKS Version Management Policy, Kubernetes upgrade policies, or CAB approval records

💡 Result:

Search Results

I found the following documents in your Notion workspace related to your search:

EKS Version Management Policy

Document Details:

  • Owner: Platform Engineering Team
  • Last Updated: October 2024
  • Status: Active

Key Information:

  • Current Approved Version: Kubernetes 1.28 LTS
  • Production Requirement: All production EKS clusters must run Kubernetes 1.28 until Q2 2025

Version Matrix:

Environment Approved Version Upgrade Window
Development 1.28 - 1.29 Anytime
Staging 1.28 After dev validation
Production 1.28 only Q2 2025 (planned)

Upgrade Process Requirements:
The policy specifies that any production Kubernetes version upgrade requires:

  1. Platform Team approval
  2. Security review sign-off
  3. 2-week staging validation period
  4. Change Advisory Board (CAB) approval ✓
  5. Scheduled maintenance window

Other Related Documents Found:

  • EKS Auto-Scaling Policy - Defines auto-scaling boundaries for EKS node groups across environments in coordination with FinOps
  • AWS Instance Type Standards - Includes Graviton (ARM64) First Policy (Owner: Cloud Infrastructure Team, Last Updated: November 2024)

🏁 Script executed:

# Explore the repository structure to understand the codebase better
git ls-files | head -20

Repository: coderabbitai/rabbits-playground

Length of output: 572

🏁 Script executed:

# Check the actual content of the file to understand context
head -30 prod/eks/terragrunt.hcl

Repository: coderabbitai/rabbits-playground

Length of output: 875

🏁 Script executed:

# Search for any policy documents or approval requirements in the repo
find . -type f -name "*policy*" -o -name "*approval*" -o -name "*VERSION*" | grep -v ".git"

Repository: coderabbitai/rabbits-playground

Length of output: 57

🏁 Script executed:

# Look for any documentation about K8s version requirements
fd -t f -e md -e txt | xargs rg -l "kubernetes.*1\.(28|29)|version.*management" 2>/dev/null | head -10

Repository: coderabbitai/rabbits-playground

Length of output: 57

🔍 Additional research query:

Search Linear for policy exceptions, Kubernetes 1.29 upgrade, CAB approval, or prod-eks-cluster related issues

💡 Result:

No relevant context found from external knowledge sources.

🏁 Script executed:

# Check git history to see if there are related commits or PRs
git log --oneline --all -- prod/eks/terragrunt.hcl | head -10

Repository: coderabbitai/rabbits-playground

Length of output: 136

🏁 Script executed:

# Look for any approval or exception documentation in the repo
rg -i "exception|approved.*1\.29|kubernetes.*upgrade" --type md --type txt 2>/dev/null | head -20

Repository: coderabbitai/rabbits-playground

Length of output: 57

🏁 Script executed:

# Check if there are comments or documentation in the file itself
cat -n prod/eks/terragrunt.hcl | head -40

Repository: coderabbitai/rabbits-playground

Length of output: 1135

CRITICAL: This upgrade violates the EKS Version Management Policy.

Your organization's EKS Version Management Policy (Platform Engineering Team, Active) explicitly requires all production EKS clusters to run Kubernetes 1.28 until Q2 2025. This change to 1.29 in prod-eks-cluster directly conflicts with that policy.

The documented upgrade process mandates:

  • Platform Team approval
  • Security review sign-off
  • 2-week staging validation period
  • Change Advisory Board (CAB) approval
  • Scheduled maintenance window

No evidence of these approvals exists in the PR.

Actions required:

  1. Obtain formal policy exception with documented stakeholder approval if this upgrade is authorized ahead of Q2 2025
  2. Provide evidence of required approvals from Platform Team, Security, and CAB
  3. Confirm staging validation — has 1.29 been running successfully in dev/staging for 2+ weeks?

vpc_id = dependency.vpc.outputs.vpc_id
private_subnet_ids = dependency.vpc.outputs.private_subnet_ids
public_subnet_ids = dependency.vpc.outputs.public_subnet_ids
Expand Down