Skip to content
152 changes: 152 additions & 0 deletions k8s/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# Kubernetes Scope Configuration

This document describes all available configuration variables for Kubernetes scopes and their priority hierarchy.

## Configuration Hierarchy

Configuration variables follow a priority hierarchy:

```
1. Existing Providers - Highest priority
- scope-configurations: Scope-specific configuration
- container-orchestration: Orchestrator configuration
- cloud-providers: Cloud provider configuration
(If there are multiple providers, the order in which they are specified determines priority)
2. Environment Variable (ENV VAR) - Allows override when no provider exists
3. Default value - Fallback when no provider or env var exists
```

**Important Note**: The order of arguments in `get_config_value` does NOT affect priority. The function always respects the order: providers > env var > default, regardless of the order in which arguments are passed.

## Configuration Variables

### Cluster

Configuration for Kubernetes cluster settings.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **K8S_NAMESPACE** | Kubernetes namespace where resources are deployed | `cluster.namespace` |
| **CREATE_K8S_NAMESPACE_IF_NOT_EXIST** | Whether to create the namespace if it doesn't exist | `cluster.create_namespace_if_not_exist` |

### Networking

#### General

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **DOMAIN** | Public domain name for the application | `networking.domain_name` |
| **PRIVATE_DOMAIN** | Private domain name for internal services | `networking.private_domain_name` |
| **USE_ACCOUNT_SLUG** | Whether to use account slug as application domain | `networking.application_domain` |
| **DNS_TYPE** | DNS provider type (route53, azure, external_dns) | `networking.dns_type` |

#### AWS Route53

Configuration specific to AWS Route53 DNS provider. Visible only when `dns_type` is `route53`.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **ALB_NAME** (public) | Public Application Load Balancer name | `networking.balancer_public_name` |
| **ALB_NAME** (private) | Private Application Load Balancer name | `networking.balancer_private_name` |
| **ALB_RECONCILIATION_ENABLED** | Whether ALB reconciliation is enabled | `networking.alb_reconciliation_enabled` |

#### Azure DNS

Configuration specific to Azure DNS provider. Visible only when `dns_type` is `azure`.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **HOSTED_ZONE_NAME** | Azure DNS hosted zone name | `networking.hosted_zone_name` |
| **HOSTED_ZONE_RG** | Azure resource group containing the DNS hosted zone | `networking.hosted_zone_rg` |
| **AZURE_SUBSCRIPTION_ID** | Azure subscription ID for DNS management | `networking.azure_subscription_id` |
| **RESOURCE_GROUP** | Azure resource group for cluster resources | `networking.resource_group` |

**Note:** These variables are obtained from the `scope-configurations` provider and exported for use in Azure DNS workflows.

#### Gateways

Gateway configuration for ingress traffic routing.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **PUBLIC_GATEWAY_NAME** | Public gateway name for ingress | `networking.gateway_public_name` |
| **PRIVATE_GATEWAY_NAME** | Private/internal gateway name for ingress | `networking.gateway_private_name` |

### Deployment

#### General

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **DEPLOY_STRATEGY** | Deployment strategy (rolling or blue-green) | `deployment.deployment_strategy` |
| **DEPLOYMENT_MAX_WAIT_IN_SECONDS** | Maximum wait time for deployments (seconds) | `deployment.deployment_max_wait_seconds` |

#### Traffic Manager

Configuration for the traffic manager sidecar container.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **TRAFFIC_CONTAINER_IMAGE** | Traffic manager sidecar container image | `deployment.traffic_container_image` |
| **TRAFFIC_MANAGER_CONFIG_MAP** | ConfigMap name with custom traffic manager configuration | `deployment.traffic_manager_config_map` |

#### Pod Disruption Budget

Configuration for Pod Disruption Budget to control pod availability during disruptions.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **POD_DISRUPTION_BUDGET_ENABLED** | Whether Pod Disruption Budget is enabled | `deployment.pod_disruption_budget_enabled` |
| **POD_DISRUPTION_BUDGET_MAX_UNAVAILABLE** | Maximum number or percentage of pods that can be unavailable | `deployment.pod_disruption_budget_max_unavailable` |

#### Manifest Backup

Configuration for backing up Kubernetes manifests.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **MANIFEST_BACKUP_ENABLED** | Whether manifest backup is enabled | `deployment.manifest_backup_enabled` |
| **MANIFEST_BACKUP_TYPE** | Backup storage type | `deployment.manifest_backup_type` |
| **MANIFEST_BACKUP_BUCKET** | S3 bucket name for storing backups | `deployment.manifest_backup_bucket` |
| **MANIFEST_BACKUP_PREFIX** | Prefix path within the bucket | `deployment.manifest_backup_prefix` |

### Security

#### Image Pull Secrets

Configuration for pulling images from private container registries.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **IMAGE_PULL_SECRETS_ENABLED** | Whether image pull secrets are enabled | `security.image_pull_secrets_enabled` |
| **IMAGE_PULL_SECRETS** | List of secret names to use for pulling images | `security.image_pull_secrets` |

#### IAM

AWS IAM configuration for Kubernetes service accounts.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **IAM_ENABLED** | Whether IAM integration is enabled | `security.iam_enabled` |
| **IAM_PREFIX** | Prefix for IAM role names | `security.iam_prefix` |
| **IAM_POLICIES** | List of IAM policies to attach to the role | `security.iam_policies` |
| **IAM_BOUNDARY_ARN** | ARN of the permissions boundary policy | `security.iam_boundary_arn` |

#### Vault

HashiCorp Vault configuration for secrets management.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **VAULT_ADDR** | Vault server address | `security.vault_address` |
| **VAULT_TOKEN** | Vault authentication token | `security.vault_token` |

### Advanced

Advanced configuration options.

| Variable | Description | Scope Configuration Property |
|----------|-------------|------------------------------|
| **K8S_MODIFIERS** | JSON string with dynamic modifications to Kubernetes objects | `object_modifiers` |
85 changes: 76 additions & 9 deletions k8s/deployment/build_context
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ if ! validate_status "$SERVICE_ACTION" "$DEPLOYMENT_STATUS"; then
exit 1
fi

DEPLOY_STRATEGY=$(get_config_value \
--env DEPLOY_STRATEGY \
--provider '.providers["scope-configurations"].deployment.deployment_strategy' \
--default "blue-green"
)

if [ "$DEPLOY_STRATEGY" = "rolling" ] && [ "$DEPLOYMENT_STATUS" = "running" ]; then
GREEN_REPLICAS=$(echo "scale=10; ($GREEN_REPLICAS * $SWITCH_TRAFFIC) / 100" | bc)
GREEN_REPLICAS=$(echo "$GREEN_REPLICAS" | awk '{printf "%d", ($1 == int($1) ? $1 : int($1)+1)}')
Expand All @@ -89,8 +95,24 @@ fi
if [[ -n "$PULL_SECRETS" ]]; then
IMAGE_PULL_SECRETS=$PULL_SECRETS
else
IMAGE_PULL_SECRETS="${IMAGE_PULL_SECRETS:-"{}"}"
IMAGE_PULL_SECRETS=$(echo "$IMAGE_PULL_SECRETS" | jq .)
# Use env var if set, otherwise build from flat properties
if [ -n "${IMAGE_PULL_SECRETS:-}" ]; then
IMAGE_PULL_SECRETS=$(echo "$IMAGE_PULL_SECRETS" | jq .)
else
PULL_SECRETS_ENABLED=$(get_config_value \
--provider '.providers["scope-configurations"].security.image_pull_secrets_enabled' \
--default "false"
)
PULL_SECRETS_LIST=$(get_config_value \
--provider '.providers["scope-configurations"].security.image_pull_secrets | @json' \
--default "[]"
)

IMAGE_PULL_SECRETS=$(jq -n \
--argjson enabled "$PULL_SECRETS_ENABLED" \
--argjson secrets "$PULL_SECRETS_LIST" \
'{ENABLED: $enabled, SECRETS: $secrets}')
fi
fi

SCOPE_TRAFFIC_PROTOCOL=$(echo "$CONTEXT" | jq -r .scope.capabilities.protocol)
Expand All @@ -101,23 +123,68 @@ if [[ "$SCOPE_TRAFFIC_PROTOCOL" == "web_sockets" ]]; then
TRAFFIC_CONTAINER_VERSION="websocket2"
fi

TRAFFIC_CONTAINER_IMAGE=${TRAFFIC_CONTAINER_IMAGE:-"public.ecr.aws/nullplatform/k8s-traffic-manager:$TRAFFIC_CONTAINER_VERSION"}
TRAFFIC_CONTAINER_IMAGE=$(get_config_value \
--env TRAFFIC_CONTAINER_IMAGE \
--provider '.providers["scope-configurations"].deployment.traffic_container_image' \
--default "public.ecr.aws/nullplatform/k8s-traffic-manager:$TRAFFIC_CONTAINER_VERSION"
)

# Pod Disruption Budget configuration
PDB_ENABLED=${POD_DISRUPTION_BUDGET_ENABLED:-"false"}
PDB_MAX_UNAVAILABLE=${POD_DISRUPTION_BUDGET_MAX_UNAVAILABLE:-"25%"}

IAM=${IAM-"{}"}
PDB_ENABLED=$(get_config_value \
--env POD_DISRUPTION_BUDGET_ENABLED \
--provider '.providers["scope-configurations"].deployment.pod_disruption_budget_enabled' \
--default "false"
)
PDB_MAX_UNAVAILABLE=$(get_config_value \
--env POD_DISRUPTION_BUDGET_MAX_UNAVAILABLE \
--provider '.providers["scope-configurations"].deployment.pod_disruption_budget_max_unavailable' \
--default "25%"
)

# IAM configuration - build from flat properties or use env var
if [ -n "${IAM:-}" ]; then
IAM="$IAM"
else
IAM_ENABLED_RAW=$(get_config_value \
--provider '.providers["scope-configurations"].security.iam_enabled' \
--default "false"
)
IAM_PREFIX=$(get_config_value \
--provider '.providers["scope-configurations"].security.iam_prefix' \
--default ""
)
IAM_POLICIES=$(get_config_value \
--provider '.providers["scope-configurations"].security.iam_policies | @json' \
--default "[]"
)
IAM_BOUNDARY=$(get_config_value \
--provider '.providers["scope-configurations"].security.iam_boundary_arn' \
--default ""
)

IAM=$(jq -n \
--argjson enabled "$IAM_ENABLED_RAW" \
--arg prefix "$IAM_PREFIX" \
--argjson policies "$IAM_POLICIES" \
--arg boundary "$IAM_BOUNDARY" \
'{ENABLED: $enabled, PREFIX: $prefix, ROLE: {POLICIES: $policies, BOUNDARY_ARN: $boundary}} |
if .ROLE.BOUNDARY_ARN == "" then .ROLE |= del(.BOUNDARY_ARN) else . end |
if .PREFIX == "" then del(.PREFIX) else . end')
fi

IAM_ENABLED=$(echo "$IAM" | jq -r .ENABLED)
IAM_ENABLED=$(echo "$IAM" | jq -r '.ENABLED // false')

SERVICE_ACCOUNT_NAME=""

if [[ "$IAM_ENABLED" == "true" ]]; then
SERVICE_ACCOUNT_NAME=$(echo "$IAM" | jq -r .PREFIX)-"$SCOPE_ID"
fi

TRAFFIC_MANAGER_CONFIG_MAP=${TRAFFIC_MANAGER_CONFIG_MAP:-""}
TRAFFIC_MANAGER_CONFIG_MAP=$(get_config_value \
--env TRAFFIC_MANAGER_CONFIG_MAP \
--provider '.providers["scope-configurations"].deployment.traffic_manager_config_map' \
--default ""
)

if [[ -n "$TRAFFIC_MANAGER_CONFIG_MAP" ]]; then
echo "🔍 Validating ConfigMap '$TRAFFIC_MANAGER_CONFIG_MAP' in namespace '$K8S_NAMESPACE'"
Expand Down
Loading