diff --git a/README.md b/README.md index 8dffe01..dfe870e 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,15 @@ Then browse to [http://localhost:8080](http://localhost:8080). The exact namespa Alternatively, you can use a terminal UI such as [k9s](https://k9scli.io/) to select the service and start a port-forward interactively (press `` on a selected service or pod). +## Tutorials + +Beyond the Vacation Planner samples, the repository includes standalone tutorials that exercise specific AKS capabilities. Unlike the samples above, they do not deploy the web app. + +| Tutorial | Description | +| ------ | ----------- | +| [policies](policies/) | Kubernetes network policy tutorials that enforce zero-trust traffic control with [Calico](https://docs.tigera.io/calico/latest/about/) and [Cilium](https://docs.cilium.io/): cluster-wide default-deny, DNS-aware (FQDN) egress, and L3/L4/L7 ingress. | +| [ccm](ccm/scripts/) | Exercises the [Azure cloud controller manager](https://cloud-provider-azure.sigs.k8s.io/) load-balancer reconcile on the emulator: public and internal `Service` type `LoadBalancer`, `loadBalancerSourceRanges` NSG rules, the nodeIP backend-pool variant, and an NGINX ingress controller. | + ## Tools The following tools are useful when working with these samples: diff --git a/ccm/scripts/00-variables.sh b/ccm/scripts/00-variables.sh new file mode 100755 index 0000000..27ce434 --- /dev/null +++ b/ccm/scripts/00-variables.sh @@ -0,0 +1,83 @@ +# Shared variables for the Cloud Controller Manager (CCM) sample tests. +# +# Source this file from every test script with: source ./00-variables.sh +# +# The AKS cluster is created by scripts/01-user-assigned-managed-identity.sh; +# these values MUST match that script (prefix "local", suffix "test", location "ItalyNorth") so the +# tests target the cluster it creates instead of standing up their own. + +# Azure Kubernetes Service (AKS) +PREFIX="local" +SUFFIX="test" +AKS_NAME="${PREFIX}-aks-${SUFFIX}" +AKS_RESOURCE_GROUP_NAME="${PREFIX}-rg" +LOCATION="ItalyNorth" + +# Node resource group (the MC_* resource group the AKS RP manages). The CCM writes the per-service +# public IP, the LoadBalancer frontend and rule, and the NSG allow-rules here, so the tests assert +# against it. Derived from the cluster so it is never hardcoded (empty when the cluster does not exist). +NODE_RESOURCE_GROUP=$(az aks show \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --query nodeResourceGroup \ + --output tsv \ + --only-show-errors 2>/dev/null) + +# The primary Standard load balancer the AKS RP pre-creates in the node resource group. The CCM adds +# each public LoadBalancer Service's frontend, rule, and backend pool to it. Real AKS and the upstream +# cloud-provider-azure Helm chart default the name (and the inbound backend pool) to "kubernetes". +PUBLIC_LOAD_BALANCER_NAME="kubernetes" +# Internal LoadBalancer Services land on a separate load balancer named "kubernetes-internal". +INTERNAL_LOAD_BALANCER_NAME="kubernetes-internal" + +# Kubernetes namespace and the nginx workload the LoadBalancer Services select (tests 1 to 4). +NAMESPACE="ccm-test" +DEPLOYMENT_NAME="nginx" +APP_LABEL="nginx" +CONTAINER_IMAGE="nginx:1.27-alpine" +SERVICE_PORT=80 + +# One Service name per scenario, so the tests can coexist on the same cluster. +PUBLIC_SERVICE_NAME="nginx-public-lb" +INTERNAL_SERVICE_NAME="nginx-internal-lb" +RESTRICTED_SERVICE_NAME="nginx-restricted-lb" +NODE_IP_SERVICE_NAME="nginx-nodeip-lb" + +# Annotation that turns a Service into an internal LoadBalancer (test 2). The frontend IP is then +# allocated privately from the cluster subnet instead of a public IP. +# https://learn.microsoft.com/en-us/azure/aks/internal-lb +INTERNAL_LB_ANNOTATION="service.beta.kubernetes.io/azure-load-balancer-internal" + +# Client CIDR allowed to reach the restricted public Service (test 3). The CCM reconciles this into an +# inbound Allow rule on the node resource group NSG. 203.0.113.0/24 is the RFC 5737 TEST-NET-3 range. +ALLOWED_SOURCE_RANGE="203.0.113.0/24" + +# NGINX ingress controller (test 5). Its own front Service is type LoadBalancer, so the CCM assigns it +# an EXTERNAL-IP the same way. Mirrors 01-user-assigned-managed-identity.sh's ingress install. +INGRESS_NAMESPACE="ingress-basic" +INGRESS_RELEASE_NAME="nginx-ingress" +INGRESS_REPO_NAME="ingress-nginx" +INGRESS_REPO_URL="https://kubernetes.github.io/ingress-nginx" +INGRESS_CHART_NAME="ingress-nginx" + +# Backend workload behind the ingress (test 5 extension). A Deployment plus a ClusterIP Service, with +# an Ingress object routing to it, so a request that passes through the controller reaches a real +# backend. The backend serves a recognizable string (via a ConfigMap) so the pass-through is assertable. +BACKEND_DEPLOYMENT_NAME="ingress-backend" +BACKEND_SERVICE_NAME="ingress-backend" +BACKEND_APP_LABEL="ingress-backend" +BACKEND_CONFIG_MAP_NAME="ingress-backend-content" +BACKEND_RESPONSE_TEXT="Hello from the ingress backend" + +# The Ingress routes every request (path "/", no host) to the backend Service, so no Host header is +# needed to reach it through the controller. It targets the "nginx" IngressClass the chart installs. +INGRESS_NAME="ingress-backend" +INGRESS_CLASS_NAME="nginx" + +# Local port used by the kubectl port-forward pass-through check against the controller Service. +PORT_FORWARD_LOCAL_PORT=8080 + +# EXTERNAL-IP polling. The CCM writes the address only after the whole reconcile (frontend public IP, +# LB rule, backend-pool membership) completes, which can take a few minutes on a loaded emulator. +EXTERNAL_IP_TIMEOUT_SECONDS=300 +SLEEP=5 diff --git a/ccm/scripts/01-test-public-loadbalancer.sh b/ccm/scripts/01-test-public-loadbalancer.sh new file mode 100755 index 0000000..8566155 --- /dev/null +++ b/ccm/scripts/01-test-public-loadbalancer.sh @@ -0,0 +1,137 @@ +#!/bin/bash + +# Test 1: a public Service of type LoadBalancer receives an EXTERNAL-IP, and the Cloud Controller +# Manager creates a dedicated public IP for it in the node resource group plus a frontend and a rule +# on the "kubernetes" load balancer. +# +# The EXTERNAL-IP is a synthetic, non-routable placeholder (the emulated load balancer has no real +# dataplane): to actually reach the service, use kubectl port-forward. +# https://learn.microsoft.com/en-us/azure/aks/load-balancer-standard + +# Variables +source ./00-variables.sh + +# Make sure the AKS cluster exists (it is created by 01-user-assigned-managed-identity.sh) +if [[ -z $NODE_RESOURCE_GROUP ]]; then + echo "Could not resolve the node resource group for the [$AKS_NAME] AKS cluster" + echo "Create the cluster first with scripts/01-user-assigned-managed-identity.sh" + exit 1 +fi + +# Merge the cluster credentials into kubeconfig and set it as the current context +echo "Merging credentials for the [$AKS_NAME] AKS cluster into kubeconfig..." +az aks get-credentials \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --overwrite-existing \ + --only-show-errors + +# Create the namespace if it does not already exist +RESULT=$(kubectl get namespace $NAMESPACE -o jsonpath='{.metadata.name}' 2>/dev/null) +if [[ -n $RESULT ]]; then + echo "The [$NAMESPACE] namespace already exists" +else + echo "Creating the [$NAMESPACE] namespace..." + kubectl create namespace $NAMESPACE +fi + +# Deploy the nginx workload the Service selects (idempotent) +echo "Deploying the [$DEPLOYMENT_NAME] nginx deployment to the [$NAMESPACE] namespace..." +cat </dev/null) + if [[ -n $EXTERNAL_IP ]]; then + break + fi + sleep $SLEEP +done + +if [[ -n $EXTERNAL_IP ]]; then + echo "The [$PUBLIC_SERVICE_NAME] service received EXTERNAL-IP [$EXTERNAL_IP]" +else + echo "The [$PUBLIC_SERVICE_NAME] service did not receive an EXTERNAL-IP within [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds" + exit 1 +fi + +# The CCM creates a dedicated inbound public IP for the service in the node resource group +echo "Looking for a public IP with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group..." +PUBLIC_IP_NAME=$(az network public-ip list \ + --resource-group $NODE_RESOURCE_GROUP \ + --query "[?ipAddress=='$EXTERNAL_IP'].name | [0]" \ + --output tsv \ + --only-show-errors) + +if [[ -n $PUBLIC_IP_NAME ]]; then + echo "Found public IP [$PUBLIC_IP_NAME] with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group" +else + echo "No public IP with address [$EXTERNAL_IP] found in the [$NODE_RESOURCE_GROUP] node resource group" + exit 1 +fi + +# The CCM adds a frontend IP configuration and a load-balancing rule to the "kubernetes" load balancer +echo "Verifying the [$PUBLIC_LOAD_BALANCER_NAME] load balancer frontend and rule in the [$NODE_RESOURCE_GROUP] node resource group..." +FRONTEND_COUNT=$(az network lb show \ + --resource-group $NODE_RESOURCE_GROUP \ + --name $PUBLIC_LOAD_BALANCER_NAME \ + --query "length(frontendIPConfigurations || frontendIpConfigurations)" \ + --output tsv \ + --only-show-errors) +RULE_COUNT=$(az network lb rule list \ + --resource-group $NODE_RESOURCE_GROUP \ + --lb-name $PUBLIC_LOAD_BALANCER_NAME \ + --query "length(@)" \ + --output tsv \ + --only-show-errors) + +if [[ -n $FRONTEND_COUNT && $FRONTEND_COUNT -ge 1 && -n $RULE_COUNT && $RULE_COUNT -ge 1 ]]; then + echo "The [$PUBLIC_LOAD_BALANCER_NAME] load balancer has [$FRONTEND_COUNT] frontend(s) and [$RULE_COUNT] rule(s)" +else + echo "The [$PUBLIC_LOAD_BALANCER_NAME] load balancer is missing a frontend or a rule (frontends: [$FRONTEND_COUNT], rules: [$RULE_COUNT])" + exit 1 +fi + +echo "SUCCESS: the [$PUBLIC_SERVICE_NAME] public LoadBalancer service is backed by node resource group resources" +echo "The EXTERNAL-IP [$EXTERNAL_IP] is a synthetic placeholder; run 'kubectl port-forward -n $NAMESPACE svc/$PUBLIC_SERVICE_NAME 8080:$SERVICE_PORT' to reach nginx" diff --git a/ccm/scripts/02-test-internal-loadbalancer.sh b/ccm/scripts/02-test-internal-loadbalancer.sh new file mode 100755 index 0000000..0c852b4 --- /dev/null +++ b/ccm/scripts/02-test-internal-loadbalancer.sh @@ -0,0 +1,136 @@ +#!/bin/bash + +# Test 2: a Service annotated as an internal LoadBalancer receives a private EXTERNAL-IP allocated +# from the cluster subnet, materialised on the separate "kubernetes-internal" load balancer, and NO +# public IP is created for it in the node resource group. +# https://learn.microsoft.com/en-us/azure/aks/internal-lb + +# Variables +source ./00-variables.sh + +# Make sure the AKS cluster exists (it is created by 01-user-assigned-managed-identity.sh) +if [[ -z $NODE_RESOURCE_GROUP ]]; then + echo "Could not resolve the node resource group for the [$AKS_NAME] AKS cluster" + echo "Create the cluster first with scripts/01-user-assigned-managed-identity.sh" + exit 1 +fi + +# Merge the cluster credentials into kubeconfig and set it as the current context +echo "Merging credentials for the [$AKS_NAME] AKS cluster into kubeconfig..." +az aks get-credentials \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --overwrite-existing \ + --only-show-errors + +# Create the namespace if it does not already exist +RESULT=$(kubectl get namespace $NAMESPACE -o jsonpath='{.metadata.name}' 2>/dev/null) +if [[ -n $RESULT ]]; then + echo "The [$NAMESPACE] namespace already exists" +else + echo "Creating the [$NAMESPACE] namespace..." + kubectl create namespace $NAMESPACE +fi + +# Deploy the nginx workload the Service selects (idempotent) +echo "Deploying the [$DEPLOYMENT_NAME] nginx deployment to the [$NAMESPACE] namespace..." +cat </dev/null) + if [[ -n $EXTERNAL_IP ]]; then + break + fi + sleep $SLEEP +done + +if [[ -n $EXTERNAL_IP ]]; then + echo "The [$INTERNAL_SERVICE_NAME] service received private EXTERNAL-IP [$EXTERNAL_IP]" +else + echo "The [$INTERNAL_SERVICE_NAME] service did not receive an EXTERNAL-IP within [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds" + exit 1 +fi + +# The internal frontend lives on a separate load balancer named "kubernetes-internal" +echo "Verifying the [$INTERNAL_LOAD_BALANCER_NAME] load balancer exists in the [$NODE_RESOURCE_GROUP] node resource group..." +INTERNAL_LB_NAME_FOUND=$(az network lb show \ + --resource-group $NODE_RESOURCE_GROUP \ + --name $INTERNAL_LOAD_BALANCER_NAME \ + --query name \ + --output tsv \ + --only-show-errors 2>/dev/null) + +if [[ -n $INTERNAL_LB_NAME_FOUND ]]; then + echo "Found the [$INTERNAL_LB_NAME_FOUND] internal load balancer; its frontend private IPs are:" + az network lb show \ + --resource-group $NODE_RESOURCE_GROUP \ + --name $INTERNAL_LOAD_BALANCER_NAME \ + --query "(frontendIPConfigurations || frontendIpConfigurations)[].privateIPAddress" \ + --output tsv \ + --only-show-errors +else + echo "The [$INTERNAL_LOAD_BALANCER_NAME] internal load balancer was not found in the [$NODE_RESOURCE_GROUP] node resource group" + exit 1 +fi + +# An internal LoadBalancer must NOT allocate a public IP: the EXTERNAL-IP is a private subnet address +echo "Confirming no public IP with address [$EXTERNAL_IP] exists in the [$NODE_RESOURCE_GROUP] node resource group..." +PUBLIC_IP_NAME=$(az network public-ip list \ + --resource-group $NODE_RESOURCE_GROUP \ + --query "[?ipAddress=='$EXTERNAL_IP'].name | [0]" \ + --output tsv \ + --only-show-errors) + +if [[ -z $PUBLIC_IP_NAME ]]; then + echo "Confirmed: the [$INTERNAL_SERVICE_NAME] service is backed by a private frontend, not a public IP" +else + echo "Unexpected: a public IP [$PUBLIC_IP_NAME] with address [$EXTERNAL_IP] exists for an internal service" + exit 1 +fi + +echo "SUCCESS: the [$INTERNAL_SERVICE_NAME] internal LoadBalancer service received a private EXTERNAL-IP [$EXTERNAL_IP] on the [$INTERNAL_LOAD_BALANCER_NAME] load balancer" +echo "The EXTERNAL-IP [$EXTERNAL_IP] is a private IP address; run 'kubectl port-forward -n $NAMESPACE svc/$INTERNAL_SERVICE_NAME 8080:$SERVICE_PORT' to reach nginx" \ No newline at end of file diff --git a/ccm/scripts/03-test-loadbalancer-source-ranges.sh b/ccm/scripts/03-test-loadbalancer-source-ranges.sh new file mode 100755 index 0000000..dded3de --- /dev/null +++ b/ccm/scripts/03-test-loadbalancer-source-ranges.sh @@ -0,0 +1,137 @@ +#!/bin/bash + +# Test 3: a public Service with spec.loadBalancerSourceRanges causes the Cloud Controller Manager to +# reconcile an inbound Allow rule on the node resource group NSG, restricted to the given client CIDR. +# https://learn.microsoft.com/en-us/azure/aks/configure-load-balancer-standard + +# Variables +source ./00-variables.sh + +# Make sure the AKS cluster exists (it is created by 01-user-assigned-managed-identity.sh) +if [[ -z $NODE_RESOURCE_GROUP ]]; then + echo "Could not resolve the node resource group for the [$AKS_NAME] AKS cluster" + echo "Create the cluster first with scripts/01-user-assigned-managed-identity.sh" + exit 1 +fi + +# Merge the cluster credentials into kubeconfig and set it as the current context +echo "Merging credentials for the [$AKS_NAME] AKS cluster into kubeconfig..." +az aks get-credentials \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --overwrite-existing \ + --only-show-errors + +# Create the namespace if it does not already exist +RESULT=$(kubectl get namespace $NAMESPACE -o jsonpath='{.metadata.name}' 2>/dev/null) +if [[ -n $RESULT ]]; then + echo "The [$NAMESPACE] namespace already exists" +else + echo "Creating the [$NAMESPACE] namespace..." + kubectl create namespace $NAMESPACE +fi + +# Deploy the nginx workload the Service selects (idempotent) +echo "Deploying the [$DEPLOYMENT_NAME] nginx deployment to the [$NAMESPACE] namespace..." +cat </dev/null) + if [[ -n $EXTERNAL_IP ]]; then + break + fi + sleep $SLEEP +done + +if [[ -n $EXTERNAL_IP ]]; then + echo "The [$RESTRICTED_SERVICE_NAME] service received EXTERNAL-IP [$EXTERNAL_IP]" +else + echo "The [$RESTRICTED_SERVICE_NAME] service did not receive an EXTERNAL-IP within [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds" + exit 1 +fi + +# Resolve the node resource group NSG the CCM reconciles service rules into +NSG_NAME=$(az network nsg list \ + --resource-group $NODE_RESOURCE_GROUP \ + --query "[0].name" \ + --output tsv \ + --only-show-errors) + +if [[ -z $NSG_NAME ]]; then + echo "No network security group found in the [$NODE_RESOURCE_GROUP] node resource group" + exit 1 +fi + +# Show the current inbound rules for visibility, then assert the restricted Allow rule is present +echo "Inbound rules on the [$NSG_NAME] network security group:" +az network nsg rule list \ + --resource-group $NODE_RESOURCE_GROUP \ + --nsg-name $NSG_NAME \ + --query "[?direction=='Inbound'].{name:name, access:access, port:destinationPortRange, source:sourceAddressPrefix, sources:sourceAddressPrefixes}" \ + --output table \ + --only-show-errors + +# The CCM writes the allowed CIDR into sourceAddressPrefix (single) or sourceAddressPrefixes (array). +# Collect both fields from inbound Allow rules and match the range as a fixed string (grep -F), which +# sidesteps JMESPath null-array handling and any casing quirks in the emulated response. +echo "Looking for an inbound Allow rule for port [$SERVICE_PORT] restricted to [$ALLOWED_SOURCE_RANGE]..." +ALLOW_RULE_SOURCES=$(az network nsg rule list \ + --resource-group $NODE_RESOURCE_GROUP \ + --nsg-name $NSG_NAME \ + --query "[?access=='Allow' && direction=='Inbound'].[sourceAddressPrefix, sourceAddressPrefixes]" \ + --output tsv \ + --only-show-errors) + +if echo "$ALLOW_RULE_SOURCES" | grep -qF "$ALLOWED_SOURCE_RANGE"; then + echo "Found an inbound Allow rule restricted to [$ALLOWED_SOURCE_RANGE]" +else + echo "No inbound Allow rule restricted to [$ALLOWED_SOURCE_RANGE] found on the [$NSG_NAME] network security group" + exit 1 +fi + +echo "SUCCESS: the [$RESTRICTED_SERVICE_NAME] service reconciled an NSG allow-rule scoped to [$ALLOWED_SOURCE_RANGE]" diff --git a/ccm/scripts/04-test-nodeip-backend-pool.sh b/ccm/scripts/04-test-nodeip-backend-pool.sh new file mode 100755 index 0000000..ec28a4d --- /dev/null +++ b/ccm/scripts/04-test-nodeip-backend-pool.sh @@ -0,0 +1,153 @@ +#!/bin/bash + +# Test 4: the "nodeIP" backend-pool variant. When the cluster is created with +# --load-balancer-backend-pool-type nodeIP, the Cloud Controller Manager puts the nodes' private IPs +# directly into the "kubernetes" backend pool (loadBalancerBackendAddresses) instead of referencing +# NIC IP configurations. A public LoadBalancer Service still receives an EXTERNAL-IP. +# https://learn.microsoft.com/en-us/azure/aks/configure-load-balancer-standard +# +# backendPoolType is a create-time cluster property, so this test does NOT create a cluster: it guards +# on the cluster's declared backendPoolType and, when it is not "nodeIP", tells you how to re-create +# the cluster before exiting. + +# Variables +source ./00-variables.sh + +# Make sure the AKS cluster exists (it is created by 01-user-assigned-managed-identity.sh) +if [[ -z $NODE_RESOURCE_GROUP ]]; then + echo "Could not resolve the node resource group for the [$AKS_NAME] AKS cluster" + echo "Create the cluster first with scripts/01-user-assigned-managed-identity.sh" + exit 1 +fi + +# This scenario is only meaningful on a nodeIP cluster; the default is nodeIPConfiguration +echo "Checking the backendPoolType of the [$AKS_NAME] AKS cluster..." +BACKEND_POOL_TYPE=$(az aks show \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --query "networkProfile.loadBalancerProfile.backendPoolType" \ + --output tsv \ + --only-show-errors) + +if [[ $BACKEND_POOL_TYPE != "nodeIP" ]]; then + echo "This test requires a cluster created with [--load-balancer-backend-pool-type nodeIP]" + echo "The [$AKS_NAME] cluster reports backendPoolType [$BACKEND_POOL_TYPE]" + echo "Add [--load-balancer-backend-pool-type nodeIP] to the az aks create command in" + echo "scripts/01-user-assigned-managed-identity.sh, re-create the cluster, then re-run" + exit 1 +fi + +echo "The [$AKS_NAME] cluster uses backendPoolType [$BACKEND_POOL_TYPE]" + +# Merge the cluster credentials into kubeconfig and set it as the current context +echo "Merging credentials for the [$AKS_NAME] AKS cluster into kubeconfig..." +az aks get-credentials \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --overwrite-existing \ + --only-show-errors + +# Create the namespace if it does not already exist +RESULT=$(kubectl get namespace $NAMESPACE -o jsonpath='{.metadata.name}' 2>/dev/null) +if [[ -n $RESULT ]]; then + echo "The [$NAMESPACE] namespace already exists" +else + echo "Creating the [$NAMESPACE] namespace..." + kubectl create namespace $NAMESPACE +fi + +# Deploy the nginx workload the Service selects (idempotent) +echo "Deploying the [$DEPLOYMENT_NAME] nginx deployment to the [$NAMESPACE] namespace..." +cat </dev/null) + if [[ -n $EXTERNAL_IP ]]; then + break + fi + sleep $SLEEP +done + +if [[ -n $EXTERNAL_IP ]]; then + echo "The [$NODE_IP_SERVICE_NAME] service received EXTERNAL-IP [$EXTERNAL_IP]" +else + echo "The [$NODE_IP_SERVICE_NAME] service did not receive an EXTERNAL-IP within [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds" + exit 1 +fi + +# The CCM creates a dedicated inbound public IP for the service in the node resource group +echo "Looking for a public IP with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group..." +PUBLIC_IP_NAME=$(az network public-ip list \ + --resource-group $NODE_RESOURCE_GROUP \ + --query "[?ipAddress=='$EXTERNAL_IP'].name | [0]" \ + --output tsv \ + --only-show-errors) + +if [[ -n $PUBLIC_IP_NAME ]]; then + echo "Found public IP [$PUBLIC_IP_NAME] with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group" +else + echo "No public IP with address [$EXTERNAL_IP] found in the [$NODE_RESOURCE_GROUP] node resource group" + exit 1 +fi + +# On the nodeIP path the "kubernetes" backend pool holds node private IPs as loadBalancerBackendAddresses +# (not NIC IP configuration references, which is what the default nodeIPConfiguration path uses) +echo "Verifying the [$PUBLIC_LOAD_BALANCER_NAME] backend pool holds node IP addresses..." +BACKEND_ADDRESS_COUNT=$(az network lb address-pool show \ + --resource-group $NODE_RESOURCE_GROUP \ + --lb-name $PUBLIC_LOAD_BALANCER_NAME \ + --name $PUBLIC_LOAD_BALANCER_NAME \ + --query "length(loadBalancerBackendAddresses)" \ + --output tsv \ + --only-show-errors 2>/dev/null) + +if [[ -n $BACKEND_ADDRESS_COUNT && $BACKEND_ADDRESS_COUNT -ge 1 ]]; then + echo "The [$PUBLIC_LOAD_BALANCER_NAME] backend pool holds [$BACKEND_ADDRESS_COUNT] node IP address(es)" +else + echo "The [$PUBLIC_LOAD_BALANCER_NAME] backend pool holds no node IP addresses (found: [$BACKEND_ADDRESS_COUNT])" + exit 1 +fi + +echo "SUCCESS: the [$NODE_IP_SERVICE_NAME] service on the nodeIP cluster received EXTERNAL-IP [$EXTERNAL_IP] with node IPs in the [$PUBLIC_LOAD_BALANCER_NAME] backend pool" diff --git a/ccm/scripts/05-test-nginx-ingress-controller.sh b/ccm/scripts/05-test-nginx-ingress-controller.sh new file mode 100755 index 0000000..7b001e4 --- /dev/null +++ b/ccm/scripts/05-test-nginx-ingress-controller.sh @@ -0,0 +1,253 @@ +#!/bin/bash + +# Test 5: an ingress controller is itself fronted by a Service of type LoadBalancer, so the Cloud +# Controller Manager assigns its controller Service an EXTERNAL-IP and creates a public IP in the node +# resource group, exactly like a plain LoadBalancer Service. This installs the NGINX ingress controller +# via Helm (mirroring 01-user-assigned-managed-identity.sh's ingress install) and asserts that. +# +# It then deploys a backend (Deployment + ClusterIP Service) and an Ingress routing to it, and verifies +# that traffic reaches the backend THROUGH the controller via kubectl port-forward. +# +# The EXTERNAL-IP is a synthetic, non-routable placeholder, but kubectl port-forward against the +# controller Service tunnels straight to the controller pod (bypassing the EXTERNAL-IP entirely), so the +# pass-through works. Without the Ingress created below, the controller has no route and returns its +# default-backend 404 (that is the "does not route" behaviour, not a port-forward failure). +# https://learn.microsoft.com/en-us/azure/aks/app-routing + +# Variables +source ./00-variables.sh + +# Make sure the AKS cluster exists (it is created by 01-user-assigned-managed-identity.sh) +if [[ -z $NODE_RESOURCE_GROUP ]]; then + echo "Could not resolve the node resource group for the [$AKS_NAME] AKS cluster" + echo "Create the cluster first with scripts/01-user-assigned-managed-identity.sh" + exit 1 +fi + +# Merge the cluster credentials into kubeconfig and set it as the current context +echo "Merging credentials for the [$AKS_NAME] AKS cluster into kubeconfig..." +az aks get-credentials \ + --name $AKS_NAME \ + --resource-group $AKS_RESOURCE_GROUP_NAME \ + --overwrite-existing \ + --only-show-errors + +# Install the NGINX ingress controller with Helm if it is not already installed +RESULT=$(helm list --namespace $INGRESS_NAMESPACE 2>/dev/null | grep $INGRESS_RELEASE_NAME | awk '{print $1}') + +if [[ -n $RESULT ]]; then + echo "The [$INGRESS_RELEASE_NAME] ingress controller already exists in the [$INGRESS_NAMESPACE] namespace" +else + # Add the ingress-nginx Helm repository if it is not already added + RESULT=$(helm repo list 2>/dev/null | grep $INGRESS_REPO_NAME | awk '{print $1}') + + if [[ -n $RESULT ]]; then + echo "The [$INGRESS_REPO_NAME] Helm repo already exists" + else + echo "Adding the [$INGRESS_REPO_NAME] Helm repo..." + helm repo add $INGRESS_REPO_NAME $INGRESS_REPO_URL + fi + + # Update the local Helm chart repository cache + echo "Updating Helm repos..." + helm repo update + + # Deploy the NGINX ingress controller (its controller Service is type LoadBalancer). The admission + # webhook is disabled so the Ingress created below applies cleanly right after install without racing + # the webhook's certificate/readiness; keep it enabled for production clusters. + echo "Deploying the [$INGRESS_RELEASE_NAME] NGINX ingress controller to the [$INGRESS_NAMESPACE] namespace..." + helm install $INGRESS_RELEASE_NAME $INGRESS_REPO_NAME/$INGRESS_CHART_NAME \ + --create-namespace \ + --namespace $INGRESS_NAMESPACE \ + --set controller.replicaCount=1 \ + --set controller.service.type=LoadBalancer \ + --set controller.admissionWebhooks.enabled=false +fi + +# Resolve the controller Service name from its labels (do not hardcode the Helm-generated name) +echo "Resolving the ingress controller LoadBalancer service in the [$INGRESS_NAMESPACE] namespace..." +CONTROLLER_SERVICE_NAME=$(kubectl get service \ + --namespace $INGRESS_NAMESPACE \ + --selector app.kubernetes.io/component=controller \ + -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) + +if [[ -z $CONTROLLER_SERVICE_NAME ]]; then + echo "Could not find the ingress controller LoadBalancer service in the [$INGRESS_NAMESPACE] namespace" + exit 1 +fi + +echo "Found the [$CONTROLLER_SERVICE_NAME] ingress controller service" + +# Wait for the Cloud Controller Manager to assign the controller Service its EXTERNAL-IP +echo "Waiting up to [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds for the [$CONTROLLER_SERVICE_NAME] service EXTERNAL-IP..." +EXTERNAL_IP="" +DEADLINE=$((SECONDS + EXTERNAL_IP_TIMEOUT_SECONDS)) +while [[ $SECONDS -lt $DEADLINE ]]; do + EXTERNAL_IP=$(kubectl get service $CONTROLLER_SERVICE_NAME -n $INGRESS_NAMESPACE \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}' 2>/dev/null) + if [[ -n $EXTERNAL_IP ]]; then + break + fi + sleep $SLEEP +done + +if [[ -n $EXTERNAL_IP ]]; then + echo "The [$CONTROLLER_SERVICE_NAME] service received EXTERNAL-IP [$EXTERNAL_IP]" +else + echo "The [$CONTROLLER_SERVICE_NAME] service did not receive an EXTERNAL-IP within [$EXTERNAL_IP_TIMEOUT_SECONDS] seconds" + exit 1 +fi + +# The CCM creates a dedicated public IP for the controller Service in the node resource group +echo "Looking for a public IP with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group..." +PUBLIC_IP_NAME=$(az network public-ip list \ + --resource-group $NODE_RESOURCE_GROUP \ + --query "[?ipAddress=='$EXTERNAL_IP'].name | [0]" \ + --output tsv \ + --only-show-errors) + +if [[ -n $PUBLIC_IP_NAME ]]; then + echo "Found public IP [$PUBLIC_IP_NAME] with address [$EXTERNAL_IP] in the [$NODE_RESOURCE_GROUP] node resource group" +else + echo "No public IP with address [$EXTERNAL_IP] found in the [$NODE_RESOURCE_GROUP] node resource group" + exit 1 +fi + +echo "The [$CONTROLLER_SERVICE_NAME] ingress controller service received EXTERNAL-IP [$EXTERNAL_IP] backed by a node resource group public IP" + +# Create the namespace if it does not already exist +RESULT=$(kubectl get namespace $NAMESPACE -o jsonpath='{.metadata.name}' 2>/dev/null) +if [[ -n $RESULT ]]; then + echo "The [$NAMESPACE] namespace already exists" +else + echo "Creating the [$NAMESPACE] namespace..." + kubectl create namespace $NAMESPACE +fi + +# Backend content: a ConfigMap with a recognizable string, mounted as the backend's index.html so the +# pass-through can be asserted (a plain response distinguishes it from the controller default-backend 404) +echo "Creating the [$BACKEND_CONFIG_MAP_NAME] config map in the [$NAMESPACE] namespace..." +cat </dev/null 2>&1 & +PORT_FORWARD_PID=$! + +PASS_THROUGH_BODY="" +DEADLINE=$((SECONDS + 60)) +while [[ $SECONDS -lt $DEADLINE ]]; do + PASS_THROUGH_BODY=$(curl --silent --max-time 5 "http://localhost:$PORT_FORWARD_LOCAL_PORT/" 2>/dev/null) + if echo "$PASS_THROUGH_BODY" | grep -qF "$BACKEND_RESPONSE_TEXT"; then + break + fi + sleep $SLEEP +done + +# Stop the background port-forward +kill $PORT_FORWARD_PID 2>/dev/null +wait $PORT_FORWARD_PID 2>/dev/null + +if echo "$PASS_THROUGH_BODY" | grep -qF "$BACKEND_RESPONSE_TEXT"; then + echo "Pass-through OK: reached the [$BACKEND_SERVICE_NAME] backend through the [$CONTROLLER_SERVICE_NAME] controller (response: [$BACKEND_RESPONSE_TEXT])" +else + echo "Pass-through check did not return the backend response within the timeout (the controller pod may still be warming up)" + echo "Retry manually with the commands below" +fi + +echo "SUCCESS: the [$CONTROLLER_SERVICE_NAME] ingress controller has EXTERNAL-IP [$EXTERNAL_IP] and the [$INGRESS_NAME] ingress routes to the [$BACKEND_SERVICE_NAME] backend" +echo "Reach the backend through the controller (this bypasses the synthetic, non-routable EXTERNAL-IP):" +echo " kubectl port-forward -n $INGRESS_NAMESPACE svc/$CONTROLLER_SERVICE_NAME $PORT_FORWARD_LOCAL_PORT:80" +echo " curl http://localhost:$PORT_FORWARD_LOCAL_PORT/" diff --git a/ccm/scripts/README.md b/ccm/scripts/README.md new file mode 100644 index 0000000..222e9f9 --- /dev/null +++ b/ccm/scripts/README.md @@ -0,0 +1,51 @@ +## Cloud Controller Manager (CCM) Load Balancer Samples + +The [Azure cloud controller manager](https://cloud-provider-azure.sigs.k8s.io/) (`cloud-provider-azure`) is the AKS component that turns Kubernetes `Service` and `Node` events into Azure API calls. When you create a `Service` of type `LoadBalancer`, the CCM provisions an Azure load balancer frontend, a public IP, and network security group rules, then writes the assigned address back as the Service `EXTERNAL-IP`. + +The CCM is a **control-plane, provisioning-time** component: it reconciles the Azure resources when a Service is created, updated, or deleted. It is **not** in the runtime data path (traffic flows through the load balancer and kube-proxy to the pods, not through the CCM). + +These scripts exercise that reconcile against the [LocalStack for Azure](https://docs.localstack.cloud/azure/) emulator: public and internal load balancers, source-range NSG rules, the nodeIP backend-pool variant, and an NGINX ingress controller. + +Because the emulated load balancer has no real dataplane, the `EXTERNAL-IP` is a synthetic, non-routable placeholder. To actually reach a workload, use `kubectl port-forward`, which tunnels straight to the pod and bypasses the `EXTERNAL-IP`. + +## Prerequisites + +- An AKS cluster reachable through `kubectl`, created by [scripts/01-user-assigned-managed-identity.sh](../../scripts/01-user-assigned-managed-identity.sh). The scripts do not create the cluster; they source `./00-variables.sh`, whose values must match it (`local-aks-test` in resource group `local-rg`, location `ItalyNorth`). +- [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/) (`az`) and [kubectl](https://kubernetes.io/docs/tasks/tools/). +- [Helm](https://helm.sh/) for `05-test-nginx-ingress-controller.sh`. + +## How to run + +Each script sources `./00-variables.sh`, merges the cluster credentials, and guards on the node resource group (exiting with instructions if the cluster does not exist). Run them from this folder, in order or individually. + +```bash +cd ccm/scripts +./01-test-public-loadbalancer.sh +./02-test-internal-loadbalancer.sh +./03-test-loadbalancer-source-ranges.sh +./05-test-nginx-ingress-controller.sh +``` + +## Scripts + +- `00-variables.sh`: Shared variables sourced by every test: cluster name, resource group, and location (which must match the cluster-creation script); the derived node resource group; the load-balancer, namespace, Service, ingress, and backend names; and the `EXTERNAL-IP` poll timeout. +- `01-test-public-loadbalancer.sh`: Creates a public `Service` of type `LoadBalancer` and asserts it receives an `EXTERNAL-IP`, that a matching public IP exists in the node resource group, and that the `kubernetes` load balancer has a frontend and a rule. +- `02-test-internal-loadbalancer.sh`: Creates an internal LoadBalancer Service (via the `service.beta.kubernetes.io/azure-load-balancer-internal` annotation) and asserts it receives a private `EXTERNAL-IP` on the `kubernetes-internal` load balancer and that no public IP is created for it. +- `03-test-loadbalancer-source-ranges.sh`: Creates a public Service with `loadBalancerSourceRanges` and asserts the CCM reconciles an inbound Allow rule scoped to that CIDR on the node resource group network security group. +- `04-test-nodeip-backend-pool.sh`: Exercises the `nodeIP` backend-pool variant. It self-guards on the cluster's `backendPoolType`: unless the cluster was created with `--load-balancer-backend-pool-type nodeIP` it prints how to re-create the cluster and exits, because `backendPoolType` is a create-time property. When it is `nodeIP`, it asserts the `kubernetes` backend pool holds node IP addresses rather than NIC IP-configuration references. +- `05-test-nginx-ingress-controller.sh`: Installs the NGINX ingress controller (its own front Service is type `LoadBalancer`, so the CCM assigns it an `EXTERNAL-IP` and a node resource group public IP), then deploys a backend Deployment, a ClusterIP Service, and an Ingress, and verifies that traffic reaches the backend through the controller with a `kubectl port-forward` pass-through check. + +## Resources + +- [Cloud Controller Manager (Kubernetes documentation)](https://kubernetes.io/docs/concepts/architecture/cloud-controller/) +- [Cluster Architecture (Kubernetes documentation)](https://kubernetes.io/docs/concepts/architecture/) +- [Developing Cloud Controller Manager (Kubernetes)](https://kubernetes.io/docs/tasks/administer-cluster/developing-cloud-controller-manager/) +- [Cloud Controller Manager Administration (Kubernetes)](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/) +- [KEP-2392: Cloud Controller Manager](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider/2392-cloud-controller-manager) +- [Cloud provider for Azure documentation site](https://cloud-provider-azure.sigs.k8s.io/) +- [Core concepts for Azure Kubernetes Service (AKS)](https://learn.microsoft.com/azure/aks/core-aks-concepts) +- [Use a public standard load balancer in AKS](https://learn.microsoft.com/azure/aks/load-balancer-standard) +- [Use an internal load balancer in AKS](https://learn.microsoft.com/azure/aks/internal-lb) +- [Use a static public IP address and DNS label with the AKS load balancer](https://learn.microsoft.com/azure/aks/static-ip) +- [Configure the public standard load balancer in AKS](https://learn.microsoft.com/azure/aks/configure-load-balancer-standard) +- [Managed NGINX ingress with the application routing add-on](https://learn.microsoft.com/azure/aks/app-routing) diff --git a/policies/README.md b/policies/README.md new file mode 100644 index 0000000..e915644 --- /dev/null +++ b/policies/README.md @@ -0,0 +1,26 @@ +## Kubernetes Network Policy Tutorials + +[Kubernetes network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) control which pods may talk to each other and to the outside world. On AKS the rules are enforced by a policy engine tied to the cluster's data plane, so the engine is chosen when the cluster is created: [scripts/01-user-assigned-managed-identity.sh](../scripts/01-user-assigned-managed-identity.sh) offers Azure, Cilium, and Calico network policy in its menu ([AKS network policies](https://learn.microsoft.com/en-us/azure/aks/use-network-policies)). + +These tutorials each build a policy scenario step by step and verify, from inside probe pods, that allowed traffic flows and everything else is blocked. + +| Tutorial | Engine | Demonstrates | +| --- | --- | --- | +| [calico/calico-policy-tutorial](calico/calico-policy-tutorial) | Calico | A cluster-wide default-deny `GlobalNetworkPolicy`, then namespaced `NetworkPolicy` rules that selectively re-open egress and ingress (zero trust). | +| [cilium/egress-tutorial](cilium/egress-tutorial) | Cilium | DNS-aware (FQDN) egress control: an exact hostname, a wildcard pattern, and a pattern locked to a single port. | +| [cilium/ingress-tutorial](cilium/ingress-tutorial) | Cilium | Identity-aware ingress, first at L3/L4 (which workloads may connect) and then at L7 (which HTTP calls they may make). | + +## Prerequisites + +- An AKS cluster reachable through `kubectl`, created with the policy engine that matches the tutorial you want to run (Calico for the Calico tutorial, Cilium for the two Cilium tutorials). +- [kubectl](https://kubernetes.io/docs/tasks/tools/) configured for the cluster. +- The engine's CLI, installed by each tutorial's `00-*.sh` script (`calicoctl` for Calico; `cilium` and `hubble` for Cilium). + +Follow the README in each tutorial folder for the full walkthrough. + +## Resources + +- [Network Policies (Kubernetes documentation)](https://kubernetes.io/docs/concepts/services-networking/network-policies/) +- [Secure traffic between pods using network policies in AKS](https://learn.microsoft.com/en-us/azure/aks/use-network-policies) +- [Project Calico documentation](https://docs.tigera.io/calico/latest/about/) +- [Cilium documentation](https://docs.cilium.io/) diff --git a/policies/calico/calico-policy-tutorial/README.md b/policies/calico/calico-policy-tutorial/README.md new file mode 100644 index 0000000..bc8695c --- /dev/null +++ b/policies/calico/calico-policy-tutorial/README.md @@ -0,0 +1,67 @@ +## Calico Network Policy Tutorial + +This tutorial reproduces the [Calico policy tutorial](https://docs.tigera.io/calico/latest/network-policy/get-started/calico-policy/calico-policy-tutorial) on an AKS cluster (or the LocalStack for Azure emulator). It builds a zero-trust posture step by step: start with open connectivity, lock everything down with a cluster-wide default-deny, then selectively re-open egress and ingress until a single allowed path works while everything else stays blocked. + +Calico policy is expressed with two [Project Calico](https://docs.tigera.io/calico/latest/reference/resources/globalnetworkpolicy) custom resources, applied with the `calicoctl` CLI rather than `kubectl`: + +- A cluster-wide `GlobalNetworkPolicy` that denies all ingress and egress for every namespace except the system namespaces. +- Namespaced `NetworkPolicy` resources that allow specific egress and ingress on top of that baseline. + +The demo runs in the `advanced-policy-demo` namespace with an `nginx` Deployment fronted by a ClusterIP Service and a busybox `access` pod used to probe connectivity. + +## Prerequisites + +- An AKS cluster reachable through `kubectl`, created with the **Calico** network-policy option of [scripts/01-user-assigned-managed-identity.sh](../../../scripts/01-user-assigned-managed-identity.sh) (the script's menu offers Azure, Cilium, and Calico network policy; pick Calico). +- [kubectl](https://kubernetes.io/docs/tasks/tools/) configured for the cluster. +- `sudo` access on the host: `00-install-calicoctl.sh` installs the `calicoctl` binary to `/usr/local/bin`. + +## How it works + +Run the numbered scripts in order. Each verification step uses `kubectl exec` into the `access` pod and compares an allowed response (HTML) against a blocked one (`bad address` or a timeout). + +| Script | What it does | Expected result | +| --- | --- | --- | +| `00-install-calicoctl.sh` | Installs the `calicoctl` CLI (required for the Calico CRDs). | CLI installed. | +| `01-deploy-demo.sh` | Deploys the `advanced-policy-demo` namespace, the `nginx` Deployment and Service, and the `access` pod. | Workloads become Ready. | +| `02-verify-access-allowed.sh` | Baseline connectivity from the `access` pod, before any policy. | `nginx` and `google.com` both reachable. | +| `03-create-default-deny-policy.sh` | Applies the cluster-wide default-deny `GlobalNetworkPolicy`. | Baseline lockdown in force. | +| `04-verify-access-denied.sh` | Re-tests connectivity under default-deny. | Both blocked; even DNS lookups fail. | +| `05-create-allow-busybox-egress-policy.sh` | Namespaced policy allowing all egress from the `access` pod. | Egress re-opened for `access`. | +| `06-verify-egress-allowed.sh` | Re-tests connectivity. | `google.com` reachable; `nginx` still blocked (no ingress rule yet). | +| `07-create-allow-nginx-ingress-policy.sh` | Namespaced policy allowing ingress to `nginx` from the `access` pod. | Ingress to `nginx` re-opened. | +| `08-verify-ingress-allowed.sh` | Re-tests connectivity. | `nginx` and `google.com` both reachable; everything else stays denied. | +| `09-get-policies.sh` | Lists and inspects the applied global and namespaced policies. | Policies shown. | +| `10-cleanup.sh` | Deletes the policies and the `advanced-policy-demo` namespace. | Demo removed. | + +```bash +cd policies/calico/calico-policy-tutorial +./00-install-calicoctl.sh +./01-deploy-demo.sh +./02-verify-access-allowed.sh +# ... continue through 10-cleanup.sh in order +``` + +## Scripts + +- `00-install-calicoctl.sh`: Downloads and installs the `calicoctl` CLI (required because Calico policies are `projectcalico.org/v3` custom resources, not stock Kubernetes objects). +- `01-deploy-demo.sh`: Applies `demo.yaml` and waits for the `nginx` Deployment and the `access` pod to become Ready. +- `02-verify-access-allowed.sh`: Confirms the baseline, where the `access` pod can reach both the in-cluster `nginx` Service and the public internet. +- `03-create-default-deny-policy.sh`: Applies `default-deny.yaml`, a cluster-wide `GlobalNetworkPolicy` denying all ingress and egress outside the system namespaces. +- `04-verify-access-denied.sh`: Confirms that under default-deny every connection from the `access` pod fails, including DNS resolution. +- `05-create-allow-busybox-egress-policy.sh`: Applies `allow-busybox-egress.yaml`, a namespaced `NetworkPolicy` allowing all egress from the `access` pod. +- `06-verify-egress-allowed.sh`: Confirms egress now works while ingress to `nginx` remains blocked. +- `07-create-allow-nginx-ingress-policy.sh`: Applies `allow-nginx-ingress.yaml`, a namespaced `NetworkPolicy` allowing ingress to `nginx` from the `access` pod. +- `08-verify-ingress-allowed.sh`: Confirms the full allowed path works while all other traffic stays denied. +- `09-get-policies.sh`: Lists the `GlobalNetworkPolicy` and namespaced `NetworkPolicy` resources and dumps the default-deny policy. +- `10-cleanup.sh`: Deletes the namespaced policies, the global default-deny policy, and the `advanced-policy-demo` namespace. + +The Kubernetes and Calico manifests applied by the scripts are `demo.yaml`, `default-deny.yaml`, `allow-busybox-egress.yaml`, and `allow-nginx-ingress.yaml`. + +## Resources + +- [Calico policy tutorial](https://docs.tigera.io/calico/latest/network-policy/get-started/calico-policy/calico-policy-tutorial) +- [Kubernetes policy, advanced tutorial](https://docs.tigera.io/calico/latest/network-policy/get-started/kubernetes-policy/kubernetes-policy-advanced) +- [Install calicoctl](https://docs.tigera.io/calico/latest/operations/calicoctl/install) +- [Calico GlobalNetworkPolicy reference](https://docs.tigera.io/calico/latest/reference/resources/globalnetworkpolicy) +- [Calico NetworkPolicy reference](https://docs.tigera.io/calico/latest/reference/resources/networkpolicy) +- [Adopt a zero trust network model for security](https://docs.tigera.io/calico/latest/network-policy/adopt-zero-trust) diff --git a/policies/cilium/egress-tutorial/11-cleanup.sh b/policies/cilium/egress-tutorial/11-cleanup.sh index 0844b01..047849f 100755 --- a/policies/cilium/egress-tutorial/11-cleanup.sh +++ b/policies/cilium/egress-tutorial/11-cleanup.sh @@ -3,17 +3,17 @@ # Variables namespace="starwars" -# Controlla se il namespace esiste nel cluster +# Check whether the namespace exists in the cluster if kubectl get namespace "$namespace" >/dev/null 2>&1; then echo "Deleting [$namespace] namespace and all its resources..." kubectl delete namespace "$namespace" --wait=false - # Aspetta il completamento, intercettando il timeout + # Wait for the deletion to complete, catching the timeout if ! kubectl wait --for=delete namespace/$namespace --timeout=60s 2>/dev/null; then echo "Namespace deletion is taking longer than expected. Forcing finalizer removal..." kubectl patch namespace "$namespace" -p '{"spec":{"finalizers":[]}}' --type=merge 2>/dev/null - # Ultima verifica breve + # Final short check kubectl wait --for=delete namespace/$namespace --timeout=10s 2>/dev/null fi echo "[$namespace] namespace and all its resources have been deleted" diff --git a/policies/cilium/egress-tutorial/README.md b/policies/cilium/egress-tutorial/README.md new file mode 100644 index 0000000..59e384c --- /dev/null +++ b/policies/cilium/egress-tutorial/README.md @@ -0,0 +1,63 @@ +## Cilium DNS-Aware Egress Policy Tutorial + +This tutorial demonstrates DNS-aware (FQDN) egress control with Cilium on an AKS cluster (or the LocalStack for Azure emulator). A `CiliumNetworkPolicy` restricts which external hostnames a pod may reach, and progressively tightens from an exact hostname, to a wildcard pattern, to a pattern locked to a single port. + +It uses the Cilium [Star Wars demo](https://cilium.io/blog/2017/5/4/demo-may-the-force-be-with-you/): a `mediabot` pod (labels `org: empire`, `class: mediabot`) in the `starwars` namespace, which issues outbound `curl` calls to GitHub hostnames. Because Cilium enforces FQDN rules by observing DNS, each policy also allows DNS to kube-dns so the pod can still resolve names. The three policies are all named `fqdn`, so each `kubectl apply` overwrites the previous one. + +## Prerequisites + +- An AKS cluster reachable through `kubectl`, created with the **Cilium** network-policy option of [scripts/01-user-assigned-managed-identity.sh](../../../scripts/01-user-assigned-managed-identity.sh) (the script's menu offers Azure, Cilium, and Calico network policy; pick Cilium). +- [kubectl](https://kubernetes.io/docs/tasks/tools/) configured for the cluster. +- The `cilium` and `hubble` CLIs, installed by `00-install-cilium-hubble-cli.sh` (needs `sudo`). `10-cilium-endpoint-list.sh` execs into the `cilium-agent` pod (`k8s-app=cilium` in `kube-system`). + +## How it works + +Run the numbered scripts in order. Each `*-call-services.sh` step execs into `mediabot` and `curl`s a set of hostnames, printing the expected allowed/blocked outcome per target. + +| Script | What it does | Expected result | +| --- | --- | --- | +| `00-install-cilium-hubble-cli.sh` | Installs the `cilium` and `hubble` CLIs. | CLIs installed. | +| `01-deploy-demo.sh` | Deploys the `starwars` namespace and the `mediabot` pod. | `mediabot` becomes Ready. | +| `02-call-services.sh` | Baseline egress, before any policy. | All GitHub hostnames reachable. | +| `03-create-dns-matchname-policy.sh` | Applies `dns-matchname.yaml`, allowing egress only to the exact FQDN `api.github.com`. | Allow-list in force. | +| `04-call-services.sh` | Re-tests. | `api.github.com` reachable; `status.github.com` and the apex `github.com` blocked. | +| `05-create-dns-pattern-policy.sh` | Applies `dns-pattern.yaml`, allowing the wildcard pattern `*.github.com`. | Pattern in force. | +| `06-call-services.sh` | Re-tests. | `api.github.com` and `status.github.com` reachable; the apex `github.com` blocked (the pattern requires a subdomain label). | +| `07-create-dns-port-policy.sh` | Applies `dns-port.yaml`, restricting `*.github.com` to port `443/TCP`. | Port restriction in force. | +| `08-call-services.sh` | Re-tests. | HTTPS to `*.github.com` reachable; HTTP (port 80) blocked; the apex `github.com` blocked. | +| `09-get-policy.sh` | Dumps the `fqdn` `CiliumNetworkPolicy`. | Policy shown. | +| `10-cilium-endpoint-list.sh` | Lists the Cilium endpoints on the node running `mediabot`. | Endpoint state shown. | +| `11-cleanup.sh` | Deletes the `starwars` namespace. | Demo removed. | + +```bash +cd policies/cilium/egress-tutorial +./00-install-cilium-hubble-cli.sh +./01-deploy-demo.sh +./02-call-services.sh +# ... continue through 11-cleanup.sh in order +``` + +## Scripts + +- `00-install-cilium-hubble-cli.sh`: Downloads and installs the `cilium` and `hubble` CLIs. +- `01-deploy-demo.sh`: Applies `dns-sw-app.yaml` (the `mediabot` pod) and waits for it to become Ready. +- `02-call-services.sh`: Baseline egress, where `mediabot` can reach every GitHub hostname over HTTP and HTTPS. +- `03-create-dns-matchname-policy.sh`: Applies `dns-matchname.yaml`, a `CiliumNetworkPolicy` allowing egress only to the exact FQDN `api.github.com` (plus DNS to kube-dns). +- `04-call-services.sh`: Confirms that only `api.github.com` is reachable and other hostnames are blocked. +- `05-create-dns-pattern-policy.sh`: Applies `dns-pattern.yaml`, widening the allow-list to the pattern `*.github.com`. +- `06-call-services.sh`: Confirms that subdomains of `github.com` are reachable while the apex domain is not. +- `07-create-dns-port-policy.sh`: Applies `dns-port.yaml`, restricting `*.github.com` egress to port `443/TCP`. +- `08-call-services.sh`: Confirms that only HTTPS to `*.github.com` succeeds and HTTP is blocked. +- `09-get-policy.sh`: Dumps the current `fqdn` `CiliumNetworkPolicy` as YAML. +- `10-cilium-endpoint-list.sh`: Runs `cilium endpoint list` from the `cilium-agent` on the node hosting `mediabot`. +- `11-cleanup.sh`: Deletes the `starwars` namespace. + +The demo pod and the three policies are defined in `dns-sw-app.yaml`, `dns-matchname.yaml`, `dns-pattern.yaml`, and `dns-port.yaml`. + +## Resources + +- [Cilium security tutorials](https://docs.cilium.io/en/latest/security/tutorial-toc/) +- [Locking down external access with DNS-based policies](https://docs.cilium.io/en/latest/security/dns/) +- [Cilium network policy](https://docs.cilium.io/en/latest/security/policy/#id1) +- [Install the Cilium CLI](https://docs.cilium.io/en/latest/gettingstarted/k8s-install-default/#install-the-cilium-cli) +- [Star Wars demo: may the force be with you](https://cilium.io/blog/2017/5/4/demo-may-the-force-be-with-you/) diff --git a/policies/cilium/ingress-tutorial/12-cleanup.sh b/policies/cilium/ingress-tutorial/12-cleanup.sh index 0844b01..047849f 100755 --- a/policies/cilium/ingress-tutorial/12-cleanup.sh +++ b/policies/cilium/ingress-tutorial/12-cleanup.sh @@ -3,17 +3,17 @@ # Variables namespace="starwars" -# Controlla se il namespace esiste nel cluster +# Check whether the namespace exists in the cluster if kubectl get namespace "$namespace" >/dev/null 2>&1; then echo "Deleting [$namespace] namespace and all its resources..." kubectl delete namespace "$namespace" --wait=false - # Aspetta il completamento, intercettando il timeout + # Wait for the deletion to complete, catching the timeout if ! kubectl wait --for=delete namespace/$namespace --timeout=60s 2>/dev/null; then echo "Namespace deletion is taking longer than expected. Forcing finalizer removal..." kubectl patch namespace "$namespace" -p '{"spec":{"finalizers":[]}}' --type=merge 2>/dev/null - # Ultima verifica breve + # Final short check kubectl wait --for=delete namespace/$namespace --timeout=10s 2>/dev/null fi echo "[$namespace] namespace and all its resources have been deleted" diff --git a/policies/cilium/ingress-tutorial/README.md b/policies/cilium/ingress-tutorial/README.md new file mode 100644 index 0000000..a529d76 --- /dev/null +++ b/policies/cilium/ingress-tutorial/README.md @@ -0,0 +1,76 @@ +## Cilium L3/L4 and L7 Ingress Policy Tutorial + +This tutorial demonstrates identity-aware ingress control with Cilium on an AKS cluster (or the LocalStack for Azure emulator), first at L3/L4 (which workloads may connect) and then at L7 (which HTTP calls they may make). + +It uses the Cilium [Star Wars demo](https://cilium.io/blog/2017/5/4/demo-may-the-force-be-with-you/) in the `starwars` namespace: a `deathstar` Deployment (two replicas, labels `org: empire`, `class: deathstar`) fronted by a ClusterIP Service on port 80, plus two client pods, `tiefighter` (`org: empire`) and `xwing` (`org: alliance`). Both policies are named `rule1`, so applying the L7 policy overwrites the L3/L4 one. + +Without any policy, every pod can reach the `deathstar` API: + +![Star Wars demo, no policy](cilium_http_gsg.png) + +An L3/L4 `CiliumNetworkPolicy` restricts the `deathstar` to empire ships on port 80/TCP, so the `xwing` is cut off entirely: + +![L3/L4 policy](cilium_http_l3_l4_gsg.png) + +An L7 policy adds an HTTP filter so even empire ships may only call `POST /v1/request-landing`; the dangerous `PUT /v1/exhaust-port` is denied: + +![L3/L4/L7 policy](cilium_http_l3_l4_l7_gsg.png) + +## Prerequisites + +- An AKS cluster reachable through `kubectl`, created with the **Cilium** network-policy option of [scripts/01-user-assigned-managed-identity.sh](../../../scripts/01-user-assigned-managed-identity.sh) (the script's menu offers Azure, Cilium, and Calico network policy; pick Cilium). +- [kubectl](https://kubernetes.io/docs/tasks/tools/) configured for the cluster. +- The `cilium` and `hubble` CLIs, installed by `00-install-cilium-hubble-cli.sh` (needs `sudo`). Several scripts exec into the `cilium-agent` pod (`k8s-app=cilium` in `kube-system`). + +## How it works + +Run the numbered scripts in order. The `*-call-*.sh` steps exec into the client pods and `curl` the `deathstar` Service, printing the expected allowed/blocked outcome. + +| Script | What it does | Expected result | +| --- | --- | --- | +| `00-install-cilium-hubble-cli.sh` | Installs the `cilium` and `hubble` CLIs. | CLIs installed. | +| `01-deploy-demo.sh` | Deploys the `starwars` namespace, the `deathstar` Deployment and Service, and the `tiefighter` and `xwing` pods. | Workloads become Ready. | +| `02-cilium-endpoint-list.sh` | Lists the Cilium endpoints on the node running `deathstar`. | Endpoint state shown. | +| `03-cilium-policy-get.sh` | Dumps the policy currently loaded in a `cilium-agent`. | Loaded policy shown. | +| `04-cilium-monitor.sh` | Interactive: streams L7 events from a selected node's `cilium-agent` (optional observability). | Live L7 monitor. | +| `05-call-request-landing-web-method.sh` | Baseline: both ships `POST /v1/request-landing`. | Both succeed (no policy yet). | +| `06-call-other-web-method.sh` | Baseline: both ships `PUT /v1/exhaust-port`. | Both succeed (no policy yet). | +| `07-create-l3-l4-policy.sh` | Applies `sw-l3-l4-policy.yaml`: only `org=empire` may reach `deathstar` on 80/TCP. | L3/L4 policy in force. | +| `08-call-deathstar-methods.sh` | Re-tests both methods from both ships (L3/L4 only). | `tiefighter` (empire) succeeds on both methods; `xwing` (alliance) times out, blocked at L3. | +| `09-create-l3-l4-l7-policy.sh` | Applies `sw-l3-l4-l7-policy.yaml`: adds an HTTP filter allowing only `POST /v1/request-landing`. | L7 policy in force. | +| `10-call-deathstar-methods.sh` | Re-tests both methods from both ships. | `tiefighter` `POST /v1/request-landing` succeeds; its `PUT /v1/exhaust-port` is denied at L7; `xwing` still blocked at L3. | +| `11-get-policy.sh` | Dumps the `rule1` `CiliumNetworkPolicy`. | Policy shown. | +| `12-cleanup.sh` | Deletes the `starwars` namespace. | Demo removed. | + +```bash +cd policies/cilium/ingress-tutorial +./00-install-cilium-hubble-cli.sh +./01-deploy-demo.sh +# ... continue through 12-cleanup.sh in order +``` + +## Scripts + +- `00-install-cilium-hubble-cli.sh`: Downloads and installs the `cilium` and `hubble` CLIs. +- `01-deploy-demo.sh`: Applies `http-sw-app.yaml` and waits for the `deathstar` Deployment and the `tiefighter` and `xwing` pods to become Ready. +- `02-cilium-endpoint-list.sh`: Runs `cilium endpoint list` from the `cilium-agent` on the node hosting `deathstar`. +- `03-cilium-policy-get.sh`: Runs `cilium policy get` from a `cilium-agent` to show the loaded policy. +- `04-cilium-monitor.sh`: Interactively selects a node and streams L7 (`--type l7`) events with `cilium monitor`, so you can watch allowed and denied HTTP calls live. +- `05-call-request-landing-web-method.sh`: Baseline, where both `tiefighter` and `xwing` can `POST /v1/request-landing`. +- `06-call-other-web-method.sh`: Baseline, where both ships can `PUT /v1/exhaust-port`. +- `07-create-l3-l4-policy.sh`: Applies `sw-l3-l4-policy.yaml`, an L3/L4 `CiliumNetworkPolicy` restricting `deathstar` ingress to `org=empire` on port 80/TCP. +- `08-call-deathstar-methods.sh`: Confirms the L3/L4 result: `tiefighter` reaches both methods while `xwing` is blocked. +- `09-create-l3-l4-l7-policy.sh`: Applies `sw-l3-l4-l7-policy.yaml`, adding an L7 HTTP filter that allows only `POST /v1/request-landing`. +- `10-call-deathstar-methods.sh`: Confirms the L7 result: `tiefighter` may only call the allowed method, and `PUT /v1/exhaust-port` is denied. +- `11-get-policy.sh`: Dumps the current `rule1` `CiliumNetworkPolicy` as YAML. +- `12-cleanup.sh`: Deletes the `starwars` namespace. + +The demo workloads and the two policies are defined in `http-sw-app.yaml`, `sw-l3-l4-policy.yaml`, and `sw-l3-l4-l7-policy.yaml`. + +## Resources + +- [Cilium security tutorials](https://docs.cilium.io/en/latest/security/tutorial-toc/) +- [Inspecting and enforcing HTTP with Cilium](https://docs.cilium.io/en/latest/security/http/#deploy-the-demo-application) +- [Cilium network policy](https://docs.cilium.io/en/latest/security/policy/#id1) +- [Install the Cilium CLI](https://docs.cilium.io/en/latest/gettingstarted/k8s-install-default/#install-the-cilium-cli) +- [Star Wars demo: may the force be with you](https://cilium.io/blog/2017/5/4/demo-may-the-force-be-with-you/)