app-routing.md

title	description	ms.subservice	ms.custom	author	ms.topic	ms.date	ms.author
Azure Kubernetes Service (AKS) managed NGINX ingress with the application routing add-on	Use the application routing add-on to securely access applications deployed on Azure Kubernetes Service (AKS).	aks-networking	devx-track-azurecli	asudbring	how-to	11/21/2023	allensu

Managed NGINX ingress with the application routing add-on

One way to route Hypertext Transfer Protocol (HTTP) and secure (HTTPS) traffic to applications running on an Azure Kubernetes Service (AKS) cluster is to use the Kubernetes Ingress object. When you create an Ingress object that uses the application routing add-on NGINX Ingress classes, the add-on creates, configures, and manages one or more Ingress controllers in your AKS cluster.

This article shows you how to deploy and configure a basic Ingress controller in your AKS cluster.

Application routing add-on with NGINX features

The application routing add-on with NGINX delivers the following:

• Easy configuration of managed NGINX Ingress controllers based on Kubernetes NGINX Ingress controller.
• Integration with Azure DNS for public and private zone management
• SSL termination with certificates stored in Azure Key Vault.

For other configurations, see:

• DNS and SSL configuration
• Application routing add-on configuration
• Configure internal NGIX ingress controller for Azure private DNS zone.

With the retirement of Open Service Mesh (OSM) by the Cloud Native Computing Foundation (CNCF), using the application routing add-on with OSM is not recommended.

Prerequisites

• An Azure subscription. If you don't have an Azure subscription, you can create a free account.
• Azure CLI version 2.54.0 or later installed and configured. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.

Limitations

• The application routing add-on supports up to five Azure DNS zones.
• All global Azure DNS zones integrated with the add-on have to be in the same resource group.
• All private Azure DNS zones integrated with the add-on have to be in the same resource group.
• Editing the ingress-nginx ConfigMap in the app-routing-system namespace isn't supported.
• The following snippet annotations are blocked and will prevent an Ingress from being configured: load_module, lua_package, _by_lua, location, root, proxy_pass, serviceaccount, {, }, '.

Enable application routing using Azure CLI

Default

Enable on a new cluster

To enable application routing on a new cluster, use the az aks create command, specifying the --enable-app-routing flag.

az aks create \
    --resource-group <ResourceGroupName> \
    --name <ClusterName> \
    --location <Location> \
    --enable-app-routing \
    --generate-ssh-keys

Enable on an existing cluster

To enable application routing on an existing cluster, use the az aks approuting enable or the az aks enable-addons command with the --addons parameter set to http_application_routing.

# az aks approuting enable
az aks approuting enable --resource-group <ResourceGroupName> --name <ClusterName>

# az aks enable-addons
az aks enable-addons --resource-group <ResourceGroupName> --name <ClusterName> --addons http_application_routing

Open Service Mesh (OSM) (retired)

Note

Open Service Mesh (OSM) has been retired by the CNCF. Creating Ingresses using the application routing add-on with OSM integration is not recommended and will be retired.

The following add-ons are required to support this configuration:

• open-service-mesh: If you require encrypted intra cluster traffic (recommended) between the NGINX Ingress and your services, the Open Service Mesh add-on is required which provides mutual TLS (mTLS).

Enable on a new cluster

Enable application routing on a new AKS cluster using the az aks create command specifying the --enable-app-routing flag and the --enable-addons parameter with the open-service-mesh add-on:

az aks create \
    --resource-group <ResourceGroupName> \
    --name <ClusterName> \
    --location <Location> \
    --enable-app-routing \
    --enable-addons open-service-mesh \
    --generate-ssh-keys 

Enable on an existing cluster

To enable application routing on an existing cluster, use the az aks approuting enable command and the az aks enable-addons command with the --addons parameter set to open-service-mesh:

az aks approuting enable --resource-group <ResourceGroupName> --name <ClusterName>
az aks enable-addons --resource-group <ResourceGroupName> --name <ClusterName> --addons open-service-mesh

Note

To use the add-on with Open Service Mesh, you should install the osm command-line tool. This command-line tool contains everything needed to configure and manage Open Service Mesh. The latest binaries are available on the OSM GitHub releases page.

Service annotations (retired)

Warning

Configuring Ingresses by adding annotations on the Service object is retired. Please consider configuring using an Ingress object.

Enable on a new cluster

To enable application routing on a new cluster, use the az aks create command, specifying --enable-app-routing flag.

az aks create \
    --resource-group <ResourceGroupName> \
    --name <ClusterName> \
    --location <Location> \
    --enable-app-routing \
    --generate-ssh-keys

Enable on an existing cluster

To enable application routing on an existing cluster, use the az aks approuting enable command:

az aks approuting enable --resource-group <ResourceGroupName> --name <ClusterName>

---

Connect to your AKS cluster

To connect to the Kubernetes cluster from your local computer, you use kubectl, the Kubernetes command-line client. You can install it locally using the az aks install-cli command. If you use the Azure Cloud Shell, kubectl is already installed.

Configure kubectl to connect to your Kubernetes cluster using the az aks get-credentials command.

az aks get-credentials --resource-group <ResourceGroupName> --name <ClusterName>

Deploy an application

The application routing add-on uses annotations on Kubernetes Ingress objects to create the appropriate resources.

Application routing add-on

1. Create the application namespace called hello-web-app-routing to run the example pods using the kubectl create namespace command.

   kubectl create namespace hello-web-app-routing

2. Create the deployment by copying the following YAML manifest into a new file named deployment.yaml and save the file to your local computer.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: aks-helloworld  
     namespace: hello-web-app-routing
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: aks-helloworld
     template:
       metadata:
         labels:
           app: aks-helloworld
       spec:
         containers:
         - name: aks-helloworld
           image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
           ports:
           - containerPort: 80
           env:
           - name: TITLE
             value: "Welcome to Azure Kubernetes Service (AKS)"

3. Create the service by copying the following YAML manifest into a new file named service.yaml and save the file to your local computer.

   apiVersion: v1
   kind: Service
   metadata:
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     type: ClusterIP
     ports:
     - port: 80
     selector:
       app: aks-helloworld

Create the Ingress object

The application routing add-on creates an Ingress class on the cluster named webapprouting.kubernetes.azure.com. When you create an Ingress object with this class, it activates the add-on.

1. Copy the following YAML manifest into a new file named ingress.yaml and save the file to your local computer.

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     ingressClassName: webapprouting.kubernetes.azure.com
     rules:
     - host: <Hostname>
       http:
         paths:
         - backend:
             service:
               name: aks-helloworld
               port:
                 number: 80
           path: /
           pathType: Prefix

2. Create the cluster resources using the kubectl apply command.

   kubectl apply -f deployment.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   deployment.apps/aks-helloworld created
   

   kubectl apply -f service.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   service/aks-helloworld created
   

   kubectl apply -f ingress.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   ingress.networking.k8s.io/aks-helloworld created
   

Open Service Mesh (retired)

1. Create a namespace called hello-web-app-routing to run the exmaple pods using the kubectl create namespace command.

   kubectl create namespace hello-web-app-routing

2. Add the application namespace to the OSM control plane using the osm namespace add command.

   osm namespace add hello-web-app-routing

3. Create the deployment by copying the following YAML manifest into a new file named deployment.yaml and save the file to your local computer.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: aks-helloworld  
     namespace: hello-web-app-routing
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: aks-helloworld
     template:
       metadata:
         labels:
           app: aks-helloworld
       spec:
         containers:
         - name: aks-helloworld
           image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
           ports:
           - containerPort: 80
           env:
           - name: TITLE
             value: "Welcome to Azure Kubernetes Service (AKS)"

4. Create the service by copying the following YAML manifest into a new file named service.yaml and save the file to your local computer.

   apiVersion: v1
   kind: Service
   metadata:
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     type: ClusterIP
     ports:
     - port: 80
     selector:
       app: aks-helloworld

Create the Ingress object

The application routing add-on creates an Ingress class on the cluster called webapprouting.kubernetes.azure.com. When you create an Ingress object with this class, it activates the add-on. The kubernetes.azure.com/use-osm-mtls: "true" annotation on the Ingress object creates an Open Service Mesh (OSM) IngressBackend to configure a backend service to accept Ingress traffic from trusted sources.

1. Copy the following YAML manifest into a new file named ingress.yaml and save the file to your local computer.

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     annotations:
       kubernetes.azure.com/use-osm-mtls: "true"
       nginx.ingress.kubernetes.io/backend-protocol: HTTPS
       nginx.ingress.kubernetes.io/configuration-snippet: |2-
         proxy_ssl_name "default.hello-web-app-routing.cluster.local";
       nginx.ingress.kubernetes.io/proxy-ssl-secret: kube-system/osm-ingress-client-cert
       nginx.ingress.kubernetes.io/proxy-ssl-verify: "on"
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     ingressClassName: webapprouting.kubernetes.azure.com
     rules:
     - host: <Hostname>
       http:
         paths:
         - backend:
             service:
               name: aks-helloworld
               port:
                 number: 80
           path: /
           pathType: Prefix

2. Create the cluster resources using the kubectl apply command.

   kubectl apply -f deployment.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   deployment.apps/aks-helloworld created
   

   kubectl apply -f service.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   service/aks-helloworld created
   

   kubectl apply -f ingress.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   ingress.networking.k8s.io/aks-helloworld created
   

Service annotations (retired)

Warning

Configuring Ingresses by adding annotations on the Service object is retired. Please consider configuring using an Ingress object.

Create application namespace

1. Create a namespace called hello-web-app-routing to run the exmaple pods using the kubectl create namespace command.

   kubectl create namespace hello-web-app-routing

2. Add the application namespace to the OSM control plane using the osm namespace add command.

   osm namespace add hello-web-app-routing

3. Create the deployment by copying the following YAML manifest into a new file named deployment.yaml and save the file to your local computer.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: aks-helloworld  
     namespace: hello-web-app-routing
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: aks-helloworld
     template:
       metadata:
         labels:
           app: aks-helloworld
       spec:
         containers:
         - name: aks-helloworld
           image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
           ports:
           - containerPort: 80
           env:
           - name: TITLE
             value: "Welcome to Azure Kubernetes Service (AKS)"

4. Create the service by copying the following YAML manifest into a new file named service.yaml and save the file to your local computer.

   apiVersion: v1
   kind: Service
   metadata:
     name: aks-helloworld
     namespace: hello-web-app-routing
   spec:
     type: ClusterIP
     ports:
     - port: 80
     selector:
       app: aks-helloworld

5. Create the cluster resources using the kubectl apply command.

   kubectl apply -f deployment.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   deployment.apps/aks-helloworld created
   

   kubectl apply -f service.yaml -n hello-web-app-routing

   The following example output shows the created resource:

   service/aks-helloworld created
   

---

Verify the managed Ingress was created

You can verify the managed Ingress was created using the kubectl get ingress command.

kubectl get ingress -n hello-web-app-routing

The following example output shows the created managed Ingress:

NAME             CLASS                                HOSTS               ADDRESS       PORTS     AGE
aks-helloworld   webapprouting.kubernetes.azure.com   myapp.contoso.com   20.51.92.19   80, 443   4m

Remove the application routing add-on

To remove the associated namespace, use the kubectl delete namespace command.

kubectl delete namespace hello-web-app-routing

To remove the application routing add-on from your cluster, use the az aks approuting disable command.

az aks approuting disable --name myAKSCluster --resource-group myResourceGroup 

When the application routing add-on is disabled, some Kubernetes resources might remain in the cluster. These resources include configMaps and secrets and are created in the app-routing-system namespace. You can remove these resources if you want.

Next steps

• Configure custom ingress configurations shows how to create an advanced Ingress configuration and configure a custom domain using Azure DNS to manage DNS zones and setup a secure ingress.

• To integrate with an Azure internal load balancer and configure a private Azure DNS zone to enable DNS resolution for the private endpoints to resolve specific domains, see Configure internal NGINX ingress controller for Azure private DNS zone.

• Learn about monitoring the ingress-nginx controller metrics included with the application routing add-on with with Prometheus in Grafana (preview) as part of analyzing the performance and usage of your application.