doc: Move Azure IPAM out of beta

[ upstream commit 5192495 ]

Mark the Azure IPAM mode as the preferred way to install on AKS and
Azure. This also allows to simplify the inlining of all Azure related
docs.

Signed-off-by: Thomas Graf <thomas@cilium.io>
Signed-off-by: Aditi Ghag <aditi@cilium.io>

Documentation/concepts/networking/ipam/azure.rst

@@ -6,9 +6,9 @@
 
 .. _ipam_azure:
 
-#################
-Azure IPAM (beta)
-#################
+##########
+Azure IPAM
+##########
 
 The Azure IPAM allocator is specific to Cilium deployments running in the Azure
 cloud and performs IP allocation based on `Azure Private IP addresses
@@ -20,11 +20,6 @@ watermark allows to maintain a number of IP addresses to be available for use
 on nodes at all time without requiring to contact the Azure API when a new pod
 is scheduled in the cluster.
 
-.. note::
-
-    This is a beta feature. Please provide feedback and file a GitHub issue if
-    you experience any problems.
-
 ************
 Architecture
 ************

Documentation/gettingstarted/cni-chaining-azure-cni.rst

@@ -8,34 +8,124 @@
 Azure CNI
 *********
 
-This guide explains how to set up Cilium in combination with Azure CNI. In this
-hybrid mode, the Azure CNI plugin is responsible for setting up the virtual
-network devices as well as address allocation (IPAM). After the initial
-networking is setup, the Cilium CNI plugin is called to attach eBPF programs to
-the network devices set up by Azure CNI to enforce network policies, perform
-load-balancing, and encryption.
-
 .. note::
 
-   If you are looking to install Cilium on Azure AKS, see the guide
-   :ref:`k8s_install_aks` for a complete guide also covering cluster setup.
+   This is not the best option to run Cilium on AKS or Azure. Please refer to
+   :ref:`k8s_install_aks` for the best guide to run Cilium in Azure Cloud.
+   Follow this guide if you specifically want to run Cilium in combination with
+   the Azure CNI in a chaining configuration.
+
+This guide explains how to set up Cilium in combination with Azure CNI in a
+chaining configuration. In this hybrid mode, the Azure CNI plugin is
+responsible for setting up the virtual network devices as well as address
+allocation (IPAM). After the initial networking is setup, the Cilium CNI plugin
+is called to attach eBPF programs to the network devices set up by Azure CNI to
+enforce network policies, perform load-balancing, and encryption.
+
+
+Create an AKS + Cilium CNI configuration
+========================================
+
+Create a ``chaining.yaml`` file based on the following template to specify the
+desired CNI chaining configuration. This ConfigMap will be installed as the CNI
+configuration file on all nodes and defines the chaining configuration. In the
+example below, the Azure CNI, portmap, and Cilium are chained together.
+
+.. code:: yaml
+
+    apiVersion: v1
+    kind: ConfigMap
+    metadata:
+      name: cni-configuration
+      namespace: cilium
+    data:
+      cni-config: |-
+        {
+          "cniVersion": "0.3.0",
+          "name": "azure",
+          "plugins": [
+            {
+              "type": "azure-vnet",
+              "mode": "transparent",
+              "ipam": {
+                 "type": "azure-vnet-ipam"
+               }
+            },
+            {
+              "type": "portmap",
+              "capabilities": {"portMappings": true},
+              "snat": true
+            },
+            {
+               "name": "cilium",
+               "type": "cilium-cni"
+            }
+          ]
+        }
+
+Create the cilium namespace:
+
+.. code:: bash
+
+   kubectl create namespace cilium
+
+
+Deploy the `ConfigMap`:
+
+.. code:: bash
+
+   kubectl apply -f chaining.yaml
+
+
+Deploy Cilium
+=============
+
+.. include:: k8s-install-download-release.rst
+
+Deploy Cilium release via Helm:
+
+.. parsed-literal::
+
+   helm install cilium |CHART_RELEASE| \\
+     --namespace cilium \\
+     --set cni.chainingMode=generic-veth \\
+     --set cni.customConf=true \\
+     --set nodeinit.enabled=true \\
+     --set cni.configMap=cni-configuration \\
+     --set tunnel=disabled \\
+     --set enableIPv4Masquerade=false
+
+This will create both the main cilium daemonset, as well as the cilium-node-init daemonset, which handles tasks like mounting the eBPF filesystem and updating the
+existing Azure CNI plugin to run in 'transparent' mode.
+
+.. include:: k8s-install-restart-pods.rst
+
+Validate the Installation
+=========================
+
+You can monitor as Cilium and all required components are being installed:
+
+.. code-block:: shell-session
+
+   $ kubectl -n cilium get pods --watch
+   cilium-2twr9                      0/1     Init:0/2            0          17s
+   cilium-fkhjv                      0/1     Init:0/2            0          17s
+   cilium-node-init-bhr5l            1/1     Running             0          17s
+   cilium-node-init-l77v9            1/1     Running             0          17s
+   cilium-operator-f8bd5cd96-qdspd   0/1     ContainerCreating   0          17s
+   cilium-operator-f8bd5cd96-tvdn6   0/1     ContainerCreating   0          17s
 
-.. include:: k8s-install-azure-cni-steps.rst
+It may take a couple of minutes for all components to come up:
 
-Restart existing pods
-=====================
+.. code-block:: shell-session
 
-The new CNI chaining configuration will *not* apply to any pod that is already
-running in the cluster. Existing pods will be reachable and Cilium will
-load-balance to them but policy enforcement will not apply to them and
-load-balancing is not performed for traffic originating from existing pods.
-You must restart these pods in order to invoke the chaining configuration on
-them.
+   cilium-operator-f8bd5cd96-tvdn6   1/1     Running             0          25s
+   cilium-operator-f8bd5cd96-qdspd   1/1     Running             0          26s
+   cilium-fkhjv                      1/1     Running             0          60s
+   cilium-2twr9                      1/1     Running             0          61s
 
-If you are unsure if a pod is managed by Cilium or not, run ``kubectl get cep``
-in the respective namespace and see if the pod is listed.
+.. include:: k8s-install-connectivity-test.rst
 
-.. include:: k8s-install-azure-cni-validate.rst
 .. include:: namespace-cilium.rst
 .. include:: hubble-enable.rst
 

Documentation/gettingstarted/k8s-install-aks.rst

@@ -4,24 +4,25 @@
     Please use the official rendered version released here:
     https://docs.cilium.io
 
+.. _k8s_azure:
 .. _k8s_install_aks:
 
 *************************
 Installation on Azure AKS
 *************************
 
-This guide covers installing Cilium into an Azure AKS environment. This guide
-will work when setting up AKS in both `Basic <https://docs.microsoft.com/en-us/azure/aks/concepts-network#kubenet-basic-networking>`_ and `Advanced 
-<https://docs.microsoft.com/en-us/azure/aks/concepts-network#azure-cni-advanced-networking>`_ networking mode.
+This guide covers installing Cilium into an Azure AKS environment using
+:ref:`ipam_azure`.
 
-This is achieved using Cilium in CNI chaining mode, with the Azure CNI plugin
-as the base CNI plugin and Cilium chaining on top to provide L3-L7
-observability, network policy enforcement, Kubernetes services
-implementation, as well as other advanced features like transparent encryption
-and clustermesh.
+Create an Azure Kubernetes cluster
+==================================
+
+Setup a Kubernetes cluster on Azure. You can use any method available as long
+as your Kubernetes cluster has CNI enabled in the kubelet configuration. For
+simplicity of this guide, we will set up a managed AKS cluster:
 
 Prerequisites
-=============
+-------------
 
 Ensure that you have the `Azure Cloud CLI 
 <https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest>`_ installed.
@@ -33,53 +34,115 @@ Kubernetes versions.
 
    az aks get-versions -l westus -o table
 
-Create an AKS Cluster
-=====================
+Deploy the Cluster
+------------------
+
+.. note:: **Do NOT specify the '--network-policy' flag** when creating the
+    cluster, as this will cause the Azure CNI plugin to push down unwanted
+    iptables rules.
+
+.. code:: bash
+
+   export RESOURCE_GROUP_NAME=aks-test
+   export CLUSTER_NAME=aks-test
+   export LOCATION=westus
+
+   az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
+   az aks create \
+       --resource-group $RESOURCE_GROUP_NAME \
+       --name $CLUSTER_NAME \
+       --location $LOCATION \
+       --node-count 2 \
+       --network-plugin azure
+
+.. note:: When setting up AKS, it is important to use the flag
+          ``--network-plugin azure`` to ensure that CNI mode is enabled.
+
+Create a service principal for cilium-operator
+==============================================
+
+In order to allow cilium-operator to interact with the Azure API, a service
+principal is required. You can reuse an existing service principal if you want
+but it is recommended to create a dedicated service principal for
+cilium-operator:
+
+.. code:: bash
+
+    az ad sp create-for-rbac --name cilium-operator > azure-sp.json
+
+The contents of ``azure-sp.json`` should look like this:
 
-You can use any method to create and deploy an AKS cluster with the exception
-of specifying the Network Policy option. Doing so will still work but will
-result in unwanted iptables rules being installed on all of your nodes.
+.. code:: bash
+
+    {
+      "appId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
+      "displayName": "cilium-operator",
+      "name": "http://cilium-operator",
+      "password": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb",
+      "tenant": "cccccccc-cccc-cccc-cccc-cccccccccccc"
+    }
 
-If you want to us the CLI to create a dedicated set of Azure resources
-(resource groups, networks, etc.) specifically for this tutorial, the following
-commands (borrowed from the AKS documentation) run as a script or manually all
-in the same terminal are sufficient.
+Extract the relevant credentials to access the Azure API:
 
-It can take 10+ minutes for the final command to be complete indicating that
-the cluster is ready.
+.. code:: bash
 
-.. include:: k8s-install-aks-create-cluster.rst
+    AZURE_SUBSCRIPTION_ID="$(az account show | jq -r .id)"
+    AZURE_CLIENT_ID="$(jq -r .appId < azure-sp.json)"
+    AZURE_CLIENT_SECRET="$(jq -r .password < azure-sp.json)"
+    AZURE_TENANT_ID="$(jq -r .tenant < azure-sp.json)"
+    AZURE_NODE_RESOURCE_GROUP="$(az aks show --resource-group $RESOURCE_GROUP_NAME --name $CLUSTER_NAME | jq -r .nodeResourceGroup)"
 
-Configure kubectl to Point to Newly Created Cluster
-===================================================
+.. note:: ``AZURE_NODE_RESOURCE_GROUP`` must be set to the resource group of the
+           node pool, *not* the resource group of the AKS cluster.
 
-Run the following commands to configure kubectl to connect to this
-AKS cluster:
+Retrieve Credentials to access cluster
+======================================
 
 .. include:: k8s-install-aks-get-credentials.rst
 
-To verify, you should see AKS in the name of the nodes when you run:
+Deploy Cilium
+=============
+
+.. include:: k8s-install-download-release.rst
 
-.. code-block:: shell-session
+Deploy Cilium release via Helm:
 
-    $ kubectl get nodes
-    NAME                                STATUS   ROLES   AGE   VERSION
-    aks-nodepool1-22314427-vmss000000   Ready    agent   91s   v1.17.11
-    aks-nodepool1-22314427-vmss000001   Ready    agent   91s   v1.17.11
+.. parsed-literal::
 
-.. include:: k8s-install-azure-cni-steps.rst
+   helm install cilium |CHART_RELEASE| \\
+     --namespace kube-system \\
+     --set azure.enabled=true \\
+     --set azure.resourceGroup=$AZURE_NODE_RESOURCE_GROUP \\
+     --set azure.subscriptionID=$AZURE_SUBSCRIPTION_ID \\
+     --set azure.tenantID=$AZURE_TENANT_ID \\
+     --set azure.clientID=$AZURE_CLIENT_ID \\
+     --set azure.clientSecret=$AZURE_CLIENT_SECRET \\
+     --set tunnel=disabled \\
+     --set ipam.mode=azure \\
+     --set enableIPv4Masquerade=false \\
+     --set nodeinit.enabled=true
 
-.. include:: k8s-install-azure-cni-validate.rst
-.. include:: namespace-cilium.rst
+.. include:: k8s-install-restart-pods.rst
+.. include:: k8s-install-validate.rst
+.. include:: namespace-kube-system.rst
 .. include:: hubble-enable.rst
 
-Delete the AKS Cluster
-======================
+.. _azure_limitations:
 
-Finally, delete the test cluster by running:
+Limitations
+===========
 
-.. code:: bash
+* All VMs and VM scale sets used in a cluster must belong to the same resource
+  group.
+
+.. _azure_troubleshooting:
+
+Troubleshooting
+===============
+* If ``kubectl exec`` to a pod fails to connect, restarting the ``tunnelfront`` pod may help.
+* Pods may fail to gain a ``.spec.hostNetwork`` status even if restarted and managed by Cilium.
+* If some connectivity tests fail to reach the ready state you may need to restart the unmanaged pods again.
+* Some connectivity tests may fail. This is being tracked in `Cilium GitHub issue #12113
+  <https://github.com/cilium/cilium/issues/12113>`_.
 
-    az aks delete \
-        --resource-group $RESOURCE_GROUP_NAME \
-        --name $CLUSTER_NAME
+  * ``hubble observe`` may report one or more nodes being unavailable and ``hubble-ui`` may fail to connect to the backends.