doc: Added doc for ingress/GwAPI host network mode

* Added documentation about how to expose the Cilium Ingress Controller / GwAPI Envoy listener on the host network * Fixed faulty Helm flag in Helm values comment * Unified GwAPI `l7Proxy=true` requirement to work everywhere with Helm flags, not agent flags. * Fixed RST title level inconsistency in the troubleshooting guide Signed-off-by: Philip Schmid <philip.schmid@isovalent.com>
cilium · May 14, 2024 · 1f53d94 · 1f53d94
1 parent cc1759e
commit 1f53d94
Show file tree

Hide file tree

Showing 10 changed files with 267 additions and 29 deletions.
diff --git a/Documentation/helm-values.rst b/Documentation/helm-values.rst
diff --git a/Documentation/network/servicemesh/gateway-api/gateway-api.rst b/Documentation/network/servicemesh/gateway-api/gateway-api.rst
@@ -35,6 +35,9 @@ tests are passed.
 
 .. include:: installation.rst
 
+.. _gs_gateway_host_network_mode:
+.. include:: host-network-mode.rst
+
 Examples
 ########
 

diff --git a/Documentation/network/servicemesh/gateway-api/host-network-mode.rst b/Documentation/network/servicemesh/gateway-api/host-network-mode.rst
@@ -0,0 +1,114 @@
+.. only:: not (epub or latex or html)
+
+    WARNING: You are looking at unreleased Cilium documentation.
+    Please use the official rendered version released here:
+    https://docs.cilium.io
+
+Host network mode
+#################
+
+.. note::
+    Supported since Cilium 1.16+
+
+Host network mode allows you to expose the Cilium Gateway API Gateway directly
+on the host network.
+This is useful in cases where a LoadBalancer Service is unavailable, such
+as in development environments or environments with cluster-external
+loadbalancers.
+
+.. note::
+    * Enabling the Cilium Gateway API host network mode automatically disables the LoadBalancer type Service mode. They are mutually exclusive.
+    * The listener is exposed on all interfaces (``0.0.0.0`` for IPv4 and/or ``::`` for IPv6).
+
+Host network mode can be enabled via Helm:
+
+.. code-block:: yaml
+
+    gatewayAPI:
+      enabled: true
+      hostNetwork:
+        enabled: true
+
+Once enabled, the host network port for a ``Gateway`` can be specified via
+``spec.listeners.port``. The port must be unique per ``Gateway``
+resource and you should choose a port number higher than ``1023`` (see
+`Bind to privileged port`_).
+
+.. warning::
+    Be aware that misconfiguration might result in port clashes. Configure unique ports that are still available on all Cilium Nodes where Gateway API listeners are exposed.
+
+Bind to privileged port
+=======================
+
+By default, the Cilium L7 Envoy process does not have any Linux capabilities
+out-of-the-box and is therefore not allowed to listen on privileged ports.
+
+If you choose a port equal to or lower than ``1023``, ensure that the Helm value
+``envoy.securityContext.capabilities.keepCapNetBindService=true`` is configured
+and to add the capability ``NET_BIND_SERVICE`` to the respective
+:ref:`Cilium Envoy container via Helm values<envoy>`:
+
+* Standalone DaemonSet mode: ``envoy.securityContext.capabilities.envoy``
+* Embedded mode: ``securityContext.capabilities.ciliumAgent``
+
+Configure the following Helm values to allow privileged port bindings in host
+network mode:
+
+.. tabs::
+
+    .. group-tab:: Standalone DaemonSet mode
+
+      .. code-block:: yaml
+
+          gatewayAPI:
+            enabled: true
+            hostNetwork:
+              enabled: true
+          envoy:
+            enabled: true
+            securityContext:
+              capabilities:
+                keepCapNetBindService: true
+                envoy:
+                # Add NET_BIND_SERVICE to the list (keep the others!)
+                - NET_BIND_SERVICE
+
+    .. group-tab:: Embedded mode
+
+      .. code-block:: yaml
+
+          gatewayAPI:
+            enabled: true
+            hostNetwork:
+              enabled: true
+          envoy:
+            securityContext:
+              capabilities:
+                keepCapNetBindService: true
+          securityContext:
+            capabilities:
+              ciliumAgent:
+              # Add NET_BIND_SERVICE to the list (keep the others!)
+              - NET_BIND_SERVICE
+
+Deploy Gateway API listeners on subset of nodes
+===============================================
+
+The Cilium Gateway API Envoy listener can be exposed on a specific subset of
+nodes. This only works in combination with the host network mode and can be
+configured via a node label selector in the Helm values:
+
+.. code-block:: yaml
+
+    gatewayAPI:
+      enabled: true
+      hostNetwork:
+        enabled: true
+        nodes:
+          matchLabels:
+            role: infra
+            component: gateway-api
+
+This will deploy the Gateway API Envoy listener only on the Cilium Nodes
+matching the configured labels. An empty selector selects all nodes and
+continues to expose the functionality on all Cilium nodes.
diff --git a/Documentation/network/servicemesh/gateway-api/installation.rst b/Documentation/network/servicemesh/gateway-api/installation.rst
@@ -5,7 +5,8 @@ Prerequisites
   ``nodePort.enabled=true`` or by enabling the kube-proxy replacement with
   ``kubeProxyReplacement=true``. For more information, see :ref:`kube-proxy
   replacement <kubeproxy-free>`.
-* Cilium must be configured with the L7 proxy enabled using the ``--enable-l7-proxy`` flag (enabled by default).
+* Cilium must be configured with the L7 proxy enabled using ``l7Proxy=true``
+  (enabled by default).
 * The below CRDs from Gateway API v1.0.0 ``must`` be pre-installed.
   Please refer to this `docs <https://gateway-api.sigs.k8s.io/guides/?h=crds#getting-started-with-gateway-api>`_
   for installation steps. Alternatively, the below snippet could be used.
@@ -25,8 +26,9 @@ Prerequisites
         $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml
         $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.0.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml
 
-* Similar to Ingress, Gateway API controller creates a service of LoadBalancer type,
-  so your environment will need to support this.
+* By default, the Gateway API controller creates a service of LoadBalancer type,
+  so your environment will need to support this. Alternatively, since Cilium 1.16+,
+  you can directly expose the Cilium L7 proxy on the :ref:`host network <gs_gateway_host_network_mode>`.
 
 Installation
 ############

diff --git a/Documentation/network/servicemesh/gateway-api/troubleshooting.rst b/Documentation/network/servicemesh/gateway-api/troubleshooting.rst
@@ -3,7 +3,7 @@ This page guides you through the different mechanics of Gateway API and how to t
 Be sure to follow the Generic and Setup Verification steps from the :ref:`Troubleshooting Ingress & Service Mesh page<troubleshooting_servicemesh>`.
 
 Checking resources
-----------------------
+==================
 
 #. Check the Gateway resource 
 
@@ -142,7 +142,7 @@ Checking resources
     customresourcedefinitions.apiextensions.k8s.io \"tlsroutes.gateway.networking.k8s.io\" not found" subsys=gateway-api
 
 Common mistakes
----------------
+===============
 
 .. include:: mistakes-warning.rst
 
@@ -184,7 +184,7 @@ Common mistakes
         Type:                  Accepted
 
 Underlying mechanics: a high level overview
--------------------------------------------
+===========================================
 
 A Cilium deployment has two parts that handle Gateway API resources: the Cilium agent and the Cilium operator.
 

diff --git a/Documentation/network/servicemesh/ingress.rst b/Documentation/network/servicemesh/ingress.rst
@@ -18,7 +18,7 @@ is also supported.
 
 .. Note::
 
-    The ingress controller creates a service of LoadBalancer type, so
+    The ingress controller creates a Service of LoadBalancer type, so
     your environment will need to support this.
 
 Cilium allows you to specify load balancer mode for the Ingress resource:
@@ -54,6 +54,10 @@ Prerequisites
   replacement <kubeproxy-free>`.
 * Cilium must be configured with the L7 proxy enabled using ``l7Proxy=true``
   (enabled by default).
+* By default, the Ingress controller creates a Service of LoadBalancer type,
+  so your environment will need to support this. Alternatively, you can change
+  this to NodePort or, since Cilium 1.16+, directly expose the Cilium L7 proxy
+  on the :ref:`host network<gs_ingress_host_network_mode>`.
 
 .. include:: installation.rst
 
@@ -82,24 +86,22 @@ Supported Ingress Annotations
      - ``LoadBalancer``
    * - ``ingress.cilium.io/insecure-node-port``
      - | The NodePort to use for the HTTP Ingress.
-       | Applicable only if ``ingress.cilium.io/service-type`` is ``NodePort``. If unspecified, a random
+       | Applicable only if ``ingress.cilium.io/service-type``
+       | is ``NodePort``. If unspecified, a random
        | NodePort will be allocated by kubernetes.
      - unspecified
    * - ``ingress.cilium.io/secure-node-port``
      - | The NodePort to use for the HTTPS Ingress.
-       | Applicable only if ``ingress.cilium.io/service-type`` is ``NodePort``. If unspecified, a random
+       | Applicable only if ``ingress.cilium.io/service-type``
+       | is ``NodePort``. If unspecified, a random
        | NodePort will be allocated by kubernetes.
      - unspecified
-   * - ``ingress.cilium.io/http-host-port``
-     - | The port to use for the HTTP listener (HTTP and HTTPS) on the host network.
-       | Applicable only for dedicated Ingress and if hostnetwork mode is enabled for IngressController.
-       | If unspecified, the default ports (80/443) are used.
-     - unspecified
-   * - ``ingress.cilium.io/tls-passthrough-host-port``
-     - | The port to use for the TLS passthrough listener on the host network.
-       | Applicable only for dedicated Ingress and if hostnetwork mode is enabled for IngressController.
-       | If unspecified, the default port (443) are used.
-     - unspecified
+   * - ``ingress.cilium.io/host-listener-port``
+     - | The port to use for the Envoy listener on the host
+       | network. Applicable and mandatory only for
+       | dedicated Ingress and if :ref:`host network mode<gs_ingress_host_network_mode>` is
+       | enabled.
+     - ``8080``
    * - ``ingress.cilium.io/tls-passthrough``
      - | Enable TLS Passthrough mode for this Ingress.
        | Applicable values are ``enabled`` and ``disabled``,
@@ -127,14 +129,17 @@ Supported Ingress Annotations
        | accepted.
        |
        | Note that if the annotation is not present, this
-       | behavior will be controlled by the ``enforce-ingress-https`` configuration
-       | file setting (or ``ingressController.enforceHttps`` in Helm).
+       | behavior will be controlled by the
+       | ``enforce-ingress-https`` configuration
+       | file setting (or ``ingressController.enforceHttps``
+       | in Helm).
        | 
-       | Any host with TLS config will have redirects to HTTPS
-       | configured for each match specified in the Ingress.
+       | Any host with TLS config will have redirects to
+       | HTTPS configured for each match specified in the
+       | Ingress.
      - unspecified
 
-Additionally, cloud-provider specific annotations for the LoadBalancer service
+Additionally, cloud-provider specific annotations for the LoadBalancer Service
 are supported.
 
 By default, annotations with values beginning with:
@@ -145,7 +150,7 @@ By default, annotations with values beginning with:
 * ``service.kubernetes.io``
 * ``cloud.google.com``
 
-will be copied from an Ingress object to the generated LoadBalancer service objects.
+will be copied from an Ingress object to the generated LoadBalancer Service objects.
 
 This setting is controlled by the Cilium Operator's ``ingress-lb-annotation-prefixes``
 config flag, and can be configured in Cilium's Helm ``values.yaml``
@@ -154,6 +159,119 @@ using the ``ingressController.ingressLBAnnotationPrefixes`` setting.
 Please refer to the `Kubernetes documentation <https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer>`_
 for more details.
 
+.. _gs_ingress_host_network_mode:
+
+Host network mode
+#################
+.. note::
+  Supported since Cilium 1.16+
+
+Host network mode allows you to expose the Cilium ingress controller (Envoy
+listener) directly on the host network.
+This is useful in cases where a LoadBalancer Service is unavailable, such
+as in development environments or environments with cluster-external
+loadbalancers.
+
+.. note::
+    * Enabling the Cilium ingress controller host network mode automatically disables the LoadBalancer/NodePort type Service mode. They are mutually exclusive.
+    * The listener is exposed on all interfaces (``0.0.0.0`` for IPv4 and/or ``::`` for IPv6).
+
+Host network mode can be enabled via Helm:
+
+.. code-block:: yaml
+
+    ingressController:
+      enabled: true
+      hostNetwork:
+        enabled: true
+
+Once enabled, host network ports can be specified with the following methods:
+
+* Shared Ingress: Globally via Helm flags
+    * ``ingressController.hostNetwork.sharedListenerPort``: Host network port to expose the Cilium ingress controller Envoy listener. The default port is ``8080``. If you change it, you should choose a port number higher than ``1023`` (see `Bind to privileged port`_).
+* Dedicated Ingress: Per ``Ingress`` resource via annotations
+    * ``ingress.cilium.io/host-listener-port``:  Host network port to expose the Cilium ingress controller Envoy listener. The default port is ``8080`` but it can only be used for a single ``Ingress`` resource as it needs to be unique per ``Ingress`` resource. You should choose a port higher than ``1023`` (see `Bind to privileged port`_). This annotation is mandatory if the global Cilium ingress controller mode is configured to ``dedicated`` (``ingressController.loadbalancerMode``) or the ingress resource sets the ``ingress.cilium.io/loadbalancer-mode`` annotation to ``dedicated`` and multiple ``Ingress`` resources are deployed.
+
+The default behavior regarding shared or dedicated ingress can be configured via
+``ingressController.loadbalancerMode``.
+
+.. warning::
+    Be aware that misconfiguration might result in port clashes. Configure unique ports that are still available on all Cilium Nodes where Cilium ingress controller Envoy listeners are exposed.
+
+Bind to privileged port
+***********************
+By default, the Cilium L7 Envoy process does not have any Linux capabilities
+out-of-the-box and is therefore not allowed to listen on privileged ports.
+
+If you choose a port equal to or lower than ``1023``, ensure that the Helm value
+``envoy.securityContext.capabilities.keepCapNetBindService=true`` is configured
+and to add the capability ``NET_BIND_SERVICE`` to the respective
+:ref:`Cilium Envoy container via Helm values<envoy>`:
+
+* Standalone DaemonSet mode: ``envoy.securityContext.capabilities.envoy``
+* Embedded mode: ``securityContext.capabilities.ciliumAgent``
+
+Configure the following Helm values to allow privileged port bindings in host
+network mode:
+
+.. tabs::
+
+    .. group-tab:: Standalone DaemonSet mode
+
+      .. code-block:: yaml
+
+          ingressController:
+            enabled: true
+            hostNetwork:
+              enabled: true
+          envoy:
+            enabled: true
+            securityContext:
+              capabilities:
+                keepCapNetBindService: true
+                envoy:
+                # Add NET_BIND_SERVICE to the list (keep the others!)
+                - NET_BIND_SERVICE
+
+    .. group-tab:: Embedded mode
+
+      .. code-block:: yaml
+
+          ingressController:
+            enabled: true
+            hostNetwork:
+              enabled: true
+          envoy:
+            securityContext:
+              capabilities:
+                keepCapNetBindService: true
+          securityContext:
+            capabilities:
+              ciliumAgent:
+              # Add NET_BIND_SERVICE to the list (keep the others!)
+              - NET_BIND_SERVICE
+
+Deploy Gateway API listeners on subset of nodes
+***********************************************
+The Cilium ingress controller Envoy listener can be exposed on a specific subset
+of nodes. This only works in combination with the host network mode and can be
+configured via a node label selector in the Helm values:
+
+.. code-block:: yaml
+
+    ingressController:
+      enabled: true
+      hostNetwork:
+        enabled: true
+        nodes:
+          matchLabels:
+            role: infra
+            component: ingress
+
+This will deploy the Ingress Controller Envoy listener only on the Cilium Nodes
+matching the configured labels. An empty selector selects all nodes and
+continues to expose the functionality on all Cilium nodes.
+
 Examples
 ########
 

diff --git a/Documentation/spelling_wordlist.txt b/Documentation/spelling_wordlist.txt
@@ -507,6 +507,7 @@ memcd
 microservice
 microservices
 minikube
+misconfiguration
 misconfigures
 mitigations
 mlx