Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 125 additions & 0 deletions content/ngf/how-to/data-plane-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,3 +375,128 @@ To view the full list of configuration options, see the `NginxProxy spec` in the

---

### Patch data plane Service, Deployment, and DaemonSet

NGINX Gateway Fabric supports advanced customization of the data plane Service, Deployment, and DaemonSet objects using patches in the `NginxProxy` resource. This allows you to apply Kubernetes-style patches to these resources, enabling custom labels, annotations, or other modifications that are not directly exposed via the NginxProxy spec.

#### Supported Patch Types

You can specify one or more patches for each of the following resources:

- `spec.kubernetes.service.patches`
- `spec.kubernetes.deployment.patches`
- `spec.kubernetes.daemonSet.patches`

Each patch has two fields:

- `type`: The patch type. Supported values are:
- `StrategicMerge` (default): Strategic merge patch (Kubernetes default for most resources)
- `Merge`: JSON merge patch (RFC 7386)
- `JSONPatch`: JSON patch (RFC 6902)
- `value`: The patch data. For `StrategicMerge` and `Merge`, this should be a JSON object. For `JSONPatch`, this should be a JSON array of patch operations.

Patches are applied in the order they appear in the array. Later patches can override fields set by earlier patches.

#### Example: Configure Service with session affinity

```yaml
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
name: ngf-proxy-patch-service
spec:
kubernetes:
service:
patches:
- type: StrategicMerge
value:
spec:
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 300
```

#### Example: Configure Deployment with custom strategy

```yaml
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
name: ngf-proxy-patch-deployment
spec:
kubernetes:
deployment:
patches:
- type: Merge
value:
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
maxSurge: 2
```

#### Example: Use JSONPatch to configure DaemonSet host networking and priority

```yaml
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
name: ngf-proxy-patch-daemonset
spec:
kubernetes:
daemonSet:
patches:
- type: JSONPatch
value:
- op: add
path: /spec/template/spec/hostNetwork
value: true
- op: add
path: /spec/template/spec/dnsPolicy
value: "ClusterFirstWithHostNet"
- op: add
path: /spec/template/spec/priorityClassName
value: "system-node-critical"
```

#### Example: Multiple patches, later patch overrides earlier

```yaml
apiVersion: gateway.nginx.org/v1alpha2
kind: NginxProxy
metadata:
name: ngf-proxy-multi-patch
spec:
kubernetes:
service:
patches:
- type: StrategicMerge
value:
spec:
sessionAffinity: ClientIP
publishNotReadyAddresses: false
- type: StrategicMerge
value:
spec:
sessionAffinity: None
publishNotReadyAddresses: true
```

In this example, the final Service will have `sessionAffinity: None` and `publishNotReadyAddresses: true` because the second patch overrides the values from the first patch.

{{< note >}}
**Which patch type should I use?**

- **StrategicMerge** is the default and most user-friendly for Kubernetes-native resources like Deployments and Services. It understands lists and merges fields intelligently (e.g., merging containers by name). Use this for most use cases.
- **Merge** (JSON Merge Patch) is simpler and works well for basic object merges, but does not handle lists or complex merging. Use this if you want to replace entire fields or for non-Kubernetes-native resources.
- **JSONPatch** is the most powerful and flexible, allowing you to add, remove, or replace specific fields using RFC 6902 operations. Use this for advanced or fine-grained changes, but it is more verbose and error-prone.

If unsure, start with StrategicMerge. Use JSONPatch only if you need to surgically modify fields that cannot be addressed by the other patch types.

Patches are applied after all other NginxProxy configuration is rendered. Invalid patches will result in a validation error and will not be applied.
{{< /note >}}

---
60 changes: 45 additions & 15 deletions content/ngf/how-to/scaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,36 +16,64 @@ It provides guidance on how to scale each plane effectively, and when you should

The data plane is the NGINX deployment that handles user traffic to backend applications. Every Gateway object created provisions its own NGINX deployment and configuration.

You have two options for scaling the data plane:
You have multiple options for scaling the data plane:

- Increasing the number of [worker connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections) for an existing deployment
- Increasing the number of replicas for an existing deployment
- Creating a new Gateway for a new data plane

#### When to increase replicas or create a new Gateway
#### When to increase worker connections, replicas, or create a new Gateway

Understanding when to increase replicas or create a new Gateway is key to managing traffic effectively.
Understanding when to increase worker connections, replicas, or create a new Gateway is key to managing traffic effectively.

Increasing data plane replicas is ideal when you need to handle more traffic without changing the configuration.
Increasing worker connections or replicas is ideal when you need to handle more traffic without changing the overall routing configuration. Setting the worker connections field allows a single NGINX data plane instance to handle more connections without needing to scale the replicas. However, scaling the replicas can be beneficial to reduce single points of failure.

For example, if you're routing traffic to `api.example.com` and notice an increase in load, you can scale the replicas from 1 to 5 to better distribute the traffic and reduce latency.
Scaling replicas can be done manually or automatically using a [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) (HPA).

All replicas will share the same configuration from the Gateway used to set up the data plane, simplifying configuration management.
To update worker connections (default: 1024), replicas, or enable autoscaling, you can edit the `NginxProxy` resource:

There are two ways to modify the number of replicas for an NGINX deployment:
```shell
kubectl edit nginxproxies.gateway.nginx.org ngf-proxy-config -n nginx-gateway
```

First, at the time of installation you can modify the field `nginx.replicas` in the `values.yaml` or add the `--set nginx.replicas=` flag to the `helm install` command:
{{< call-out "note" >}}

```shell
helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway --set nginx.replicas=5
The NginxProxy resource in this example lives in the control plane namespace (default: `nginx-gateway`) and applies to the GatewayClass, but you can also define one per Gateway. See the [Data plane configuration]({{< ref "/ngf/how-to/data-plane-configuration.md" >}}) document for more information.

{{< /call-out >}}

- Worker connections is set using the `workerConnections` field:

```yaml
spec:
workerConnections: 4096
```

Secondly, you can update the `NginxProxy` resource while NGINX is running to modify the `kubernetes.deployment.replicas` field and scale the data plane deployment dynamically:
- Replicas are set using the `kubernetes.deployment.replicas` field:

```shell
kubectl edit nginxproxies.gateway.nginx.org ngf-proxy-config -n nginx-gateway
```yaml
spec:
kubernetes:
deployment:
replicas: 3
```

- Autoscaling can be enabled using the `kubernetes.deployment.autoscaling` field. The default `replicas` value will be used until the Horizontal Pod Autoscaler is running.

```yaml
spec:
kubernetes:
deployment:
autoscaling:
enable: true
maxReplicas: 10
```

The alternate way to scale the data plane is by creating a new Gateway. This is beneficial when you need distinct configurations, isolation, or separate policies.
See the `NginxProxy` section of the [API reference]({{< ref "/ngf/reference/api.md" >}}) for the full specification.

All of these fields are also available at installation time by setting them in the [helm values](https://github.com/nginx/nginx-gateway-fabric/blob/main/charts/nginx-gateway-fabric/values.yaml).

An alternate way to scale the data plane is by creating a new Gateway. This is beneficial when you need distinct configurations, isolation, or separate policies.

For example, if you're routing traffic to a new domain `admin.example.com` and require a different TLS certificate, stricter rate limits, or separate authentication policies, creating a new Gateway could be a good approach.

Expand All @@ -60,7 +88,9 @@ Scaling the control plane can be beneficial in the following scenarios:
1. _Higher availability_ - When a control plane pod crashes, runs out of memory, or goes down during an upgrade, it can interrupt configuration delivery. By scaling to multiple replicas, another pod can quickly step in and take over, keeping things running smoothly with minimal downtime.
1. _Faster configuration distribution_ - As the number of connected NGINX instances grows, a single control plane pod may become a bottleneck in handling connections or streaming configuration updates. Scaling the control plane improves concurrency and responsiveness when delivering configuration over gRPC.

To scale the control plane, use the `kubectl scale` command on the control plane deployment to increase or decrease the number of replicas. For example, the following command scales the control plane deployment to 3 replicas:
To automatically scale the control plane, you can create a [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) (HPA) in the control plane namespace (default: `nginx-gateway`). At installation time, the [NGINX Gateway Fabric helm chart](https://github.com/nginx/nginx-gateway-fabric/blob/main/charts/nginx-gateway-fabric/values.yaml) allows you to set the HPA configuration in the `nginxGateway.autoscaling` section, which will provision an HPA for you. If NGINX Gateway Fabric is already running, then you can manually define the HPA and deploy it.

To manually scale the control plane, use the `kubectl scale` command on the control plane deployment to increase or decrease the number of replicas. For example, the following command scales the control plane deployment to 3 replicas:

```shell
kubectl scale deployment -n nginx-gateway ngf-nginx-gateway-fabric --replicas 3
Expand Down
79 changes: 24 additions & 55 deletions content/ngf/install/upgrade-version.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,13 @@ It covers the necessary steps for minor versions as well as major versions (such

Many of the nuances in upgrade paths relate to how custom resource definitions (CRDs) are managed.

{{< call-out "tip" >}}

To avoid interruptions, review the [Delay pod termination for zero downtime upgrades](#configure-delayed-pod-termination-for-zero-downtime-upgrades) section.
## Minor NGINX Gateway Fabric upgrades

{{< call-out "important" >}}
Upgrading from v2.0.x to v2.1 requires the NGINX Gateway Fabric control plane to be uninstalled and then reinstalled to avoid any downtime to user traffic. CRDs do not need to be removed. The NGINX data plane deployment is not affected by this process, and traffic should still flow uninterrupted. The steps are described below.
{{< /call-out >}}


## Minor NGINX Gateway Fabric upgrades

{{< call-out "important" >}} NGINX Plus users need a JWT secret before upgrading from version 1.4.0 to 1.5.x.

Follow the steps in [Set up the JWT]({{< ref "/ngf/install/nginx-plus.md#set-up-the-jwt" >}}) to create the Secret.
Expand Down Expand Up @@ -72,14 +70,23 @@ Warning: kubectl apply should be used on resource created by either kubectl crea

{{% tab name="Helm" %}}

{{< call-out "important" >}} If you are using NGINX Plus and have a different Secret name than the default `nplus-license` name, specify the Secret name by setting `--set nginx.usage.secretName=<secret-name>` when running `helm upgrade`. {{< /call-out >}}
{{< call-out "important" >}} If you are using NGINX Plus and have a different Secret name than the default `nplus-license` name, specify the Secret name by setting `--set nginx.usage.secretName=<secret-name>` when running `helm install` or `helm upgrade`. {{< /call-out >}}

To upgrade the release with Helm, you can use the OCI registry, or download the chart and upgrade from the source.

If needed, replace `ngf` with your chosen release name.

**Upgrade from the OCI registry**

To avoid downtime when upgrading from v2.0.x to v2.1, run the following commands. Be sure to include your previous installation flags and values if necessary. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process.

```shell
helm uninstall ngf -n nginx-gateway
helm install ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric -n nginx-gateway
```

Otherwise, for all other version upgrades:

```shell
helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric -n nginx-gateway
```
Expand All @@ -88,7 +95,14 @@ helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric -n nginx-gatewa

{{< include "/ngf/installation/helm/pulling-the-chart.md" >}}

To upgrade, run the following command:
To avoid downtime when upgrading from v2.0.x to v2.1, run the following. Be sure to include your previous installation flags and values if necessary. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process.

```shell
helm uninstall ngf -n nginx-gateway
helm install ngf . -n nginx-gateway
```

Otherwise, for all other version upgrades:

```shell
helm upgrade ngf . -n nginx-gateway
Expand All @@ -98,7 +112,9 @@ helm upgrade ngf . -n nginx-gateway

{{% tab name="Manifests" %}}

Select the deployment manifest that matches your current deployment from options available in the [Deploy NGINX Gateway Fabric]({{< ref "/ngf/install/manifests.md#deploy-nginx-gateway-fabric-1">}}) section and apply it.
Select the deployment manifest that matches your current deployment from options available in the [Deploy NGINX Gateway Fabric]({{< ref "/ngf/install/manifests.md#deploy-nginx-gateway-fabric-1">}}) section.

To avoid downtime when upgrading from v2.0.x to v2.1, delete the previous NGINX Gateway Fabric control plane deployment in the `nginx-gateway` namespace, using `kubectl delete deployment`. Then `kubectl apply` the updated manifest file. This will not affect user traffic, as the NGINX data plane deployment won't be removed as part of this process.

{{% /tab %}}

Expand Down Expand Up @@ -259,50 +275,3 @@ To upgrade from NGINX Open Source to NGINX Plus, update the Helm command to incl
```shell
helm upgrade ngf oci://ghcr.io/nginx/charts/nginx-gateway-fabric --set nginx.image.repository=private-registry.nginx.com/nginx-gateway-fabric/nginx-plus --set nginx.plus=true --set nginx.imagePullSecret=nginx-plus-registry-secret -n nginx-gateway
```

## Delay pod termination for zero downtime upgrades {#configure-delayed-pod-termination-for-zero-downtime-upgrades}

{{< include "/ngf/installation/delay-pod-termination/delay-pod-termination-overview.md" >}}

Follow these steps to configure delayed pod termination:

1. Open the `values.yaml` for editing.

1. **Add delayed shutdown hooks**:

- In the `values.yaml` file, add `lifecycle: preStop` hooks to both the `nginx` and `nginx-gateway` container definitions. These hooks instruct the containers to delay their shutdown process, allowing time for connections to close gracefully. Update the `sleep` value to what works for your environment.

```yaml
nginxGateway:
<...>
lifecycle:
preStop:
exec:
command:
- /usr/bin/gateway
- sleep
- --duration=40s # This flag is optional, the default is 30s

nginx:
<...>
lifecycle:
preStop:
exec:
command:
- /bin/sleep
- "40"
```

1. **Set the termination grace period**:

- {{< include "/ngf/installation/delay-pod-termination/termination-grace-period.md">}}

1. Save the changes.

{{< call-out "note" >}}
For additional information on configuring and understanding the behavior of containers and pods during their lifecycle, refer to the following Kubernetes documentation:

- [Container Lifecycle Hooks](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks)
- [Pod Lifecycle](https://kubernetes.io/docs/concepts/workloads/Pods/Pod-lifecycle/#Pod-termination)

{{< /call-out>}}
5 changes: 4 additions & 1 deletion content/ngf/overview/gateway-api-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,11 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command
- `ResolvedRefs/True/ResolvedRefs`
- `ResolvedRefs/False/InvalidCertificateRef`
- `ResolvedRefs/False/InvalidRouteKinds`
- `ResolvedRefs/False/RefNotPermitted`
- `Conflicted/True/ProtocolConflict`
- `Conflicted/True/HostnameConflict`
- `Conflicted/False/NoConflicts`
- `OverlappingTLSConfig/True/OverlappingHostnames`

### HTTPRoute

Expand Down Expand Up @@ -167,7 +169,7 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command
- `requestHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest.
- `urlRewrite`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest. Incompatible with `requestRedirect`.
- `responseHeaderModifier`: Supported. If multiple filters are configured, NGINX Gateway Fabric will choose the first and ignore the rest.
- `requestMirror`: Supported. Multiple mirrors can be specified.
- `requestMirror`: Supported. Multiple mirrors can be specified. Percent and fraction-based mirroring are supported.
- `extensionRef`: Supported for SnippetsFilters.
- `backendRefs`: Partially supported. Backend ref `filters` are not supported.
- `status`
Expand All @@ -189,6 +191,7 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command
- `ResolvedRefs/False/BackendNotFound`
- `ResolvedRefs/False/UnsupportedValue`: Custom reason for when one of the HTTPRoute rules has a backendRef with an unsupported value.
- `ResolvedRefs/False/InvalidIPFamily`: Custom reason for when one of the HTTPRoute rules has a backendRef that has an invalid IPFamily.
- `ResolvedRefs/False/UnsupportedProtocol`
- `PartiallyInvalid/True/UnsupportedValue`

### GRPCRoute
Expand Down
Loading
Loading