Skip to content

feat: Add OpenTelemetry documentation #824

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ If you encounter the error `error [emerg] 13#13: "zone_sync" directive is duplic
|*otel-exporter-header-value* | The value of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-name` required. | N/A | *"custom-value"* |
|*otel-service-name* | Sets the `service.name` attribute of the OTel resource. `otel-exporter-endpoint` required. | N/A | *"nginx-ingress-controller:nginx"* |
| *otel-trace-in-http* | Enables [OpenTelemetry](https://opentelemetry.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Set this to *"false"* to enable OpenTelemetry for individual routes with snippets. `otel-exporter-endpoint` required. | *"false"* | *"true"* |
|*opentracing* | Removed in v5.0.0. Enables [OpenTracing](https://opentracing.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Note: requires the Ingress Controller image with OpenTracing module and a tracer. See the [docs]({{< ref "/nic/installation/integrations/opentracing.md" >}}) for more information. | *False* | |
|*opentracing* | Removed in v5.0.0. Enables [OpenTracing](https://opentracing.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Note: requires the Ingress Controller image with OpenTracing module and a tracer. See the [docs]({{< ref "/nic/logging-and-monitoring/opentracing.md" >}}) for more information. | *False* | |
|*opentracing-tracer* | Removed in v5.0.0. Sets the path to the vendor tracer binary plugin. | N/A | |
|*opentracing-tracer-config* | Removed in v5.0.0. Sets the tracer configuration in JSON format. | N/A | |
|*app-protect-compressed-requests-action* | Sets the *app_protect_compressed_requests_action* [global directive](/nginx-app-protect/configuration/#global-directives). | *drop* | |
Expand Down
5 changes: 1 addition & 4 deletions content/nic/logging-and-monitoring/_index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
---
title: Logging And Monitoring
title: Logging and monitoring
description:
weight: 1500
url: /nginx-ingress-controller/logging-and-monitoring
menu:
docs:
parent: NGINX Ingress Controller
---
24 changes: 16 additions & 8 deletions content/nic/logging-and-monitoring/logging.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,37 @@
---
title: Logging
title: Logs available from NGINX Ingress Controller
toc: true
weight: 1800
weight: 100
nd-content-type: reference
nd-product: NIC
nd-docs: DOCS-613
---

This document gives an overview of logging provided by NGINX Ingress Controller.
This document gives an overview of logging provided by F5 NGINX Ingress Controller.

NGINX Ingress Controller exposes the logs of the Ingress Controller process (the process that generates NGINX configuration and reloads NGINX to apply it) and NGINX access and error logs. All logs are sent to the standard output and error of the NGINX Ingress Controller process. To view the logs, you can execute the `kubectl logs` command for an Ingress Controller pod. For example:
NGINX Ingress Controller exposes the logs of the Ingress Controller process (The process that generates NGINX configuration and reloads NGINX to apply it) and NGINX access and error logs.

All logs are sent to the standard output and error of the NGINX Ingress Controller process. To view the logs, you can execute the `kubectl logs` command for an Ingress Controller pod.

For example:

```shell
kubectl logs <nginx-ingress-pod> -n nginx-ingress
```

## NGINX Ingress Controller Process Logs

The NGINX Ingress Controller process logs are configured through the `-log-level` command-line argument of the NGINX Ingress Controller, which sets the log level. The default value is `info`. Other options include: `trace`, `debug`, `info`, `warning`, `error` and `fatal`. The value `debug` is useful for troubleshooting: you will be able to see how NGINX Ingress Controller gets updates from the Kubernetes API, generates NGINX configuration and reloads NGINX.
The NGINX Ingress Controller process logs are configured through the `-log-level` command-line argument of the NGINX Ingress Controller, which sets the log level.

The default value is `info`. Other options include: `trace`, `debug`, `info`, `warning`, `error` and `fatal`.

The value `debug` is useful for troubleshooting: you will be able to see how NGINX Ingress Controller gets updates from the Kubernetes API, generates NGINX configuration and reloads NGINX.

See also the doc about NGINX Ingress Controller [command-line arguments]({{< ref "/nic/configuration/global-configuration/command-line-arguments.md" >}}).
Read more about NGINX Ingress Controller [command-line arguments]({{< ref "/nic/configuration/global-configuration/command-line-arguments.md" >}}).

## NGINX Logs

The NGINX includes two logs:
NGINX includes two logs:

- *Access log*, where NGINX writes information about client requests in the access log right after the request is processed. The access log is configured via the [logging-related]({{< ref "/nic/configuration/global-configuration/configmap-resource.md#logging" >}}) ConfigMap keys:
- `log-format` for HTTP and HTTPS traffic.
Expand All @@ -32,4 +40,4 @@ The NGINX includes two logs:
Additionally, you can disable access logging with the `access-log-off` ConfigMap key.
- *Error log*, where NGINX writes information about encountered issues of different severity levels. It is configured via the `error-log-level` [ConfigMap key]({{< ref "/nic/configuration/global-configuration.md#configmap-resource#logging" >}}). To enable debug logging, set the level to `debug` and also set the `-nginx-debug` [command-line argument]({{< ref "/nic/configuration/global-configuration.md#command-line-arguments" >}}), so that NGINX is started with the debug binary `nginx-debug`.

See also the doc about [NGINX logs]({{< ref "/nginx/admin-guide/monitoring/logging.md" >}}) from NGINX Admin guide.
Read more about [NGINX logs]({{< ref "/nginx/admin-guide/monitoring/logging.md" >}}) from NGINX Admin guide.
105 changes: 105 additions & 0 deletions content/nic/logging-and-monitoring/opentelemetry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
# We use sentence case and present imperative tone
title: "Enable OpenTelemetry"
# Weights are assigned in increments of 100: determines sorting order
weight: 300
# Creates a table of contents and sidebar, useful for large documents
toc: true
# Types have a 1:1 relationship with Hugo archetypes, so you shouldn't need to change this
nd-content-type: how-to
# Intended for internal catalogue and search, case sensitive:
# Agent, N4Azure, NIC, NIM, NGF, NAP-DOS, NAP-WAF, NGINX One, NGINX+, Solutions, Unit
nd-product: NIC
---

This topic describes how to enable [OpenTelemetry](https://opentelemetry.io/) for F5 NGINX Ingress Controller using the [native NGINX module](https://nginx.org/en/docs/ngx_otel_module.html).

## Before you begin

To complete this guide, you need the following pre-requisites:

- An [NGINX Ingress Controller installation]({{< ref "/nic/installation/" >}}) with OpenTelemetry (v5.1.0+)

## Load the OpenTelemetry module

To enable OpenTelemetry, you must first load the module by adding the [_otel-exporter-endpoint_ ConfigMap key]({{< ref "/nic/configuration/global-configuration/configmap-resource.md#modules" >}}), which takes an endpoint argument.

The following is an example of a OpenTelemetry collector running in your cluster as the target for exporting data:

```yaml
otel-exporter-endpoint: "http://otel-collector.default.svc.cluster.local:4317"
```

A complete ConfigMap example with all OpenTelemetry options could look as follows:

{{< ghcode "https://raw.githubusercontent.com/nginx/kubernetes-ingress/refs/heads/main/examples/shared-examples/otel/nginx-config.yaml" >}}

## Enable OpenTelemetry

Once you have loaded the module, you can now enable OpenTelemetry.

You can configure it globally for all resources, or on a per resource basis.

### Global

To enable OpenTelemetry for all resources, set the _otel-trace-in-http_ ConfigMap key to `true`:

```yaml
otel-trace-in-http: "true"
```

### Per resource

You can configure OpenTelemetry on a per resource basis in NGINX Ingress Controller.

For this functionality, you must [enable snippets]({{< ref "/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md" >}}) with the `-enable-snippets` command-line argument.

Based on the state of global configuration, you can selectively enable or disable metrics for each resource.

#### Enable a specific resource or path

With OpenTelemetry **disabled** globally, you can enable it for a specific resource using the server snippet annotation:

```yaml
nginx.org/server-snippets: |
otel_trace on;
```

You can enable it for specific paths using [Mergeable Ingress resources]({{< ref "/nic/configuration/ingress-resources/cross-namespace-configuration.md" >}}).

Use the server snippet annotation for the paths of a specific Minion Ingress resource:

```yaml
nginx.org/location-snippets: |
otel_trace on;
```

#### Disable a specific resource or path

With OpenTelemetry **enabled** globally, you can disable it for a specific resource using the server snippet annotation:

```yaml
nginx.org/server-snippets: |
otel_trace off;
```

You can disable it for specific paths using [Mergeable Ingress resources]({{< ref "/nic/configuration/ingress-resources/cross-namespace-configuration.md" >}}).

Use the server snippet annotation for the paths of a specific Minion Ingress resource:

```yaml
nginx.org/location-snippets: |
otel_trace off;
```

## Customize OpenTelemetry

{{< call-out "note" >}}

You cannot modify the additional directives in the _otel_exporter_ block using snippets.

{{< /call-out >}}

You can customize OpenTelemetry through the supported [OpenTelemetry module directives](https://nginx.org/en/docs/ngx_otel_module.html).

Use the `location-snippets` ConfigMap keys or annotations to insert those directives into the generated NGINX configuration.
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
---
nd-docs: DOCS-618
doctypes:
- ''
title: OpenTracing (Removed in v5.0.0)
title: Enable OpenTracing (Removed in v5.0.0)
toc: true
weight: 500
weight: 700
nd-content-type: how-to
nd-product: NIC
nd-docs: DOCS-618
---
OpenTracing support has been removed from v5.0.0 of F5 NGINX Ingress Controller.


Learn how to use OpenTracing with F5 NGINX Ingress Controller.
This topic describes how to OpenTracing with F5 NGINX Ingress Controller.

NGINX Ingress Controller supports [OpenTracing](https://opentracing.io/) with the third-party module [opentracing-contrib/nginx-opentracing](https://github.com/opentracing-contrib/nginx-opentracing).

{{< call-out "warning" >}}

OpenTracing support has been removed from v5.0.0 of NGINX Ingress Controller.

From v5.1.0 onwards, you should follow the guidance in [Configure OpenTelemetry]({{< ref "/nic/logging-and-monitoring/opentelemetry.md" >}}).

{{< /call-out >}}

## Prerequisites

1. Use a NGINX Ingress Controller image that contains OpenTracing.
Expand Down
53 changes: 30 additions & 23 deletions content/nic/logging-and-monitoring/prometheus.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,41 @@
---
nd-docs: DOCS-614
doctypes:
- concept
title: Prometheus
title: Enable Prometheus metrics
toc: true
weight: 2000
weight: 400
nd-content-type: how-to
nd-product: NIC
nd-docs: DOCS-614
---

NGINX Ingress Controller exposes metrics in the [Prometheus](https://prometheus.io/) format. Those include NGINX/NGINX Plus and the Ingress Controller metrics.
This topic describes how to enable [Prometheus metrics](https://prometheus.io/) for F5 NGINX Ingress Controller.

The metrics exposed include NGINX Ingress Controller data, as well as NGINX Open Source and NGINX Plus.

## Enabling Metrics

### Using Helm

To enable Prometheus metrics when using *Helm* to install NGINX Ingress Controller, configure the `prometheus.*` parameters of the Helm chart.

See the [Installation with Helm]({{< ref "/nic/installation/installing-nic/installation-with-helm.md" >}}) topic.

#### Using ServiceMonitor

When deploying with *Helm*, you can deploy a `Service` and `ServiceMonitor` resource using the `prometheus.service.*` and `prometheus.serviceMonitor.*` parameters.
When these resources are deployed, Prometheus metrics exposed by NGINX Ingress Controller can be captured and enumerated using a `Prometheus` resource alongside a Prometheus Operator deployment.

To view metrics captured this way, you will need:

* A working [Prometheus resource and Prometheus Operator](https://prometheus-operator.dev/docs/getting-started/introduction/)
* The latest ServiceMonitor CRD from the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) repository:

```shell
LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name)
curl https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/$LATEST/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml | kubectl create -f -
```

### Using Manifests

If you're using *Kubernetes manifests* (Deployment or DaemonSet) to install the Ingress Controller, to enable Prometheus metrics:

1. Run the Ingress Controller with the `-enable-prometheus-metrics` [command-line argument]({{< ref "/nic/configuration/global-configuration/command-line-arguments.md" >}}). As a result, the Ingress Controller will expose NGINX or NGINX Plus metrics in the Prometheus format via the path `/metrics` on port `9113` (customizable via the `-prometheus-metrics-listen-port` command-line argument).
Expand All @@ -32,23 +56,6 @@ If you're using *Kubernetes manifests* (Deployment or DaemonSet) to install the
prometheus.io/scheme: http
```

### Using Helm

If you're using *Helm* to install the Ingress Controller, to enable Prometheus metrics, configure the `prometheus.*` parameters of the Helm chart. See the [Installation with Helm]({{< ref "/nic/installation/installing-nic/installation-with-helm.md" >}}) doc.

### Using ServiceMonitor

When deploying with *Helm*, you can deploy a `Service` and `ServiceMonitor` resource using the `prometheus.service.*` and `prometheus.serviceMonitor.*` parameters.
When these resources are deployed, Prometheus metrics exposed by NGINX Ingress Controller can be captured and enumerated using a `Prometheus` resource alongside a Prometheus Operator deployment.

To view metrics captured this way, the following is required:
* The latest ServiceMonitor CRD from the [prometheus-operator](https://github.com/prometheus-operator/prometheus-operator) repository:
```shell
LATEST=$(curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name)
curl https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/$LATEST/example/prometheus-operator-crd/monitoring.coreos.com_servicemonitors.yaml | kubectl create -f -
```
* A working [Prometheus resource and Prometheus Operator](https://prometheus-operator.dev/docs/getting-started/introduction/)

## Available Metrics

The Ingress Controller exports the following metrics:
Expand Down
4 changes: 2 additions & 2 deletions content/nic/logging-and-monitoring/service-insight.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Service Insight
title: Enable Service Insight
toc: true
weight: 2100
weight: 600
nd-content-type: how-to
nd-product: NIC
nd-docs: DOCS-1180
Expand Down
4 changes: 2 additions & 2 deletions content/nic/logging-and-monitoring/status-page.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Status Page
title: View the NGINX status page
toc: true
weight: 1900
weight: 200
nd-content-type: how-to
nd-product: NIC
nd-docs: DOCS-615
Expand Down
Loading