diff --git a/dev.sh b/dev.sh old mode 100644 new mode 100755 diff --git a/docs/configuration/pgdog.toml/otel.md b/docs/configuration/pgdog.toml/otel.md index 39eae335..a8767c12 100644 --- a/docs/configuration/pgdog.toml/otel.md +++ b/docs/configuration/pgdog.toml/otel.md @@ -67,6 +67,27 @@ How often, in milliseconds, to push metrics to the OTLP endpoint. _Default:_ `10000` +### `temporality_preference` + +How exported metric points describe their value over time. See the [OpenTelemetry metric points spec](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#metric-points). + +| Value | Behavior | +| ------------ | --------------------------------------------------------------- | +| `Cumulative` | Points report the value accumulated since the exporter started. | +| `Delta` | Points report the change since the last export. | + + + + +_Default:_ `Cumulative`, or `Delta` when `datadog_api_key` is set. + +Values are case-insensitive. + +!!! warning "Datadog" +Sending `Cumulative` sums and histograms to Datadog is stateful and lossy: all points on a timeseries must reach the same Agent/exporter (constraining how you scale collectors), the first point of a new series may be dropped (causing gaps on restart), and histogram min/max may be missing or approximated. See [Datadog's OTLP delta temporality guide](https://docs.datadoghq.com/opentelemetry/guide/otlp_delta_temporality/?tab=python#implications-of-using-cumulative-aggregation-temporality). + + PgDog logs a warning when `datadog_api_key` is set alongside `Cumulative`. Set `IGNORE_DATADOG_CUMULATIVE_WARNING=1` to silence it. + ## Environment variables PgDog honors the standard OpenTelemetry environment variables. When set, they override the @@ -79,9 +100,11 @@ corresponding `pgdog.toml` values (or supply a value when the setting is omitted | `OTEL_METRIC_EXPORT_INTERVAL` | Push interval in milliseconds. Equivalent to `push_interval`. | | `OTEL_SERVICE_NAME` | Sets the `service.name` resource attribute. Takes precedence over any value set via `OTEL_RESOURCE_ATTRIBUTES`. _Default:_ `pgdog`. | | `OTEL_RESOURCE_ATTRIBUTES` | Comma-separated `key=value` pairs added as resource attributes on every metric (e.g. `env=prod,region=us-east-1`). Values may be percent-encoded. | +| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | Aggregation temporality for exported metrics. Equivalent to `temporality_preference`. | | `PGDOG_OTEL_NAMESPACE` | Metric name prefix. Equivalent to `namespace`. | | `DD_API_KEY` | Datadog API key. Equivalent to `datadog_api_key`. | + PgDog automatically sets the following resource attributes on every exported metric: `service.name` (defaults to `pgdog`), `service.instance.id`, and `host.name`. Use `OTEL_RESOURCE_ATTRIBUTES` to add or override any of these. diff --git a/docs/features/metrics.md b/docs/features/metrics.md index 16b200bc..abccdf16 100644 --- a/docs/features/metrics.md +++ b/docs/features/metrics.md @@ -34,8 +34,9 @@ You can connect to the admin database using any PostgreSQL client. It supports c ## OpenMetrics -[OpenMetrics](https://openmetrics.io/) is a standard for displaying metrics that can be ingested by a multitude of agents, e.g., Datadog, Prometheus, etc. The metrics are shown in a human-readable text format -and available from an HTTP endpoint. +[OpenMetrics](https://openmetrics.io/) is a standard for displaying metrics that can be ingested by a multitude of agents, e.g., Datadog, Prometheus, etc. The metrics are shown in a human-readable text format and available from an HTTP endpoint. + +We maintain multiple examples of OpenMetrics integrations, including [DataDog](https://github.com/pgdogdev/pgdog/tree/main/examples/datadog) and [Grafana using Prometheus](https://github.com/pgdog/pgdog/tree/main/examples/grafana_prometheus). ### Configuration @@ -146,9 +147,11 @@ The following metrics are exported via the OpenMetrics endpoint: |------------|-------------|------| | `clients` | Total number of connected clients. | `gauge` | | `cl_waiting` | Clients waiting for a connection from a pool. | `gauge` | +| `cl_locked` | Clients currently locked to a specific connection. | `gauge` | | `sv_active` | Servers currently serving client requests. | `gauge` | | `sv_idle` | Servers available for clients to use. | `gauge` | | `sv_idle_xact` | Servers currently idle in transaction. | `gauge` | +| `sv_locked` | Servers currently locked. | `gauge` | | `maxwait` | How long the first (oldest) client in the queue has waited (in seconds). | `gauge` | | `errors` | Errors that connections in the pool have experienced. | `counter` | | `out_of_sync` | Connections that have been returned to the pool in a broken state. | `counter` |