Skip to content

OBSDOCS-3341: Fix span metrics procedure #110687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
max-cx wants to merge 1 commit into
base: standalone-distributed-tracing-docs-main
Choose a base branch
from
+8 −35
Open

OBSDOCS-3341: Fix span metrics procedure #110687

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 8 additions & 35 deletions modules/distr-tracing-tempo-config-spanmetrics.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Prometheus that you deploy in your user-workload monitoring stack scrapes metric

.Procedure

. In the `OpenTelemetryCollector` custom resource of the OpenTelemetry Collector, enable the Spanmetrics Connector (`spanmetrics`), which derives metrics from traces and exports the metrics in the Prometheus format.
. In the `OpenTelemetryCollector` custom resource of the OpenTelemetry Collector, enable the Span Metrics Connector, which derives metrics from traces and exports the metrics in the Prometheus format.
+
The following is an example `OpenTelemetryCollector` custom resource for span RED:
+
Expand Down Expand Up @@ -83,12 +83,12 @@ spec:
where:

`enableMetrics`:: Creates the `ServiceMonitor` custom resource to enable scraping of the Prometheus exporter.
`spanmetrics`:: The Spanmetrics connector receives traces and exports metrics.
`spanmetrics`:: The Span Metrics Connector receives traces and exports metrics.
`otlp`:: The OTLP receiver receives spans in the OpenTelemetry protocol.
`prometheus`:: The Prometheus exporter exports metrics in the Prometheus format.
`prometheus`:: The Prometheus Exporter exports metrics in the Prometheus format.
`enabled`:: Resource attributes are dropped by default.
`exporters`:: The Spanmetrics connector is configured as an exporter in the traces pipeline.
`receivers`:: The Spanmetrics connector is configured as a receiver in the metrics pipeline.
`exporters`:: The Span Metrics Connector is configured as an exporter in the traces pipeline.
`receivers`:: The Span Metrics Connector is configured as a receiver in the metrics pipeline.

. In the `TempoStack` custom resource, enable the *Monitor* tab and set the Prometheus endpoint to the Thanos querier service to query the data from your user-defined monitoring stack.
+
Expand Down Expand Up @@ -128,37 +128,9 @@ where:
`enabled`:: Enables the monitoring tab in the Jaeger console.
`prometheusEndpoint`:: Service name for Thanos Querier from user-workload monitoring.

. Optional: Use the span RED metrics generated by the `spanmetrics` connector with alerting rules. For example, for alerts about a slow service or to define service level objectives (SLOs), the connector creates a `duration_bucket` histogram and the `calls` counter metric. These metrics have labels that identify the service, API name, operation type, and other attributes.
. Optional: Use the span RED metrics that the Span Metrics Connector generates with alerting rules.
+
The following are the labels of the metrics created in the `spanmetrics` connector:
+
[options="header"]
[cols="a, a, a"]
|===
|Label |Description |Values

|`service_name`
|Service name set by the `otel_service_name` environment variable.
|`frontend`

|`span_name`
| Name of the operation.
|
* `/`
* `/customer`

|`span_kind`
|Identifies the server, client, messaging, or internal operation.
|
* `SPAN_KIND_SERVER`
* `SPAN_KIND_CLIENT`
* `SPAN_KIND_PRODUCER`
* `SPAN_KIND_CONSUMER`
* `SPAN_KIND_INTERNAL`

|===
+
The following example `PrometheusRule` custom resource defines an alerting rule for SLO when not serving 95% of requests within 2000 ms on the front-end service:
For example, for alerts about a slow service or to define service level objectives (SLOs), the connector creates a `duration_bucket` histogram and the `calls` counter metric. The following example `PrometheusRule` custom resource defines an alerting rule for SLO when not serving 95% of requests within 2000 ms on the front-end service:
+
[source,yaml]
----
Expand All @@ -185,3 +157,4 @@ where:
[role="_additional-resources"]
.Additional resources
* link:https://docs.redhat.com/en/documentation/red_hat_openshift_distributed_tracing_platform/latest/html-single/installing_the_distributed_tracing_platform/index#configuring-permissions-and-tenants_distr-tracing-tempo-installing[Configuring the permissions and tenants]
* link:https://docs.redhat.com/en/documentation/red_hat_build_of_opentelemetry/latest/html/configuring_the_collector/otel-collector-connectors#otel-connectors-spanmetrics-connector_otel-collector-connectors[Span Metrics Connector]