From 38dca45d6c009d0f71373e929fa6ee83e4100565 Mon Sep 17 00:00:00 2001 From: Israel Blancas Date: Wed, 2 Oct 2024 04:40:53 +0200 Subject: [PATCH 1/3] TRACING-4615: improve Routing Connector documentation Signed-off-by: Israel Blancas --- .../otel-collector-connectors.adoc | 161 +++++++++++++----- 1 file changed, 120 insertions(+), 41 deletions(-) diff --git a/observability/otel/otel-collector/otel-collector-connectors.adoc b/observability/otel/otel-collector/otel-collector-connectors.adoc index eeb52eb26876..072a9c032445 100644 --- a/observability/otel/otel-collector/otel-collector-connectors.adoc +++ b/observability/otel/otel-collector/otel-collector-connectors.adoc @@ -8,47 +8,6 @@ toc::[] A connector connects two pipelines. It consumes data as an exporter at the end of one pipeline and emits data as a receiver at the start of another pipeline. It can consume and emit data of the same or different data type. It can generate and emit data to summarize the consumed data, or it can merely replicate or route data. -[id="routing-connector_{context}"] -== Routing Connector - -The Routing Connector routes logs, metrics, and traces to specified pipelines according to resource attributes and their routing conditions, which are written as OpenTelemetry Transformation Language (OTTL) statements. - -:FeatureName: The Routing Connector -include::snippets/technology-preview.adoc[] - -.OpenTelemetry Collector custom resource with an enabled Routing Connector -[source,yaml] ----- - config: | - connectors: - routing: - table: # <1> - - statement: route() where attributes["X-Tenant"] == "dev" # <2> - pipelines: [traces/dev] # <3> - - statement: route() where attributes["X-Tenant"] == "prod" - pipelines: [traces/prod] - default_pipelines: [traces/dev] # <4> - error_mode: ignore # <5> - match_once: false # <6> - service: - pipelines: - traces/in: - receivers: [otlp] - exporters: [routing] - traces/dev: - receivers: [routing] - exporters: [otlp/dev] - traces/prod: - receivers: [routing] - exporters: [otlp/prod] ----- -<1> Connector routing table. -<2> Routing conditions written as OTTL statements. -<3> Destination pipelines for routing the matching telemetry data. -<4> Destination pipelines for routing the telemetry data for which no routing condition is satisfied. -<5> Error-handling mode: The `propagate` value is for logging an error and dropping the payload. The `ignore` value is for ignoring the condition and attempting to match with the next one. The `silent` value is the same as `ignore` but without logging the error. The default is `propagate`. -<6> When set to `true`, the payload is routed only to the first pipeline whose routing condition is met. The default is `false`. - [id="forward-connector_{context}"] == Forward Connector @@ -94,6 +53,126 @@ service: # ... ---- +[id="routing-connector_{context}"] +== Routing Connector + +The Routing Connector routes logs, metrics, and traces to specified pipelines according to resource attributes and their routing conditions, which are written as OpenTelemetry Transformation Language (OTTL) statements. + +:FeatureName: The Routing Connector +include::snippets/technology-preview.adoc[] + +.OpenTelemetry Collector custom resource with an enabled Routing Connector +[source,yaml] +---- + config: | + connectors: + routing: + table: + - statement: route() where attributes["X-Tenant"] == "dev" # <1> + pipelines: [traces/dev] # <2> + - statement: route() where attributes["X-Tenant"] == "prod" + pipelines: [traces/prod] + default_pipelines: [traces/dev] # <3> + error_mode: ignore + match_once: false + service: + pipelines: + traces/in: + receivers: [otlp] + exporters: [routing] + traces/dev: + receivers: [routing] + exporters: [otlp/dev] + traces/prod: + receivers: [routing] + exporters: [otlp/prod] +---- +<1> Routing conditions written as OTTL statements. +<2> Destination pipelines for routing the matching telemetry data. In this case, the `traces/dev` pipeline when the `X-Tenant` attribute is equal to `dev`. +<3> In this example, if `X-Tenant` attribute is not `dev` or `prod`, the telemetry data will be forwarded to those pipelines. + +[NOTE] +==== +The connector routes exclusively based on OTTL statements, which are limited to resource attributes. Currently, it does not support matching against context values. +==== + +.Parameters used by the Routing Connector +[options="header"] +[cols="a,a,a"] +|=== +|Parameter |Description |Default + +|`table` +|Connector routing table containing routing conditions and destination pipelines. +|`[]` + +|`table.statement` +|Routing conditions written as OTTL statements. +|N/A + +|`table.pipelines` +|Destination pipelines for routing the matching telemetry data. +|N/A + +|`default_pipelines` +|Destination pipelines for routing the telemetry data for which no routing condition is satisfied. +|`[]` + +|`error_mode` +|Error-handling mode: `propagate` logs an error and drops the payload, `ignore` ignores the condition and attempts to match with the next one, and `silent` is the same as `ignore` but without logging the error. +|`propagate` + +|`match_once` +|When set to `true`, the payload is routed only to the first pipeline whose routing condition is met. +|`false` +|=== + +=== Troubleshooting the Routing Connector + +If you encounter issues with the Routing Connector, use the following troubleshooting steps to help diagnose and resolve common problems. + +==== Pipeline not receiving telemetry data + +If telemetry data is not being routed to the expected pipeline, check the following: + +.Procedure + +- Verify the OTTL statement syntax in the routing table: ensure that the `attributes` used in your `statement` match the telemetry resource attributes exactly, including correct attribute names and values. Typos in the OTTL expressions can prevent routing. ++ +- Ensure that the destination pipeline exists in the `service.pipelines` configuration: for example, ensure that `traces/dev` or `traces/prod` pipelines are defined correctly in the OpenTelemetry Collector custom resource. + +==== Incorrect routing conditions + +If telemetry data is not being routed to the intended pipelines, the routing condition in the OTTL statement might be incorrect. + +.Procedure + +- Confirm that the attribute used in the condition exists in the telemetry resource: use the <> to inspect telemetry data and verify that the `X-Tenant` attribute or other used attributes are present and correctly populated. ++ +- Test with a simplified condition: try routing based on simpler conditions (e.g., `route() where attributes["X-Tenant"] != ""`) to isolate potential issues with the logic in more complex expressions. + +==== Default pipeline is not applied + +If telemetry data that doesn’t match any routing conditions is not being routed to the default pipeline, check the following: + +.Procedure + +- Verify the `default_pipelines` parameter: ensure that `default_pipelines` is correctly defined. For example, the `traces/dev` pipeline should be listed in `default_pipelines`. ++ +- Ensure that the attribute used in the routing conditions is correct: if the attribute doesn't exist, the Routing Connector will skip routing and use the default pipeline. + +==== Error handling behavior not as expected + +If errors are not handled as expected, check the `error_mode` parameter: + +.Procedure + +- `propagate`: Logs errors and drops the payload. Ensure that the logging level is set to capture these errors. ++ +- `ignore`: Ignores the error and attempts to match the next condition. Ensure this behavior matches your intended setup. ++ +- `silent`: Does not log errors. If you're not seeing errors in logs, confirm that `silent` is not set in your configuration. + [id="spanmetrics-connector_{context}"] == Spanmetrics Connector From 81d923b6da24632fb61f761742a89a57b64fd199 Mon Sep 17 00:00:00 2001 From: Israel Blancas Date: Wed, 2 Oct 2024 05:07:23 +0200 Subject: [PATCH 2/3] Fix link Signed-off-by: Israel Blancas --- .../otel/otel-collector/otel-collector-connectors.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/observability/otel/otel-collector/otel-collector-connectors.adoc b/observability/otel/otel-collector/otel-collector-connectors.adoc index 072a9c032445..7b5b0a2f7999 100644 --- a/observability/otel/otel-collector/otel-collector-connectors.adoc +++ b/observability/otel/otel-collector/otel-collector-connectors.adoc @@ -147,7 +147,7 @@ If telemetry data is not being routed to the intended pipelines, the routing con .Procedure -- Confirm that the attribute used in the condition exists in the telemetry resource: use the <> to inspect telemetry data and verify that the `X-Tenant` attribute or other used attributes are present and correctly populated. +- Confirm that the attribute used in the condition exists in the telemetry resource: use the link:otel-collector-exporters.html#debug-exporter_otel-collector-exporters[Debug exporter] to inspect telemetry data and verify that the `X-Tenant` attribute or other used attributes are present and correctly populated. + - Test with a simplified condition: try routing based on simpler conditions (e.g., `route() where attributes["X-Tenant"] != ""`) to isolate potential issues with the logic in more complex expressions. @@ -203,3 +203,4 @@ include::snippets/technology-preview.adoc[] [id="additional-resources_otel-collector-connectors_{context}"] == Additional resources * link:https://opentelemetry.io/docs/specs/otlp/[OpenTelemetry Protocol (OTLP) documentation] +* link:https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/README.md[OpenTelemetry Transformation Language (OTTL) documentation] From a60c413f516bea649b7294f8f90b2d7e96c49b6f Mon Sep 17 00:00:00 2001 From: Israel Blancas Date: Wed, 2 Oct 2024 05:07:52 +0200 Subject: [PATCH 3/3] Fix link Signed-off-by: Israel Blancas --- .../otel/otel-collector/otel-collector-connectors.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/observability/otel/otel-collector/otel-collector-connectors.adoc b/observability/otel/otel-collector/otel-collector-connectors.adoc index 7b5b0a2f7999..dc69702202c1 100644 --- a/observability/otel/otel-collector/otel-collector-connectors.adoc +++ b/observability/otel/otel-collector/otel-collector-connectors.adoc @@ -153,7 +153,7 @@ If telemetry data is not being routed to the intended pipelines, the routing con ==== Default pipeline is not applied -If telemetry data that doesn’t match any routing conditions is not being routed to the default pipeline, check the following: +If telemetry data that doesn't match any routing conditions is not being routed to the default pipeline, check the following: .Procedure