Skip to content

Commit 75945b3

Browse files
committed
pipeline: router: add metrics and traces routing examples
Add conditional routing examples for metrics and traces signals: - Route metrics by name pattern (CPU, memory metrics) - Route traces by service name and span attributes - Route multiple signal types (logs, metrics, traces) in one config Signed-off-by: Eric D. Schabell <eric@schabell.org>
1 parent bf68553 commit 75945b3

File tree

1 file changed

+251
-0
lines changed

1 file changed

+251
-0
lines changed

pipeline/router.md

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -502,6 +502,257 @@ pipeline:
502502
{% endtab %}
503503
{% endtabs %}
504504

505+
### Route metrics by name or labels
506+
507+
This example routes metrics to different backends based on metric name patterns:
508+
509+
{% tabs %}
510+
{% tab title="fluent-bit.yaml" %}
511+
512+
```yaml
513+
pipeline:
514+
inputs:
515+
- name: node_exporter_metrics
516+
tag: node.metrics
517+
routes:
518+
metrics:
519+
- name: cpu_metrics
520+
condition:
521+
op: or
522+
rules:
523+
- field: "$metric.name"
524+
op: regex
525+
value: "^node_cpu_.*"
526+
- field: "$metric.name"
527+
op: regex
528+
value: "^process_cpu_.*"
529+
to:
530+
outputs:
531+
- cpu_metrics_output
532+
533+
- name: memory_metrics
534+
condition:
535+
op: or
536+
rules:
537+
- field: "$metric.name"
538+
op: regex
539+
value: "^node_memory_.*"
540+
- field: "$metric.name"
541+
op: regex
542+
value: "^process_resident_memory_.*"
543+
to:
544+
outputs:
545+
- memory_metrics_output
546+
547+
- name: all_metrics
548+
condition:
549+
default: true
550+
to:
551+
outputs:
552+
- general_metrics_output
553+
554+
outputs:
555+
- name: prometheus_remote_write
556+
alias: cpu_metrics_output
557+
host: prometheus-cpu.example.com
558+
uri: /api/v1/write
559+
560+
- name: prometheus_remote_write
561+
alias: memory_metrics_output
562+
host: prometheus-memory.example.com
563+
uri: /api/v1/write
564+
565+
- name: prometheus_remote_write
566+
alias: general_metrics_output
567+
host: prometheus.example.com
568+
uri: /api/v1/write
569+
```
570+
571+
{% endtab %}
572+
{% endtabs %}
573+
574+
This configuration routes CPU-related metrics to a dedicated Prometheus instance, memory metrics to another, and all other metrics to a general metrics backend.
575+
576+
### Route traces by service or attributes
577+
578+
This example routes traces to different backends based on service name and span attributes:
579+
580+
{% tabs %}
581+
{% tab title="fluent-bit.yaml" %}
582+
583+
```yaml
584+
pipeline:
585+
inputs:
586+
- name: opentelemetry
587+
listen: 0.0.0.0
588+
port: 4318
589+
tag: otel.traces
590+
routes:
591+
traces:
592+
- name: critical_service_traces
593+
condition:
594+
op: or
595+
rules:
596+
- field: "$resource['service.name']"
597+
op: in
598+
value: ["payment-service", "auth-service", "order-service"]
599+
context: otel_resource_attributes
600+
- field: "$span.status_code"
601+
op: eq
602+
value: "ERROR"
603+
to:
604+
outputs:
605+
- critical_traces_output
606+
607+
- name: high_latency_traces
608+
condition:
609+
op: and
610+
rules:
611+
- field: "$span.duration"
612+
op: gt
613+
value: 5000000000
614+
to:
615+
outputs:
616+
- latency_traces_output
617+
618+
- name: default_traces
619+
condition:
620+
default: true
621+
to:
622+
outputs:
623+
- general_traces_output
624+
625+
outputs:
626+
- name: opentelemetry
627+
alias: critical_traces_output
628+
host: jaeger-critical.example.com
629+
port: 4317
630+
631+
- name: opentelemetry
632+
alias: latency_traces_output
633+
host: jaeger-latency.example.com
634+
port: 4317
635+
636+
- name: opentelemetry
637+
alias: general_traces_output
638+
host: jaeger.example.com
639+
port: 4317
640+
```
641+
642+
{% endtab %}
643+
{% endtabs %}
644+
645+
This configuration routes traces from critical services and error traces to a dedicated tracing backend, high-latency traces to a latency analysis backend, and all other traces to a general backend.
646+
647+
### Route multiple signal types
648+
649+
This example demonstrates routing logs, metrics, and traces with a single input configuration:
650+
651+
{% tabs %}
652+
{% tab title="fluent-bit.yaml" %}
653+
654+
```yaml
655+
pipeline:
656+
inputs:
657+
- name: opentelemetry
658+
listen: 0.0.0.0
659+
port: 4318
660+
tag: otel.data
661+
routes:
662+
logs:
663+
- name: error_logs
664+
condition:
665+
op: and
666+
rules:
667+
- field: "$severity_text"
668+
op: in
669+
value: ["ERROR", "FATAL", "CRITICAL"]
670+
to:
671+
outputs:
672+
- error_logs_output
673+
674+
- name: all_logs
675+
condition:
676+
default: true
677+
to:
678+
outputs:
679+
- general_logs_output
680+
681+
metrics:
682+
- name: application_metrics
683+
condition:
684+
op: and
685+
rules:
686+
- field: "$resource['service.namespace']"
687+
op: eq
688+
value: "production"
689+
context: otel_resource_attributes
690+
to:
691+
outputs:
692+
- prod_metrics_output
693+
694+
- name: all_metrics
695+
condition:
696+
default: true
697+
to:
698+
outputs:
699+
- general_metrics_output
700+
701+
traces:
702+
- name: sampled_traces
703+
condition:
704+
op: and
705+
rules:
706+
- field: "$resource['environment']"
707+
op: eq
708+
value: "production"
709+
context: otel_resource_attributes
710+
to:
711+
outputs:
712+
- prod_traces_output
713+
714+
- name: all_traces
715+
condition:
716+
default: true
717+
to:
718+
outputs:
719+
- general_traces_output
720+
721+
outputs:
722+
- name: loki
723+
alias: error_logs_output
724+
host: loki-errors.example.com
725+
726+
- name: loki
727+
alias: general_logs_output
728+
host: loki.example.com
729+
730+
- name: prometheus_remote_write
731+
alias: prod_metrics_output
732+
host: prometheus-prod.example.com
733+
uri: /api/v1/write
734+
735+
- name: prometheus_remote_write
736+
alias: general_metrics_output
737+
host: prometheus.example.com
738+
uri: /api/v1/write
739+
740+
- name: opentelemetry
741+
alias: prod_traces_output
742+
host: jaeger-prod.example.com
743+
port: 4317
744+
745+
- name: opentelemetry
746+
alias: general_traces_output
747+
host: jaeger.example.com
748+
port: 4317
749+
```
750+
751+
{% endtab %}
752+
{% endtabs %}
753+
754+
This configuration shows how to define separate routing rules for each signal type within the same input, enabling unified observability data collection with differentiated routing.
755+
505756
## Choosing a routing approach
506757

507758
Use the following guidelines to choose between tag-based and conditional routing:

0 commit comments

Comments
 (0)