You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pipeline: router: add metrics and traces routing examples
Add conditional routing examples for metrics and traces signals:
- Route metrics by name pattern (CPU, memory metrics)
- Route traces by service name and span attributes
- Route multiple signal types (logs, metrics, traces) in one config
Signed-off-by: Eric D. Schabell <eric@schabell.org>
Copy file name to clipboardExpand all lines: pipeline/router.md
+251Lines changed: 251 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -502,6 +502,257 @@ pipeline:
502
502
{% endtab %}
503
503
{% endtabs %}
504
504
505
+
### Route metrics by name or labels
506
+
507
+
This example routes metrics to different backends based on metric name patterns:
508
+
509
+
{% tabs %}
510
+
{% tab title="fluent-bit.yaml" %}
511
+
512
+
```yaml
513
+
pipeline:
514
+
inputs:
515
+
- name: node_exporter_metrics
516
+
tag: node.metrics
517
+
routes:
518
+
metrics:
519
+
- name: cpu_metrics
520
+
condition:
521
+
op: or
522
+
rules:
523
+
- field: "$metric.name"
524
+
op: regex
525
+
value: "^node_cpu_.*"
526
+
- field: "$metric.name"
527
+
op: regex
528
+
value: "^process_cpu_.*"
529
+
to:
530
+
outputs:
531
+
- cpu_metrics_output
532
+
533
+
- name: memory_metrics
534
+
condition:
535
+
op: or
536
+
rules:
537
+
- field: "$metric.name"
538
+
op: regex
539
+
value: "^node_memory_.*"
540
+
- field: "$metric.name"
541
+
op: regex
542
+
value: "^process_resident_memory_.*"
543
+
to:
544
+
outputs:
545
+
- memory_metrics_output
546
+
547
+
- name: all_metrics
548
+
condition:
549
+
default: true
550
+
to:
551
+
outputs:
552
+
- general_metrics_output
553
+
554
+
outputs:
555
+
- name: prometheus_remote_write
556
+
alias: cpu_metrics_output
557
+
host: prometheus-cpu.example.com
558
+
uri: /api/v1/write
559
+
560
+
- name: prometheus_remote_write
561
+
alias: memory_metrics_output
562
+
host: prometheus-memory.example.com
563
+
uri: /api/v1/write
564
+
565
+
- name: prometheus_remote_write
566
+
alias: general_metrics_output
567
+
host: prometheus.example.com
568
+
uri: /api/v1/write
569
+
```
570
+
571
+
{% endtab %}
572
+
{% endtabs %}
573
+
574
+
This configuration routes CPU-related metrics to a dedicated Prometheus instance, memory metrics to another, and all other metrics to a general metrics backend.
575
+
576
+
### Route traces by service or attributes
577
+
578
+
This example routes traces to different backends based on service name and span attributes:
This configuration routes traces from critical services and error traces to a dedicated tracing backend, high-latency traces to a latency analysis backend, and all other traces to a general backend.
646
+
647
+
### Route multiple signal types
648
+
649
+
This example demonstrates routing logs, metrics, and traces with a single input configuration:
650
+
651
+
{% tabs %}
652
+
{% tab title="fluent-bit.yaml" %}
653
+
654
+
```yaml
655
+
pipeline:
656
+
inputs:
657
+
- name: opentelemetry
658
+
listen: 0.0.0.0
659
+
port: 4318
660
+
tag: otel.data
661
+
routes:
662
+
logs:
663
+
- name: error_logs
664
+
condition:
665
+
op: and
666
+
rules:
667
+
- field: "$severity_text"
668
+
op: in
669
+
value: ["ERROR", "FATAL", "CRITICAL"]
670
+
to:
671
+
outputs:
672
+
- error_logs_output
673
+
674
+
- name: all_logs
675
+
condition:
676
+
default: true
677
+
to:
678
+
outputs:
679
+
- general_logs_output
680
+
681
+
metrics:
682
+
- name: application_metrics
683
+
condition:
684
+
op: and
685
+
rules:
686
+
- field: "$resource['service.namespace']"
687
+
op: eq
688
+
value: "production"
689
+
context: otel_resource_attributes
690
+
to:
691
+
outputs:
692
+
- prod_metrics_output
693
+
694
+
- name: all_metrics
695
+
condition:
696
+
default: true
697
+
to:
698
+
outputs:
699
+
- general_metrics_output
700
+
701
+
traces:
702
+
- name: sampled_traces
703
+
condition:
704
+
op: and
705
+
rules:
706
+
- field: "$resource['environment']"
707
+
op: eq
708
+
value: "production"
709
+
context: otel_resource_attributes
710
+
to:
711
+
outputs:
712
+
- prod_traces_output
713
+
714
+
- name: all_traces
715
+
condition:
716
+
default: true
717
+
to:
718
+
outputs:
719
+
- general_traces_output
720
+
721
+
outputs:
722
+
- name: loki
723
+
alias: error_logs_output
724
+
host: loki-errors.example.com
725
+
726
+
- name: loki
727
+
alias: general_logs_output
728
+
host: loki.example.com
729
+
730
+
- name: prometheus_remote_write
731
+
alias: prod_metrics_output
732
+
host: prometheus-prod.example.com
733
+
uri: /api/v1/write
734
+
735
+
- name: prometheus_remote_write
736
+
alias: general_metrics_output
737
+
host: prometheus.example.com
738
+
uri: /api/v1/write
739
+
740
+
- name: opentelemetry
741
+
alias: prod_traces_output
742
+
host: jaeger-prod.example.com
743
+
port: 4317
744
+
745
+
- name: opentelemetry
746
+
alias: general_traces_output
747
+
host: jaeger.example.com
748
+
port: 4317
749
+
```
750
+
751
+
{% endtab %}
752
+
{% endtabs %}
753
+
754
+
This configuration shows how to define separate routing rules for each signal type within the same input, enabling unified observability data collection with differentiated routing.
755
+
505
756
## Choosing a routing approach
506
757
507
758
Use the following guidelines to choose between tag-based and conditional routing:
0 commit comments