Add support for prometheus expoter output plugin for fluentbit #840
Conversation
Signed-off-by: karan k <karan.k@oracle.com>
| @@ -92,6 +92,8 @@ type OutputSpec struct { | |||
| OpenSearch *output.OpenSearch `json:"opensearch,omitempty"` | |||
| // OpenTelemetry defines OpenTelemetry Output configuration. | |||
| OpenTelemetry *output.OpenTelemetry `json:"opentelemetry,omitempty"` | |||
| // PrometheusExporter_types defines Prometheus exporter configuration to expose metrics from Fluent Bit. | |||
There was a problem hiding this comment.
We might need to add comments here and in the values.yaml that it's recommended to add Prometheus exporter output only when the FluentBit is deployed as a single replica collector mode (A statefulset with 1 replica) because prometheus might scrape duplicated metrics if it's added to the default agent mode (FluentBit DaemonSet)
https://github.com/fluent/fluent-operator/blob/master/controllers/collector_controller.go
There was a problem hiding this comment.
Each Fluentbit pod will expose metrics related to itself, and it is probable for multiple pods to share identical metrics while exhibiting distinct values. Typically, customers require to scrape all metrics exposed from each pod.
There was a problem hiding this comment.
Got it, you're using the Prometheus exporter output to collect metrics of all fluentbit pods. Then it make sense to me. I mean for other cases other than the metrics of fluentbit itself
There was a problem hiding this comment.
@benjaminhuo Let me know, if we are good
There was a problem hiding this comment.
@benjaminhuo Let me know, if we are good
yep, I'm ready to merge
What this PR does / why we need it:
Prometheus exporter output plugin is added for fluentbit so that fluentbit metrics can be exposed on the specific port such it can be scraped by the prometheus.
Which issue(s) this PR fixes:
Fixes #841
Does this PR introduced a user-facing change?
Additional documentation, usage docs, etc.: