prometheus-scrape

[nagarnar]

Has anyone successfully used prometheus-scrape input plugin along with other filters? I saw similar question on stack-overflow https://stackoverflow.com/questions/74658644/fluent-bit-prometheus-scrape-input-is-not-record but have not seen any responses.
I have a need where I was looking to add/modify fields along with possibly using kubernetes filter to pull in the metadata which I believe currently works with logs.

[patrick-stephens]

This comes up a bit and likely needs better clarification in the docs.

Short answer: metrics inputs currently follow a different pipeline to log inputs, they do not have filters applied currently and require an output to support metric input data (not all do).

• https://fluent-all.slack.com/archives/C0CTQGHKJ/p1675247061156709
• https://fluent-all.slack.com/archives/C0CTQGHKJ/p1676371615976299

There is ongoing work in this area though.

[nagarnar]

Could you also tell me how I can get invited to the above channels you mentioned. Thanks

[patrick-stephens]

Via the docs there is a link: https://slack.fluentd.org/
You join, it's not by invite.

[edsiper]

@nagarnar what type of filtering on top of metrics are you looking for ?

[nagarnar]

I can think of all options like dropping metrics based on grep filter, modify records and of course the Kubernetes filter to add meta data similar to what metric beat's https://www.elastic.co/guide/en/beats/metricbeat/current/add-kubernetes-metadata.html

[nagarnar]

Also scraping some kubernetes targets say from kube-state-metrics and forwarding them using forward to fluentd aggregator throws error on fluentd end. I did post on fluentd and they think it is probably from the sender side fluent/fluentd#4083

[patrick-stephens]

What versions of each component are you using? There is a compatibility matrix somewhere... @cosmo0920 do you know?

[nagarnar]

fluentd: 1.13.3
fluentbit 2.0.9