Improve prometheus plugin

[titilambert]

Hello, this is an improvement of prometheus input plugin:

• Prometheus parser added
• "prometheus_" prefix removed from metric names (could be readded using "name_prefix" option)
• Add includes/excludes options to select metric you (don't) want

Question:
@sparrc If those changes are fine to you, I would like to add "configuration examples" of Prometheus input plugin to get some data.
It could be something like (for kubelet):

[[inputs.prometheus]]
  urls = ["http://kube-node-1:4194/metrics"]
  excludes = ["^container_.*"]
  name_prefix = "k8s_"
  [inputs.prometheus.tags]
    kubeservice = "kubelet"

Where can I put such examples ? in plugins/inputs/prometheus/examples/*.conf folder ?

[sparrc]

These options in Config are more meant to be something that users would specify in the telegraf config file. From my understanding of the prometheus parser, PromFormat is something that gets specified by the headers in the http response?

ie, I don't think this needs to go here

[titilambert]

Ok ! You're right, it's http header...
How can I pass this stuff to the Prometheus parser ?

[sparrc]

@titilambert would any other plugin use this parser other than the prometheus input? The idea behind generic parsers was that it could get used by things like message broker consumers and the exec input. In this instance I don't think it's really the case? If so I think the prometheus parser should just live within the prometheus plugin directory.

[titilambert]

@sparrc good point ! I don't know if could be interesting for exec plugin ... (I'm not a Prometheus user)
If not I will move it as you suggest.

[sparrc]

let's just have it in the prometheus directory for now....it'd be easier to break it out into the general parsers later than the other way around

[titilambert]

perfect !

[titilambert]

@sparrc I just made the change !

[sparrc]

@titilambert I'd prefer not to have the includes and excludes options added here, is there a reason that pass and drop don't work for you?

If so I'd prefer we come up with a solution that applies to all plugins (similar to pass and drop)

[titilambert]

@sparrc
When I query a Prometheus page I get a lot of data/metrics and sometimes I just want some of them.
includes/excludes is to filter on metrics name but pass/drop is to filter on fields name...

So I'm agree, I will add it like pass/drop and tagpass/tagdrop. Do you have any suggestion about names: metricpass/metricdrop ???

[sparrc]

This should probably be in a separate PR, I would say:

1. Rename pass/drop to field_pass/field_drop (but keep legacy support)
2. Add metric_pass/metric_drop for dropping by metric names
3. Also make this consistent in output plugins. I believe output plugins currently pass/drop by metric name (I don't think we need to implement field_pass/field_drop for them yet)
4. Keep it all glob matching for now....if we need regex we can implement as separate arg or add a prefix to indicate regex

[titilambert]

@sparrc Perfect !

[titilambert]

This PR is now split (see #730)

[sparrc]

@titilambert now that #730 is merged can you rebase this?

I think that the additional config examples should go in the README (which might not exist yet)

[titilambert]

Rebased and Readme added

[jchauncey]

So I have been using this PR for a few days and it works really well. Only thing that should be done is add the ability to specify the bearer token and verify ssl attribute that I posted in the gist on the kubernetes issue.

[sparrc]

@titilambert is this ready to go?

[titilambert]

@sparrc yes, it's good for me !
As @jchauncey said it' just missed token handling but I remember a PR for this ... but I can not found it :(

[jchauncey]

Look in the kubernetes plugin issue. I posted a gist that has the code.

[jchauncey]

If you don't have it. I can post a branch with your code and my token stuff so you can pull that over.

[titilambert]

@jchauncey oki ! So @sparrc can merge this PR then we can add your token stuff with a new PR ?

[titilambert]

I just created an issue for this: #864

[sparrc]

thanks again @titilambert !