Support StatisticValues in cloudwatch output plugin (#4318) #4364
Conversation
danielnelson
added
feat
| } | ||
|
|
||
| // Try to parse statisticSet first, then build statistic/value datum accordingly. | ||
| set, ok := getStatisticSet(point) |
There was a problem hiding this comment.
What about if the telegraf.Metric contains multiple aggregated field sets like this (leaving out some required aggregations):
cpu foo_min=1,foo_max=2,bar_min=1,bar_max=2
There was a problem hiding this comment.
If getStatisticSet() could not get all required fields, it just keeps the original behavior and sends them as independent metrics. It would generate 4 metrics to cloudwatch: cpu_foo_min, cpu_foo_max, cpu_bar_min and cpu_bar_max.
There was a problem hiding this comment.
Sorry, my example was somewhat lazy. What if there are two full sets of statistics in a single telegraf.Metric, does it handle this case?
cpu foo_min=1,foo_max=2,foo_sum=4,foo_count=2,bar_min=1,bar_max=2,bar_sum=4,bar_count=2
There was a problem hiding this comment.
Good point. Will support this multiple sets case.
| ## calculate required statistic fields (count, min, max, and sum) and | ||
| ## enable this flag. This plugin would try to parse those fields and | ||
| ## send statistic values to Cloudwatch. | ||
| # enable_statistic_values = false |
There was a problem hiding this comment.
Let's call this option write_statistics, this is little shorter, but also communicates that it is something that affects the output data. So long as you are in here, can you mention in the comment that if a field does not have the required statistic fields then it will still be sent as a custom/raw metric.
There was a problem hiding this comment.
okay, will update the field name and comments
|
@danielnelson any update for this PR? |
|
Already supported multiple statistic sets. |
| switch { | ||
| case strings.HasSuffix(name, "_max"): | ||
| sType = statisticTypeMax | ||
| fieldName = name[:len(name)-len("_max")] |
There was a problem hiding this comment.
You can use string.TrimSuffix(name, "_max") here
|
|
||
| func (f *valueField) buildDatum() []*cloudwatch.MetricDatum { | ||
|
|
||
| // Do CloudWatch boundary checking |
There was a problem hiding this comment.
Don't we still need to do the boundary checking on the statistic datums? Maybe we can do this where we call convert?
All field values would apply to the same boundary checking.
|
Requested changes are fixed. |
|
|
||
| ### write_statistics | ||
|
|
||
| If you have a large amount of metrics, you should consider to send statistic |
There was a problem hiding this comment.
Some minor grammar:
- s/consider to send/consider sending/
- s/ which could/. This should/
- s/If enable this flag/If this flag is enabled/
|
|
||
| if f.hasAllFields() { | ||
| // If we have all required fields, we build datum with StatisticValues | ||
| min, _ := f.values[statisticTypeMin] |
There was a problem hiding this comment.
You don't need to change this, but just using min := f.values[statisticTypeMin] is sufficient, since you are ignoring the bool value anyways.
| @@ -50,6 +172,14 @@ var sampleConfig = ` | |||
|
|
|||
| ## Namespace for the CloudWatch MetricDatums | |||
| namespace = "InfluxData/Telegraf" | |||
|
|
|||
| ## If you have a large amount of metrics, you should consider to send statistic | |||
There was a problem hiding this comment.
Same grammar suggestions as above.
| // Make a MetricDatum from telegraf.Metric. It would check if all required fields of | ||
| // cloudwatch.StatisticSet are available. If so, it would build MetricDatum from statistic values. | ||
| // Otherwise, fields would still been built independently. | ||
| func BuildMetricDatum(buildStatistic bool, point telegraf.Metric) []*cloudwatch.MetricDatum { |
There was a problem hiding this comment.
In general, changing an exported function's signature is advised against. Since nothing outside of this package uses it, maybe add a note that BuildMetricDatum(telegraf.Metric) is deprecated and create a new (unexported) buildMetricDatum(bool, Metric) function.
There was a problem hiding this comment.
This isn't something we are concerned about right now, maybe someday but not now. The only API we provide stability for is the TOML file. I keep meaning to document this, but haven't gotten to it.
There was a problem hiding this comment.
If this is not an issue, I could help to unexport all exported functions to avoid confusion.
How do you think?
| value = t | ||
| case bool: | ||
| if t { | ||
| value = 1 |
There was a problem hiding this comment.
I see this is how it was before, but should this be float64(1) and float64(0) in order to preserve the type when sending to an output? cc @danielnelson
There was a problem hiding this comment.
This looks okay to me, value is a float64 and we are assigning a const to it.
Fix #4318
Required for all PRs:
Add
enable_statistic_valuesflag. If true, this plugin would try to parse required statistic fields and send StatisticSet to Cloudwatch.