Feature Request

[alimousazy]

Feature Request

Implement aggregation/stats for metrics

Proposal:

I currently embed the aggregation code inside the Gather function for tail (input plugin) and allow the user to select which function to apply over metrics

` statFuncList: map[string]func(stats.Float64Data) (float64, error){

•                   "median": stats.Median,
  

•                   "sum":    stats.Sum,
  

•                   "max":    stats.Max,
  

•                   "min":    stats.Min,
  

•                   "dev":    stats.StandardDeviation,
  

•                   "var":    stats.Variance,
  

•                   "count":  func(p stats.Float64Data) (res float64, err error) { err = nil; res = float64(len(p)); return },
  

•                   "p95": func(p stats.Float64Data) (res float64, err error) {
  

•                           res, err = stats.PercentileNearestRank(p, 95)
  

•                           return
  

•                   },
  

•                   "p90": func(p stats.Float64Data) (res float64, err error) {
  

•                           res, err = stats.PercentileNearestRank(p, 90)
  

•                           return
  

•                   },
  

•                   "p10": func(p stats.Float64Data) (res float64, err error) {
  

•                           res, err = stats.PercentileNearestRank(p, 10)
  

•                           return
  

•                   },
  

•           },`
  

Aggregation logic inside the gather function

` if len(t.StatList) == 0 {

•           return nil
  

•   }
  

•   fieldMap := make(map[string]map[string][]float64)
  

•   metricTags := make(map[string]map[string]string)
  

•   current_elemnt := len(t.metCH)
  

•   for current_elemnt > 0 {
  

•           select {
  

•           case metric := <-t.metCH:
  

•                   name := metric.Name()
  

•                   if fieldMap[name] == nil {
  

•                           fieldMap[name] = make(map[string][]float64)
  

•                   }
  

•                   if metricTags[name] == nil {
  

•                           metricTags[name] = make(map[string]string)
  

•                   }
  

•                   metricTags[name] = metric.Tags()
  

•                   for key, val := range metric.Fields() {
  

•                           switch v := val.(type) {
  

•                           case float64:
  

•                                   fieldMap[name][key] = append(fieldMap[name][key], v)
  

•                           default:
  

•                                   return fmt.Errorf("When stats enabled all the fields should be of type float64")
  

•                           }
  

•                   }
  

•                   current_elemnt--
  

•           default:
  

•                   break
  

•           }
  

•   }
  

•   for name, fields := range fieldMap {
  

•           mFields := make(map[string]interface{})
  

•           var err error
  

•           for key, val := range fields {
  

•                   for statName, _ := range t.StatList {
  

•                           mFields[key+"_"+statName], err = t.statFuncList[statName](val)
  

•                           if err != nil {
  

•                                   return err
  

•                           }
  

•                   }
  

•           }
  

•           m, _ := telegraf.NewMetric(name, metricTags[name], mFields)
  

•           t.acc.AddFields(m.Name(), m.Fields(), m.Tags(), time.Now())`
  

Current behavior:

No aggregation implemented

Desired behavior:

Should I provide a pull request for the changes or is there any better place to do the aggregation ?

Use case: [Why is this important (helps with prioritizing requests)]

Currently we use customize tail plug which generate a lot of metrics in 1 min, We only interested in aggregation of these metrics (stats)

Note: I'm using https://github.com/montanaflynn/stats for generating statistics

[sparrc]

there is already an issue for this: #380