Support for histograms

[jcellary]

It would be very useful to have histograms displayed

Example:

[torkelo]

Yes this is definitely something I want to do!

[sahilthapar]

+1

[jcellary]

Actually after giving more though to it the histogram I pasted in before doesn't make much sense since we are displaying data in time, so the chart needs to reflect that. So the diagram could look like this:

http://fastly.github.io/epoch/real-time/#heatmap

Actually it might be even possible to use the linked code in Grafana.

Is this item on the roadmap for the nearest future? Just asking if we should expect it any time soon :)

Thanks!

[torkelo]

well I think the histogram is nice as well (to see the distribution of values over a time period). The heatmap is also nice, but the question is how to fetch the data for it, since I need all values for each time point.

[XANi]

InfluxDB have histogram function

select histogram(value, bucket_size) from dataseries

and if you group by time you will get histogram for each period, effectively from sth like

select histogram(value) from /dc1.es1.ipmi.temperature.Ambient_Temp_front_panel_board.*/ where time > now() - 22h group by time(10m)

you get

|          time | bucket_start | count |
|---------------+--------------+-------|
| 1406728800000 |           18 |     1 |
| 1406728200000 |           17 |     3 |
| 1406728200000 |           18 |     9 |
| 1406728200000 |           19 |    11 |
| 1406728200000 |           20 |     7 |
| 1406728200000 |           21 |     9 |
| 1406728200000 |           22 |    16 |
| 1406728200000 |           23 |     5 |
| 1406727600000 |           23 |     3 |
| 1406727600000 |           22 |    42 |
| 1406727600000 |           21 |    15 |
| 1406727000000 |           21 |    12 |
| 1406727000000 |           20 |    17 |
| 1406727000000 |           19 |    11 |
| 1406727000000 |           18 |    15 |
| 1406727000000 |           17 |     5 |
| 1406726400000 |           17 |     8 |
| 1406726400000 |           18 |    12 |
| 1406726400000 |           19 |    10 |
| 1406726400000 |           20 |     9 |
| 1406726400000 |           21 |     7 |
| 1406726400000 |           22 |     9 |
| 1406726400000 |           23 |     5 |
| 1406725800000 |           22 |    51 |
| 1406725800000 |           23 |     9 |

which can be mapped:

time - X axis
bucket_start - Y axis
count - Z axis (intensity of color, or place in color palette)

effectively resulting in heat map

[xinuc]

👍 for this. At least we can support histogram for influxdb, if the other backends cannot support it.

[aybabtme]

heatmaps would be awesome, very much needed

[mschenck]

👍

[Dieterbe]

+1 for both histogram for all data across entire timerange, as well as
+1 for histogram over time (buckets for small time intervals, rendered in heatmap)

!!! thanks :D

[Dieterbe]

for this. At least we can support histogram for influxdb, if the other backends cannot support it.

actually you can do histograms over time with graphite too. if you use for example statsd then you have your data in a metric per bin (see https://github.com/etsy/statsd/blob/master/docs/metric_types.md) so you have metric.bin_50, metric.bin_inf, so you could put that in a heatmap.
(note, i don't think many people actually use this statsd feature)

to get a single histogram for an entire timerange in graphite, we could do the binning in js :)

[torkelo]

@Dieterbe true, what I have never understood through is the percentile functions in graphite, never understood how they work or if they work. What would be nice is being able to use the summarize graphite function with a percentile function, like summarize(my.high.resolution.metric, '1h', "percentiles 0.9 0.95")

[biancalana]

+1

[jeromegit]

+1

[cambridgemike]

+1

[cypreess]

+1 for both

[fsauer65]

++ 1

also this would be an aswesome way to visualize a percentiles "histogram": http://hdrhistogram.github.io/HdrHistogram/plotFiles.html

[gsaray101]

+1

[vitillo]

+1

[jheusser]

+1

[MalteJ]

+1

[makapuf]

+1 !

[jmorlock]

+1

[jessetan]

What would be the preferred way to support histograms in Grafana?
The PR in #949 (which @bergquist mentioned may be eligible for inclusion pending merging in master and adding tests) can be ported to the latest version. I've gone so far as to verify it actually works, no tests yet and some minor UI issues. Since there is quite some overlap with the other graph code, this would make sense.
On the other hand, a histogram is a totally different type of graph and can not be mixed with existing graph types (lines, points, bars). So a separate plugin would be more logical then (similar to the piechart plugin). @mtanda already did work on this, but it is not compatible with the plugin API in 3.0

[XANi]

IMO the only case where it would make sense to have it in same "type" would be if you could say have "normal" over time data on one layer and histogram on other, to basically have histogram bars as bacground with over-time data in foreground. If that way is not supported it doesn't really make sense to put it in same place.

Especially that over time "histogram plugin" could be expanded to do "histogram over time"/heatmap visualisations

[jessetan]

A "histogram over time" / heatmap is possible and would make sense to put in the graph panel, but a "regular" histogram can not be combined with other graph types since the x-axis shows buckets, not time.

[mtanda]

Hi, I try to support Grafana 3.0.
I don't test it well, but it seems to work.
https://github.com/mtanda/grafana-panel-plugin-histogram

I extend GraphCtrl class, and add HistogramCtrl. So, some part of code is shared with graph panel.

[jessetan]

@mtanda great, this is easier than patching the code myself ;)
I'll file any issues I see to your repo, I don't think it is ready for grafana.net yet. But once it is, I think this issue can be closed.

[mtanda]

@jessetan Thanks! I'm waiting for your feedback.

[hgfischer]

This issue seems abandoned by the Grafana team.

[torkelo]

@hgfischer in what way? not sure we have promised this :)

[mtanda]

I'm working on heatmap panel too.
https://github.com/mtanda/grafana-heatmap-epoch-panel
Currently, it doesn't support InfluxDB/Elasticsearch histogram response.

If #3164 is implemented, the panel becomes more useful.

[johnou]

@mtanda would you please attach some sample screenshots of the histogram and heat map plugin panels?

[mtanda]

@johnou These are screenshots. I notice there are some bugs in query editor, but other part works well.

Histogram:

Heatmap:

[LuboVarga]

+1

[WIZARD-CXY]

+1

[cederigo]

+1

[cattt84]

+1

[GillesBIANNIC]

+1

[felixge]

@mtanda your heatmap plugin looks great. I hope to be able to try it out soon. Also +1 for something similar to be included in grafana core.

[torkelo]

histogram support added to main graph panel:

#8053

[Dieterbe]

Wow so the same old graph panel can now do histograms ? That's even better then a separate panel. Thanks Alex and torkel

[Jan-Axoom]

I still have a problem using this new feature. I want to see the distribution of for example the run times of a certain function. For that I connected an elasticsearch source and filled it with the desired data. The thing is that the new function aggregates over the timestamp when the data was written and not, what would be more logical, over the displayed data. With the current state I have to choose an interval for the group by clause, so that no data points are combined. Otherwise the shown distribution is completely misleading.

(I tested min / max (used in the pictures) / average as metric, all of them are equally bad)
The following pictures involve run times from 100ms to 1000ms with interval set to 1. 100ms (how it should look), 2. 1s and 3. 10s (auto)

Auto-interval makes no sense here because the inspected time interval (here last 24 hours) can be changed, what will result in a whole other graph as well.

[alexanderzobnin]

@Jan-Axoom could you please share your ES query? This mode just transform timeseries data to histogram. More source points make more precise result. But for ES and other datasources supporting histogram aggregation make more sense to use aggregated data like Heatmap panel does (#11087). We think about extracting histogram to the separate panel, so I think it would be a nice feature to have.

[mrdeveloperdude]

I am trying to use this with MySQL but there is no documentation. (http://docs.grafana.org/features/datasources/mysql/)

How can I write my SQL to work with this feature?

Please shed some light!