Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for box plot panels #3227

Open
Krinkle opened this issue Nov 12, 2015 · 47 comments
Open

Support for box plot panels #3227

Krinkle opened this issue Nov 12, 2015 · 47 comments

Comments

@Krinkle
Copy link
Contributor

Krinkle commented Nov 12, 2015

This may be a bit unconventional for typical time series display, but I think it could be very valuable.

Right now I emulate this by drawing a time series panel with line graphs with different percentiles. Together they provide information similar to that of a heat map.

screen shot

screen shot

And for better insight into data that has daily and weekly patterns, time series overlay help a lot to identify changes in data that always go up and down, but then you can compare it to how it was yesterday, and how it was the same day last week.
screen shot 2015-11-12 at 21 44 57

I think box plots would better serve some of these cases. It does have the obvious draw back of flattening the "time" part of "time series", but one could render multiple ones. E.g. One for last 24 hours, and one for the 24 hours before that, similar to Graphite summarize(24h).

Example box plot from https://codeascraft.com/2015/11/10/q3-2015-site-performance-report/
server-side

@spinus
Copy link

spinus commented Jan 20, 2016

+1 for this feature request

@recursionbane
Copy link

recursionbane commented May 5, 2016

+1

@jimrazmus
Copy link

+1 This would be a great addition.

@RobGruhl
Copy link

RobGruhl commented Jul 5, 2016

+1 would love this for showing perf test results

@roadfromatoz
Copy link

+1

7 similar comments
@seesiva
Copy link

seesiva commented Jul 18, 2016

+1

@bdschaap
Copy link

bdschaap commented Sep 1, 2016

+1

@jhorowitz
Copy link

+1

@yrodrigez
Copy link

+1

@lgmyrek
Copy link

lgmyrek commented Jan 1, 2017

+1

@hailpam
Copy link

hailpam commented Feb 6, 2017

+1

@devbrand
Copy link

devbrand commented Apr 6, 2017

+1

@torkelo
Copy link
Member

torkelo commented Apr 12, 2017

Would this be in addition to heatmap panel or would a heatmap panel better?

This new heatmap panel is available in nightly builds (in core Grafana)
image

We have also added a Histogram mode to the core Graph panel.

@elvarb
Copy link

elvarb commented Apr 12, 2017

I would think a box plot would require a new panel.

Stealing this information from Netuative
https://help.netuitive.com/Content/Reports/utilization_boxplot_report.htm?Highlight=boxplot

For example if you are running a virtual environment and want to see which vm's are over specced a dashboard showing memory usage of all vm's as boxplots would be extremely helpful.

image

@tcloaa
Copy link

tcloaa commented Jul 4, 2017

So is Grafana supporting box-plot now?

(I don't think so..)

@daniellee
Copy link
Contributor

@tcloaa no, there is no box-plot plugin/panel. It looks interesting but is not on the road map for this year.

@matclayton
Copy link

@torkelo is there a good example of using heatmaps with graphite (statsd histograms?), struggling to find a working setup, but your screenshot suggests its possible.

@paurullan
Copy link

@daniellee I am planning to do some hackaton this Easter and try to implement the box plot as a single-stat panel.
Is this the correct place to discuss about it or should I open a thread on the discuss https://community.grafana.com/c/development ?

If here is the correct way: (if not I will repeat the questions on discuss)

  1. How would you approach the multiple (5) values needed?
  2. Given that there is already a plotly plugin and plotly supports box plots, does it seem like a good way to expand the plots or better work against the single stat?
    https://grafana.com/plugins/natel-plotly-panel
    https://plot.ly/javascript/box-plots/

@daniellee
Copy link
Contributor

@paurullan here or community site is fine. Sorry I missed answering this - how did your hackathon go? I would say that it would be a lot faster to use the plotly box plot so I would start there and test how well it works.

If you are using plotly, then it looks like all you have to do it is to give a box plot the data. Your job would be map the aggregated data to a box plot and then figure out which options you would like to add (units, colors etc.).

Or are you planning on showing multiple box plots? Maybe a box plot per hour for example?

@paurullan
Copy link

@daniellee I worked on it and managed to load data on a boxplot. The styling was very complicated and did not get anywhere.

If anybody wants to work on it, the only thing to activate the box plot in the end was to add it on the plugin list:
https://github.com/NatelEnergy/grafana-plotly-panel/blob/master/src/module.ts#L29

      settings: {
        type: 'scatter',

@rmminusrslash
Copy link

👍

@matheusmota
Copy link

+1 for this request

1 similar comment
@llmuculusll
Copy link

+1 for this request

@yannlv
Copy link

yannlv commented Jul 23, 2019

+1 for this request, it may be very usefull for displaying perf test trend over runs, especially with a tool like gatling.io

@nhaouari
Copy link

+1 for this request.

@Sumeshks29
Copy link

+1 for this request

@franciscopaniskaseker
Copy link

Very useful feature. +1

@DiegoDantas001
Copy link

  • 1 for this feature.

@blackerking
Copy link

1 for this feature

@b3nj1
Copy link

b3nj1 commented Apr 1, 2021

I've approximated a boxplot using built-in options. It's a few steps, but works for me. I'm sharing here because this is where I stumbled upon when looking for how to do this...

image

Not sure the easiest way to share the queries and panel setups, but I'll paste them below. Generally speaking,

  • Queries
    • query options: set min interval to the grouping desired - 12h in example
    • queries: 1 query for each percentile required, set the aggregation function to min, max, percentile(10), ... (I'm using 10,90% lines in this example because that's what I wanted vs the standard box-plot.)
  • Panel: Visualization: Time series
  • Field: Style: bar
  • Overrides:
    • min/max: changed to points (a bar with fill to itself might be OK, couldn't find a way to reduce the bar width)
    • percentiles: changed to fill to the percentile below it (eg, 90% fills to 50%, while 10% fills to 10% since nothing is below it)
{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 8,
  "links": [],
  "panels": [
    {
      "datasource": null,
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "bars",
            "fillOpacity": 10,
            "gradientMode": "none",
            "hideFrom": {
              "graph": false,
              "legend": false,
              "tooltip": false
            },
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "never",
            "spanNulls": true
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "short"
        },
        "overrides": [
          {
            "matcher": {
              "id": "byFrameRefID",
              "options": "D"
            },
            "properties": [
              {
                "id": "custom.drawStyle",
                "value": "points"
              }
            ]
          },
          {
            "matcher": {
              "id": "byFrameRefID",
              "options": "E"
            },
            "properties": [
              {
                "id": "custom.drawStyle",
                "value": "points"
              }
            ]
          },
          {
            "matcher": {
              "id": "byFrameRefID",
              "options": "A"
            },
            "properties": [
              {
                "id": "custom.fillBelowTo",
                "value": "10% ptile"
              }
            ]
          },
          {
            "matcher": {
              "id": "byFrameRefID",
              "options": "B"
            },
            "properties": [
              {
                "id": "custom.fillBelowTo",
                "value": "10% ptile"
              }
            ]
          },
          {
            "matcher": {
              "id": "byFrameRefID",
              "options": "C"
            },
            "properties": [
              {
                "id": "custom.fillBelowTo",
                "value": "50% ptile"
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 16,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "interval": "12h",
      "options": {
        "graph": {},
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom"
        },
        "tooltipOptions": {
          "mode": "multi"
        }
      },
      "pluginVersion": "7.5.1",
      "targets": [
        {
          "alias": "min",
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "W",
          "orderByTime": "ASC",
          "policy": "default",
          "refId": "D",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "value"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "min"
              }
            ]
          ],
          "tags": [
            {
              "key": "entity_id",
              "operator": "=",
              "value": "envoy_current_energy_consumption"
            }
          ]
        },
        {
          "alias": "10% ptile",
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "measurement": "W",
          "orderByTime": "ASC",
          "policy": "default",
          "refId": "A",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "value"
                ],
                "type": "field"
              },
              {
                "params": [
                  "10"
                ],
                "type": "percentile"
              }
            ]
          ],
          "tags": [
            {
              "key": "entity_id",
              "operator": "=",
              "value": "envoy_current_energy_consumption"
            }
          ]
        },
        {
          "alias": "50% ptile",
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "W",
          "orderByTime": "ASC",
          "policy": "default",
          "refId": "B",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "value"
                ],
                "type": "field"
              },
              {
                "params": [
                  "50"
                ],
                "type": "percentile"
              }
            ]
          ],
          "tags": [
            {
              "key": "entity_id",
              "operator": "=",
              "value": "envoy_current_energy_consumption"
            }
          ]
        },
        {
          "alias": "90% ptile",
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "W",
          "orderByTime": "ASC",
          "policy": "default",
          "refId": "C",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "value"
                ],
                "type": "field"
              },
              {
                "params": [
                  "90"
                ],
                "type": "percentile"
              }
            ]
          ],
          "tags": [
            {
              "key": "entity_id",
              "operator": "=",
              "value": "envoy_current_energy_consumption"
            }
          ]
        },
        {
          "alias": "max",
          "groupBy": [
            {
              "params": [
                "$__interval"
              ],
              "type": "time"
            },
            {
              "params": [
                "null"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "W",
          "orderByTime": "ASC",
          "policy": "default",
          "refId": "E",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "value"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "max"
              }
            ]
          ],
          "tags": [
            {
              "key": "entity_id",
              "operator": "=",
              "value": "envoy_current_energy_consumption"
            }
          ]
        }
      ],
      "timeFrom": null,
      "title": "Power Consumption - Boxplots",
      "type": "timeseries"
    }
  ],
  "schemaVersion": 27,
  "style": "dark",
  "tags": [
    "power"
  ],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-7d",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "Power - Percentiles",
  "uid": "u02ThGlGz",
  "version": 20
}

@SoftwareApe
Copy link

Is there any plugin that supports this now?

@HogeBlekker
Copy link

Since Grafana now uses uPlot for its timeseries panel and uPlot support box plots, I suspect this feature should be within reach.

@leeoniya
Copy link
Contributor

yep, it's coming ;)

i'm currently working on a candlestick/financial panel and then migrating/rewriting the Heatmap panel to uPlot. box/whisker will likely be after that. maybe someone else will pick up the work sooner. eta is a few more point releases.

@HogeBlekker
Copy link

In the meantime, the current Candlestick panel can be used to mimic box plots. Beware, it's a bit hacky.

What works:

  • Lower, upper, q25 and q75 boundaries. In the example below, I've used 4 different queries to obtain these.
  • Adding the median as a dot (again, from a separate query).

What's missing:

  • The horizontal whiskers for the upper and lower boundaries
  • The horizontal line in the box for indicating the median

Here's an example of how we managed to put that together (with demo data):
https://demo.factry.io/d/LusUylfnz/box-whisker-with-candlestick?orgId=1

image

image

@ovidiubuligan
Copy link

Violin plot might be even better than box plot pannels :
https://en.wikipedia.org/wiki/Violin_plot
image

@pablodz
Copy link

pablodz commented Sep 13, 2022

Still relevant

@phutoan31299
Copy link

@HogeBlekker How to detect outliers in box plot? Thanks

@SoftwareApe
Copy link

@phutoan31299 the idea behind box plots is that median, lower quartile and upper quartile are stable against outliers. I.e. a big delta between max and upper quartile, and min and lower quartile, indicates outliers.

@yosiasz
Copy link

yosiasz commented Jun 7, 2023

this works with plotty panel grafana plugin

https://community.grafana.com/t/how-to-use-plotly-panel-in-grafana/87948/2

@tobiasfalk
Copy link

After searching if Grafana has support for box plots, I first found this issue and after some more search I found that a newer and still maintained version of Plotly(https://github.com/nline/nline-plotlyjs-panel) is able to do them and there is also a good explanation on how to: https://plotly.com/javascript/box-plots/

@yosiasz
Copy link

yosiasz commented Aug 26, 2023

@tobiasfalk check this one out

VolkovLabs/volkovlabs.io#484

@torkelo
Copy link
Member

torkelo commented Nov 23, 2023

candlestick supports this

@torkelo torkelo closed this as completed Nov 23, 2023
@torkelo torkelo reopened this Nov 23, 2023
@leeoniya
Copy link
Contributor

leeoniya commented Nov 23, 2023

i think candlestick doesnt support this well.

we should add logic to calc the stats over a defined window in the ui as well. also need to draw means/medians in the boxes as well as T-shaped box whiskers for quartiles and maybe draw outliers. it has some way to go, still.

@alexandergerber
Copy link

I am working on a plugin which uses plotly to create boxplots without using code. My current version looks like this:

image
image

At the moment, I am not sure if this might be viable for a public plugin and what kind of feature it would have to support.

Please let me know what you think about this approach

@Nasisadr
Copy link

I created box plots and wanted to share my experience here
image
image

To plot your data in Grafana using the Plotty Plot plugin and the provided script panel code, you need to ensure that your query is correctly formatted and that the variable names match those used in the script panel code. Here's how you can do it:
install plotty plot plugin in Grafana.
Make sure your query returns the necessary fields with the correct names.
` SELECT
bin(time, INTERVAL '$interval' MINUTE) AS time_interval,
MIN(measurement_value_double) AS lowerfence,
MAX(measurement_value_double) AS upperfence,
AVG(measurement_value_double) AS mean,
approx_percentile(measurement_value_double,0.5) AS median,
approx_percentile(measurement_value_double,0.25) AS q1,
approx_percentile(measurement_value_double,0.75) AS q3,
stddev(measurement_value_double) AS sd

FROM
"testTimestreamDB"."testtimestreamTable"
WHERE
series_number = '27'
AND customer_name = '$CustomerName'
AND CAST(day(time) AS VARCHAR) = '$day'
AND CAST(month(time) AS VARCHAR) = '$month'
AND CAST(YEAR(time) AS VARCHAR) = '$year'
GROUP BY
bin(time, INTERVAL '$interval' MINUTE)
**add this code to script pannel:** let jumbo = [];
const timeIntervalField = data.series[0].fields.find(field => field.name === 'time_interval');
const timeIntervals = timeIntervalField.values.buffer.map(interval => Date.parse(interval));

data.series.forEach((s) => {
let seriesData = {};
s.fields.forEach((field) => {
seriesData[field.name] = field.values.buffer;
});
seriesData["x"] = seriesData["time_interval"]
seriesData['type'] = "box";
jumbo.push(seriesData);
});

const layout = {
title: 'Motor Temperature Plot for Each $interval-Minute Interval',
xaxis: {
title: 'Time Interval',
},
yaxis: {
autorange: true,
showgrid: true,
title: 'Celsius',
zeroline: true,
zerolinecolor: 'rgb(255, 255, 255)',
},
margin: {
l: 50,
r: 50,
b: 20,
t: 50
}
};

return {
data: jumbo,
layout: layout,
};

`
Another problem that I had that might be someone else's problem was that I couldn't find the place to see the result of console.log to debug the code. The solution is very easy! just write click on Grafana plotty panel and press inspect and go to console tab!

@ianks
Copy link

ianks commented May 25, 2024

Wanted to chime in and say this is the one feature missing from Grafana for me. I spent hours trying to recreate it with candlesticks but ultimately failed. Sometimes it works OK for time series data, but brittle.

What I’d really love to see is a “stat” version of box plot, to compare percentiles of things (without time series). That would be incredibly useful.

@alexandergerber
Copy link

@ianks What exactly do you mean by "stat" version?

I still have a private boxplot plugin based on plotly boxplots. But since there was no real interest / support I lost interest working on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests