Limit max data values of Grafana panels #3812
Conversation
|
This PR is brought by scale test. We expect the bug-fix PRs of scale test to be checked in in both Antrea and Theia repo. |
Codecov Report
@@ Coverage Diff @@
## main #3812 +/- ##
===========================================
- Coverage 64.38% 46.05% -18.33%
===========================================
Files 280 247 -33
Lines 39754 35914 -3840
===========================================
- Hits 25596 16541 -9055
- Misses 12163 17738 +5575
+ Partials 1995 1635 -360
Flags with carried forward coverage won't be shown. Click here to find out more.
|
heanlan
added
the
area/flow-visibility
build/yamls/flow-visibility/base/provisioning/dashboards/pod_to_pod_dashboard.json
Show resolved
Hide resolved
build/yamls/flow-visibility/base/provisioning/dashboards/pod_to_pod_dashboard.json
Show resolved
Hide resolved
docs/network-flow-visibility.md
Outdated
| - Also note that we set max number of values displayed on panels. For Sankey | ||
| diagrams, the limit is 50. For time-series line graphs, the limit is 100. For | ||
| pie charts, the limit is 25. The motivation is, when the input data is very | ||
| large, we want to keep the charts readable and avoid consuming too much time | ||
| and resources to render them. |
There was a problem hiding this comment.
The changes look good to me. I only have a question about the thresholds. Did we have a few tests to choose proper values? For "proper", I mean it will be enough for most small/middle size deployments, so that most customers won't be bothered to change the values.
There was a problem hiding this comment.
update: I've changed the limit for line graphs from 100 to 50. As the line graphs and sankey diagrams are displaying the same group of objects. We'd better make them consistent.
Given the limit, to render the dashboard, Google chrome GPU takes ~300-400MB memory, renderer takes ~100-200MB memory, which I think is manageable for most of the case.
One of the major use case I can think of changing the limit is: when users add global filters to a dashboard. The filter will be added on top of the query results. That means, the query returns 50 values, while the filter can reduce the result to a smaller value less than 50. In that situation, users may want to increase the query limit, in order to show more values with the filter applied.
The reason for choosing 50 is because it is almost the upper bound to make the graph readable and displayed normally. When I created 60 connections. The left sankey diagram is with limit 60. The right one is with limit 50. We can tell with limit 60. The links can be failed to show.
heanlan
force-pushed
the
limit-on-query
branch
2 times, most recently
from
01dd38c
to
b9f7d4f
Compare
docs/network-flow-visibility.md
Outdated
| ClickHouse SQL query for each individual panel. | ||
| ClickHouse and Grafana. | ||
|
|
||
| - Also note that we set max number of values displayed on panels. For Sankey |
There was a problem hiding this comment.
Also note that we limit the number of values displayed on panels.
| ClickHouse and Grafana. | ||
|
|
||
| - Also note that we set max number of values displayed on panels. For Sankey | ||
| diagrams, the limit is 50. For time-series line graphs, the limit is 50. For |
There was a problem hiding this comment.
I see a limit of 10,000 for some queries, but it's not mentioned here
docs/network-flow-visibility.md
Outdated
| If you want to stop filtering traffic by Namespace, or edit the panel limit, | ||
| you will need to edit the ClickHouse SQL query for each individual panel. Please | ||
| follow the [dashboards customization](#dashboards-customization) section for | ||
| more information. Especially, to edit the panel limit for pie charts, instead of |
There was a problem hiding this comment.
s/Especially/As a special case
docs/network-flow-visibility.md
Outdated
| you will need to edit the ClickHouse SQL query for each individual panel. Please | ||
| follow the [dashboards customization](#dashboards-customization) section for | ||
| more information. Especially, to edit the panel limit for pie charts, instead of | ||
| editing query, please follow the [doc](https://grafana.com/docs/grafana/latest/visualizations/pie-chart-panel/#limit) |
Add limit to sql queries to set the max number of values displayed on each panel. 1. keep the charts readable when the input data is very large 2. avoid taking too much time and resources to render the panels Signed-off-by: heanlan <hanlan@vmware.com>
|
/test-all |
Add limit to sql queries to set the max number of values displayed
on each panel.