Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poor performance showing a device's graphs on a tree #2543

Closed
sysres-dev opened this issue Mar 19, 2019 · 3 comments
Closed

Poor performance showing a device's graphs on a tree #2543

sysres-dev opened this issue Mar 19, 2019 · 3 comments
Labels
enhancement General tag for an enhancement performance Performance related affects big sites
Milestone

Comments

@sysres-dev
Copy link
Contributor

Describe the bug
Displaying graphs in /graph_view.php?action=tree_content page is very slow with 26900 Devices and 458980 graphs.
This is due to a for loop in lib/html_tree.php, in the get_host_graph_list() function which produces 2 SQL request for each graph_template.

In our case this for loop iterates 900 times resulting in 1800 queries being executed, cumulating to an average of 12 seconds for the http request to complete.

Here is the extract of the code in lib/html_tree.php related to the issue (start at line 1409):

    if (cacti_sizeof($final_templates)) {
            foreach ($final_templates as $graph_template) {
                    $sql_where = '';

                    if (get_request_var('rfilter') != '') {
                            $sql_where = " (gtg.title_cache RLIKE '" . get_request_var('rfilter') . "')";
                    }

                    if ($host_id > 0) {
                            $sql_where .= ($sql_where != '' ? ' AND ':'') . 'gl.host_id=' . $host_id;
                    }

                    $sql_where .= ($sql_where != '' ? ' AND ':'') . 'gl.graph_template_id=' . $graph_template['id'];

                    $graphs = get_allowed_graphs($sql_where);
                    $agg    = get_allowed_aggregate_graphs($sql_where);
                    $graphs = array_merge($graphs, $agg);

                    /* let's sort the graphs naturally */
                    usort($graphs, 'naturally_sort_graphs');

                    if (cacti_sizeof($graphs)) {
                            foreach ($graphs as $graph) {
                                    $graph['graph_template_name'] = $graph_template['name'];
                                    array_push($graph_list, $graph);
                            }
                    }
            }
    }

These queries are executed in:

  • lib/auth.php, get_allowed_graphs();
  • lib/auth.php, get_allowed_aggregate_graphs();

To Reproduce
Performance issue is due vast amount of graphs, devices and graph_templates in database resulting on many SQL query being executed.

  1. Go to 'Graphs' tab
  2. Click on a host in tree view mode to display its graphs

Expected behavior
Decent loading time of graph_view.php when displaying list of graphs.

Screenshots
loading_time

Additional context

Technical Support [Résumé]
Informations générales
Date | Tue, 19 Mar 2019 15:56:54 +0100
Cacti Version | 1.2.3
Cacti OS | unix
RRDtool Version Configuré | 1.4.0+
RRDtool Version Trouvé | 1.4.8
Équipements | 26425
Graphiques | 445487

Note
I have a fix ready to be evaluated that I will present in a PR soon.

@sysres-dev
Copy link
Contributor Author

I've done the following PR regarding this issue : #2544.

@netniV netniV added enhancement General tag for an enhancement performance Performance related affects big sites labels Mar 19, 2019
@netniV netniV added this to the v1.2.3 milestone Mar 19, 2019
netniV pushed a commit that referenced this issue Mar 21, 2019
Related to #2543.
Reduces the number of SQL queries executed in graph_view's tree_content action by populating an array with graph_template_ids in order to have just two query using IN statement instead of 2 queries by  graph_template_id.

Request completion time before commit : 12s
![loading_time](https://user-images.githubusercontent.com/45565495/54622351-bf1ed080-4a69-11e9-83c9-1f25fb01d9ae.png)

Request completion time after commit : 3s
![loading_time_post_commit](https://user-images.githubusercontent.com/45565495/54622377-cc3bbf80-4a69-11e9-81d7-6afe79d8b52e.jpg)
@sysres-dev
Copy link
Contributor Author

Thank you, I will run a check and close the issue.

@netniV
Copy link
Member

netniV commented Mar 22, 2019

Thanks for your contriubtion @sysres-dev

@netniV netniV changed the title The graph_view page takes a long time to complete when displaying a host's graphs Poor performance showing a device's graphs on a tree Mar 30, 2019
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement General tag for an enhancement performance Performance related affects big sites
Projects
None yet
Development

No branches or pull requests

2 participants