Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Query reindexing leads gaps in Graphs #2372

Closed
RadekHvizdos opened this issue Feb 4, 2019 · 15 comments
Closed

Data Query reindexing leads gaps in Graphs #2372

RadekHvizdos opened this issue Feb 4, 2019 · 15 comments
Labels
blocker Blocking release bug Undesired behaviour resolved A fixed issue

Comments

@RadekHvizdos
Copy link

Hello,

I have just updated my 1.1.38 installation to 1.2.1, and several graphs from Net-SNMP (Available Disk Space) stopped updating with new data. It seems like dskPath Index Type does not work as it used to.

Here is my configuration:

ds_config

And these messages are logged, notice the OID is malformed:
ds_log

Thanks

@cigamit
Copy link
Member

cigamit commented Feb 4, 2019

When you edit the Data Query, what is it using the the XML file path? If it's using net-snmp_disk.xml, change the index order type to put the diskIndex first in the list, and then re-index the queries, and everything will return to normal. If it's host_disk.xml, we need to know. Either way, this should not have happened. It was an untested path that we will have to work around.

@cigamit cigamit added bug Undesired behaviour blocker Blocking release labels Feb 5, 2019
@RadekHvizdos
Copy link
Author

Hi @cigamit,

It is indeed using net-snmp_disk.xml, I have made the following change:

diff /usr/share/cacti/resource/snmp_queries/net-snmp_disk.xml /usr/share/cacti/resource/snmp_queries/net-snmp_disk.xml.bak
4c4
<       <index_order>dskIndex:dskDevice</index_order>
---
>       <index_order>dskDevice:dskIndex</index_order>

And performed a reindex (how I think it is done anyways):

 php /usr/share/cacti/cli/poller_reindex_hosts.php --id=all
WARNING: Do not interrupt this script.  Reindexing can take quite some time..

Now I am only getting the warnings, but no info (white) messages:
image

Am I doing this right?

:)

@cigamit
Copy link
Member

cigamit commented Feb 5, 2019

Wait, diskIindex was supposed to be first. However, I found the source of the issue. I have to do a little testing before committing. Wished we would have found this during the beta.

cigamit added a commit that referenced this issue Feb 5, 2019
SNMP DS with dskPath Index Type does not work
@cigamit
Copy link
Member

cigamit commented Feb 5, 2019

Okay, update lib/data_query.php and then re-index. Hopefully this properly re-maps the indexes. To force the re-mapping, you will have to reverse the order so that Cacti does a re-map through, then you can reverse it to the way you had it.

@jdcoats
Copy link

jdcoats commented Feb 5, 2019

2019/02/05 08:41:31 - CMDPHP SQL Backtrace: (/cli/poller_reindex_hosts.php[141]:run_data_query(), /lib/data_query.php[142]:query_check_suitable(), /lib/data_query.php[416]:db_execute_prepared())
2019/02/05 08:41:31 - CMDPHP ERROR: A DB Exec Failed!, Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE host_id = '1' AND snmp_query_id = NULL' at line 1
2019/02/05 08:41:31 - CMDPHP PHP ERROR NOTICE Backtrace: (/cli/poller_reindex_hosts.php[141]:run_data_query(), /lib/data_query.php[142]:query_check_suitable(), /lib/data_query.php[416]:CactiErrorHandler())
2019/02/05 08:41:31 - ERROR PHP NOTICE: Undefined variable: data_query_id in file: /var/www/localhost/htdocs/cacti/lib/data_query.php on line: 416
2019/02/05 08:41:31 - CMDPHP PHP ERROR NOTICE Backtrace: (/cli/poller_reindex_hosts.php[141]:run_data_query(), /lib/data_query.php[134]:update_data_query_sort_cache(), /lib/data_query.php[1908]:get_ordered_index_type_list(), /lib/data_query.php[1772]:CactiErrorHandler())
2019/02/05 08:41:31 - ERROR PHP NOTICE: Undefined index: index_order in file: /var/www/localhost/htdocs/cacti/lib/data_query.php on line: 1772

cigamit added a commit that referenced this issue Feb 5, 2019
@cigamit
Copy link
Member

cigamit commented Feb 5, 2019

Updated again.

@cigamit
Copy link
Member

cigamit commented Feb 8, 2019

@jdcoats, I know you are busy, and chance you can get us feedback? We have a hope and a prayer of getting a release out this weekend. Only a few issues remain to be confirmed.

@jdcoats
Copy link

jdcoats commented Feb 8, 2019

I installed a clean version of the repo pulled on the morning 2/6/2019. It is labeled 1.2.2 and spine is still 1.2.1 but everything appears to be working at this time. I have almost 1800 devices added back. I still need to get the interface operational status query added without overwriting the current template settings which differ from the ones provided on the forums years ago. Mactrack still doesn't work. But the feedback about cacti itself is it appears everything is good.

@cigamit
Copy link
Member

cigamit commented Feb 8, 2019

Thanks for the update. Look forward to seeing what's new for MacTrack. One thing at a time.

@jdcoats
Copy link

jdcoats commented Feb 8, 2019

I just sync'd up with current. Ill let you know if I notice anything. Thanks!

@jdcoats
Copy link

jdcoats commented Feb 8, 2019

okay these are the only things that I noticed out of place in the logs after working at it all day. Happened a few times.

2019/02/08 14:23:51 - CMDPHP SQL Backtrace: (/host.php[64]:form_actions(), /host.php[269]:api_device_remove_multi(), /lib/api_device.php[194]:api_delete_graphs(), /lib/api_graph.php[73]:db_fetch_assoc(), /lib/database.php[450]:db_fetch_assoc_prepared(), /lib/database.php[464]:db_execute_prepared())
2019/02/08 14:23:51 - CMDPHP ERROR: A DB Row Failed!, Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AND gti.local_graph_id IS NULL AND gti.local_graph_id NOT IN(SELECT local_graph_' at line 1
2019/02/08 14:23:51 - CMDPHP SQL Backtrace: (/host.php[64]:form_actions(), /host.php[269]:api_device_remove_multi(), /lib/api_device.php[194]:api_delete_graphs(), /lib/api_graph.php[55]:db_fetch_assoc(), /lib/database.php[450]:db_fetch_assoc_prepared(), /lib/database.php[464]:db_execute_prepared())
2019/02/08 14:23:51 - CMDPHP ERROR: A DB Row Failed!, Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
2019/02/08 14:23:51 - CMDPHP SQL Backtrace: (/host.php[64]:form_actions(), /host.php[269]:api_device_remove_multi(), /lib/api_device.php[194]:api_delete_graphs(), /lib/api_graph.php[37]:db_fetch_assoc(), /lib/database.php[450]:db_fetch_assoc_prepared(), /lib/database.php[464]:db_execute_prepared())
2019/02/08 14:23:51 - CMDPHP ERROR: A DB Row Failed!, Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AND gti.local_graph_id NOT IN(SELECT local_graph_id FROM aggregate_graphs) AND d' at line 1

and this...

2019/02/08 16:11:25 - CMDPHP PHP ERROR WARNING Backtrace: (/graphs.php[123]:graph_management(), /graphs.php[2065]:filter_value(), /lib/functions.php[59]:preg_replace(), CactiErrorHandler())
2019/02/08 16:11:25 - ERROR PHP WARNING: preg_replace(): Unknown modifier ')' in file: /var/www/localhost/htdocs/cacti/lib/functions.php on line: 59
2019/02/08 16:11:25 - CMDPHP PHP ERROR WARNING Backtrace: (/graphs.php[123]:graph_management(), /graphs.php[2064]:filter_value(), /lib/functions.php[59]:preg_replace(), CactiErrorHandler())
2019/02/08 16:11:25 - ERROR PHP WARNING: preg_replace(): Unknown modifier ')' in file: /var/www/localhost/htdocs/cacti/lib/functions.php on line: 59

@cigamit
Copy link
Member

cigamit commented Feb 9, 2019

What were you searching for in your filter?

cigamit added a commit that referenced this issue Feb 9, 2019
As I can not find an issue here except potentially for a to the remove
function with an empty array.  So, I will add those checks now.
@cigamit
Copy link
Member

cigamit commented Feb 9, 2019

I made a check to short circuit the API call if there is an empty error. Hopefully, that is the reason for the errors.

@cigamit cigamit added the resolved A fixed issue label Feb 9, 2019
@cigamit
Copy link
Member

cigamit commented Feb 9, 2019

Resolved now.

@cigamit cigamit closed this as completed Feb 9, 2019
@RadekHvizdos
Copy link
Author

Thank you, this has fixed the issue.

@cigamit cigamit changed the title [1.2.1] SNMP DS with dskPath Index Type does not work Flaws in new Data Query re-index verification logic leads gaps in Graphs Feb 24, 2019
@netniV netniV changed the title Flaws in new Data Query re-index verification logic leads gaps in Graphs Data Query reindexing leads gaps in Graphs Feb 24, 2019
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
blocker Blocking release bug Undesired behaviour resolved A fixed issue
Projects
None yet
Development

No branches or pull requests

3 participants