When polling is disabled, boost may crash and creates many arch tables

[bmfmancini]

Hey Guys,

I noticed that either when you have polling disabled or no devices online residing on the main poller
the Cacti log displays these errors no SQL back trace errors

2021-09-06 13:25:02 - BOOST ERROR: Failed to retrieve archive table name
--
2021-09-06 13:25:01 - BOOST SVR WARNING: Detected Poller Boost Overrun, Possible Boost Poller Crash
2021-09-06 13:25:01 - SYSTEM THOLD POLLER STATS: TotalTime:0.131 TotalDevices:58 DownDevices:2 NewDownDevices:0 Threads:1 Thresholds:56
2021-09-06 13:25:01 - SYSTEM STATS: Time:0.0134 Method:disabled Processes:10 Threads:1 Hosts:53 HostsPerProcess:0 DataSources:92 RRDsProcessed:0
2021-09-06 13:25:01 - POLLER: Poller[Main Poller] PID[7542] WARNING: The Cacti Data Collector is currently disabled!
2021-09-06 13:24:02 - SYSTEM SYSLOG STATS: Time:0.09 Deletes:0 Incoming:0 Removes:0 XFers:0 Alerts:1 Alarms:0 Reports:0
2021-09-06 13:24:01 - BOOST ERROR: Failed to retrieve archive table name
2021-09-06 13:24:01 - BOOST SVR WARNING: Detected Poller Boost Overrun, Possible Boost Poller Crash
2021-09-06 13:24:01 - SYSTEM THOLD POLLER STATS: TotalTime:0.210 TotalDevices:58 DownDevices:2 NewDownDevices:0 Threads:1 Thresholds:56
2021-09-06 13:24:01 - SYSTEM STATS: Time:0.0103 Method:disabled Processes:10 Threads:1 Hosts:53 HostsPerProcess:0 DataSources:92 RRDsProcessed:0

There is no boost process running

[root@master1 ~]# ps -ef | grep -i  boost
root      7600  7586  0 13:27 pts/0    00:00:00 grep --color=auto boost
[root@master1 ~]# 

Also there is no poller_output_boost items

MariaDB [cacti]> select * from poller_output_boost;
Empty set (0.001 sec)

MariaDB [cacti]> 

in gui boost is showing in running state

MariaDB [cacti]> select * from poller_output_boost_processes;
Empty set (0.002 sec)

MariaDB [cacti]> 

Boost is also creating tons of arch tables

MariaDB [cacti]> show tables like '%boost%';
+-------------------------------------+
| Tables_in_cacti (%boost%)           |
+-------------------------------------+
| poller_output_boost                 |
| poller_output_boost_arch_1630939981 |
| poller_output_boost_arch_1630940041 |
| poller_output_boost_arch_1630940101 |
| poller_output_boost_arch_1630940161 |
| poller_output_boost_arch_1630940221 |
| poller_output_boost_arch_1630940281 |
| poller_output_boost_arch_1630940341 |
| poller_output_boost_arch_1630940401 |
| poller_output_boost_arch_1630940461 |
| poller_output_boost_arch_1630940521 |
| poller_output_boost_arch_1630940581 |
| poller_output_boost_arch_1630940641 |
| poller_output_boost_arch_1630940702 |
| poller_output_boost_arch_1630940761 |
| poller_output_boost_arch_1630940821 |
| poller_output_boost_arch_1630940881 |
| poller_output_boost_arch_1630940941 |
| poller_output_boost_arch_1630941001 |
| poller_output_boost_arch_1630941061 |
| poller_output_boost_arch_1630941121 |
| poller_output_boost_arch_1630941181 |
| poller_output_boost_arch_1630941241 |
| poller_output_boost_arch_1630941301 |
| poller_output_boost_arch_1630941361 |
| poller_output_boost_arch_1630941421 |
| poller_output_boost_arch_1630941481 |
| poller_output_boost_arch_1630941542 |
| poller_output_boost_arch_1630941601 |
| poller_output_boost_arch_1630941661 |
| poller_output_boost_arch_1630941721 |
| poller_output_boost_arch_1630941781 |
| poller_output_boost_arch_1630941841 |
| poller_output_boost_arch_1630941901 |
| poller_output_boost_arch_1630941961 |
| poller_output_boost_arch_1630942022 |
| poller_output_boost_arch_1630942081 |
| poller_output_boost_arch_1630942141 |
| poller_output_boost_arch_1630942201 |
| poller_output_boost_arch_1630942262 |
| poller_output_boost_arch_1630942321 |
| poller_output_boost_arch_1630942381 |
| poller_output_boost_arch_1630942441 |
| poller_output_boost_arch_1630942502 |
| poller_output_boost_arch_1630942561 |
| poller_output_boost_arch_1630942621 |
| poller_output_boost_arch_1630942681 |
| poller_output_boost_arch_1630942741 |
| poller_output_boost_arch_1630942801 |
| poller_output_boost_arch_1630942861 |
| poller_output_boost_arch_1630942921 |
| poller_output_boost_arch_1630942981 |
| poller_output_boost_arch_1630943041 |
| poller_output_boost_arch_1630943101 |
| poller_output_boost_arch_1630943161 |
| poller_output_boost_arch_1630943221 |
| poller_output_boost_arch_1630943281 |
| poller_output_boost_arch_1630943341 |
| poller_output_boost_arch_1630943401 |
| poller_output_boost_arch_1630943461 |
| poller_output_boost_arch_1630943521 |
| poller_output_boost_arch_1630943581 |
| poller_output_boost_arch_1630943641 |
| poller_output_boost_arch_1630943702 |
| poller_output_boost_arch_1630943761 |
| poller_output_boost_arch_1630943821 |
| poller_output_boost_arch_1630943881 |
| poller_output_boost_arch_1630943941 |
| poller_output_boost_arch_1630944001 |
| poller_output_boost_arch_1630944061 |
| poller_output_boost_arch_1630944121 |
| poller_output_boost_arch_1630944181 |
| poller_output_boost_arch_1630944241 |
| poller_output_boost_arch_1630944302 |
| poller_output_boost_arch_1630944361 |
| poller_output_boost_arch_1630944422 |
| poller_output_boost_arch_1630944482 |
| poller_output_boost_arch_1630944541 |
| poller_output_boost_arch_1630944601 |
| poller_output_boost_arch_1630944662 |
| poller_output_boost_arch_1630944721 |
| poller_output_boost_arch_1630944782 |
| poller_output_boost_arch_1630944841 |
| poller_output_boost_arch_1630944901 |
| poller_output_boost_arch_1630944962 |
| poller_output_boost_arch_1630945021 |
| poller_output_boost_arch_1630945081 |
| poller_output_boost_arch_1630945141 |
| poller_output_boost_arch_1630945202 |
| poller_output_boost_arch_1630945261 |
| poller_output_boost_arch_1630945321 |
| poller_output_boost_arch_1630945381 |
| poller_output_boost_arch_1630945441 |
| poller_output_boost_arch_1630945501 |
| poller_output_boost_arch_1630945561 |
| poller_output_boost_arch_1630945621 |
| poller_output_boost_arch_1630945681 |
| poller_output_boost_arch_1630945741 |
| poller_output_boost_arch_1630945801 |
| poller_output_boost_arch_1630945862 |
| poller_output_boost_arch_1630945921 |
| poller_output_boost_arch_1630945981 |
| poller_output_boost_arch_1630946041 |
| poller_output_boost_arch_1630946101 |
| poller_output_boost_arch_1630946161 |
| poller_output_boost_arch_1630946221 |
| poller_output_boost_arch_1630946281 |
| poller_output_boost_arch_1630946341 |
| poller_output_boost_arch_1630946402 |
| poller_output_boost_arch_1630946461 |
| poller_output_boost_arch_1630946522 |
| poller_output_boost_arch_1630946581 |
| poller_output_boost_arch_1630946641 |
| poller_output_boost_arch_1630946701 |
| poller_output_boost_arch_1630946762 |
| poller_output_boost_arch_1630946821 |
| poller_output_boost_arch_1630946881 |
| poller_output_boost_arch_1630946942 |
| poller_output_boost_arch_1630947001 |
| poller_output_boost_arch_1630947061 |
| poller_output_boost_arch_1630947121 |
| poller_output_boost_arch_1630947181 |
| poller_output_boost_arch_1630947241 |
| poller_output_boost_arch_1630947301 |
| poller_output_boost_arch_1630947362 |
| poller_output_boost_arch_1630947421 |
| poller_output_boost_arch_1630947481 |
| poller_output_boost_arch_1630947541 |
| poller_output_boost_arch_1630947601 |
| poller_output_boost_arch_1630947661 |
| poller_output_boost_arch_1630947722 |
| poller_output_boost_arch_1630947781 |
| poller_output_boost_arch_1630947841 |
| poller_output_boost_arch_1630947901 |
| poller_output_boost_arch_1630947961 |
| poller_output_boost_arch_1630948021 |
| poller_output_boost_arch_1630948081 |
| poller_output_boost_arch_1630948141 |
| poller_output_boost_arch_1630948201 |
| poller_output_boost_arch_1630948261 |
| poller_output_boost_arch_1630948321 |
| poller_output_boost_arch_1630948381 |
| poller_output_boost_arch_1630948441 |
| poller_output_boost_arch_1630948501 |
| poller_output_boost_arch_1630948561 |
| poller_output_boost_arch_1630948621 |
| poller_output_boost_arch_1630948681 |
| poller_output_boost_arch_1630948741 |
| poller_output_boost_arch_1630948801 |
| poller_output_boost_arch_1630948861 |
| poller_output_boost_arch_1630948921 |
| poller_output_boost_arch_1630948981 |
| poller_output_boost_arch_1630949041 |
| poller_output_boost_arch_1630949101 |
| poller_output_boost_arch_1630949162 |
| poller_output_boost_arch_1630949222 |
| poller_output_boost_arch_1630949282 |
| poller_output_boost_arch_1630949341 |
| poller_output_boost_arch_1630949402 |
| poller_output_boost_local_data_ids  |
| poller_output_boost_processes       |
+-------------------------------------+
161 rows in set (0.004 sec)

[bmfmancini]

Results when trying to force boost

[root@master1 cacti]# php poller_boost.php --force --debug
DEBUG: Checking if Boost is ready to run.
DEBUG: Last Runtime was 2021-09-06 09:53:01 (1630936381).
DEBUG: Next Runtime is 2021-09-06 10:53:01 (1630939981).
DEBUG: Records Found:0, Max Threshold:1000000.
DEBUG: Time to Run Boost, Force Run is true!
DEBUG: Parallel Process Setup Begins.
[root@master1 cacti]# 

[bmfmancini]

After enabling polling

[root@master1 cacti]# php poller_boost.php --force --debug
DEBUG: Checking if Boost is ready to run.
DEBUG: Last Runtime was 2021-09-06 09:53:01 (1630936381).
DEBUG: Next Runtime is 2021-09-06 10:53:01 (1630939981).
DEBUG: Records Found:123, Max Threshold:1000000.
DEBUG: Time to Run Boost, Force Run is true!
DEBUG: Parallel Process Setup Begins.
DEBUG: Data Sources:90, Concurrent Processes:1
DEBUG: Parallel Process Setup Complete.  Ready to spawn children.
DEBUG: About to launch 1 processes.
DEBUG: Launching Boost Process Number 1
2021-09-06 13:37:50 - POLLER: Poller[1] PID[8056] DEBUG: About to Spawn a Remote Process [CMD: /usr/bin/php, ARGS: /var/www/html/cacti/poller_boost.php --child=1 --debug]
2021-09-06 13:37:52 - SYSTEM BOOST STATS: Time:2.13 RRDUpdates:123

MariaDB [cacti]> show tables like '%boost%';
+------------------------------------+
| Tables_in_cacti (%boost%)          |
+------------------------------------+
| poller_output_boost                |
| poller_output_boost_local_data_ids |
| poller_output_boost_processes      |
+------------------------------------+
3 rows in set (0.001 sec)

[jdcoats]

I've had this same problem before, let me see if I can track down how it was resolved.

[jdcoats]

My issue surfaced when modifying boost settings during a boost run. I had to clean it all up manually. Different issue in your case but the same result.

[bmfmancini]

Interesting In my case this is a fresh lab install no modifications I had polling disabled to prevent double polling from my lab instance this was to test out the new thold code

…

On Mon., Sep. 6, 2021, 16:19 jdcoats, ***@***.***> wrote: My issue surfaced when modifying boost settings during a boost run. I had to clean it all up manually. Different issue in your case but the same result. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4391 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGEXTC4TXFQ2H66235TTWTUAUO3XANCNFSM5DQ2ZYZQ> .

[TheWitness]

This is partially resolved. I don't know why the archive tables are stacking up, so I'm marking resolved, but there may be another commit on this.

[TheWitness]

Okay, looking at @bmfmancini comments, we should be okay to mark this one resolved.