Parallel boost restart due to timeout can result in errors.

[jdcoats]

Boost is hanging up and leaving a process running even after completion. When i find it, I have to manually kill it. It will continue as long as I leave it.

~ # grep 16:16:3 /var/log/cacti.log
2021/04/09 16:16:34 - SYSTEM BOOST STATS: Time:15.72 ProcessNumber:2 RRDUpdates:2731838
2021/04/09 16:16:35 - SYSTEM BOOST STATS: Time:17.54 RRDUpdates:2731838
2021/04/09 16:16:35 - CMDPHP ERROR: A DB Row Failed!, Error: Table 'cacti.poller_output_boost_arch_1617999315' doesn't exist
2021/04/09 16:16:35 - CMDPHP SQL Backtrace:  (/poller_boost.php[216]:boost_output_rrd_data(), /poller_boost.php[573]:boost_process_local_data_ids(), /poller_boost.php[673]:db_fetch_assoc(), /lib/database.php[500]:db_fetch_assoc_prepared(), /lib/database.php[514]:db_execute_prepared())
2021/04/09 16:16:35 - CMDPHP ERROR: A DB Row Failed!, Error: Table 'cacti.poller_output_boost_arch_1617999315' doesn't exist
2021/04/09 16:16:35 - CMDPHP SQL Backtrace:  (/poller_boost.php[216]:boost_output_rrd_data(), /poller_boost.php[573]:boost_process_local_data_ids(), /poller_boost.php[673]:db_fetch_assoc(), /lib/database.php[500]:db_fetch_assoc_prepared(), /lib/database.php[514]:db_execute_prepared())
2021/04/09 16:16:35 - CMDPHP ERROR: A DB Row Failed!, Error: Table 'cacti.poller_output_boost_arch_1617999315' doesn't exist
2021/04/09 16:16:35 - CMDPHP SQL Backtrace:  (/poller_boost.php[216]:boost_output_rrd_data(), /poller_boost.php[573]:boost_process_local_data_ids(), /poller_boost.php[673]:db_fetch_assoc(), /lib/database.php[500]:db_fetch_assoc_prepared(), /lib/database.php[514]:db_execute_prepared())
.....
2021/04/09 17:24:33 - CMDPHP SQL Backtrace: (/poller_boost.php[216]:boost_output_rrd_data(), /poller_boost.php[573]:boost_process_local_data_ids(), /poller_boost.php[673]:db_fetch_assoc(), /lib/database.php[500]:db_fetch_assoc_prepared(), /lib/database.php[514]:db_execute_prepared())
2021/04/09 17:24:33 - CMDPHP ERROR: A DB Row Failed!, Error: Table 'cacti.poller_output_boost_arch_1617999315' doesn't exist 

~ # ps aux| grep boost
root     12528 76.1  0.1 114380 81048 ?        R    16:16  51:47 /usr/bin/php /var/www/localhost/htdocs/cacti/poller_boost.php --child=1
root     22250  0.0  0.0   8000  2372 pts/1    S+   17:24   0:00 grep --colour=auto boost
~ # kill -9 12528

[jdcoats]

I was running it at 8 processes/cpu's and i cut it back to 2. I will change to 1 and see if it continues.

[TheWitness]

So, my guess is that you changed the setting while boost was running. Let me take a look at the code. There might be something around that change that would cause boost to exit before everything was done, and then dropping the table while the legacy setting process was still busy.

[TheWitness]

I can possibly see how this could happen if you exceed the maximum runtime by 3x, boost will forcibly restart itself. Adding some logic to 'fix/workaround' that.

[TheWitness]

So, technically, you should not get the errors any longer, but you will have to check for warnings to ensure that you don't stack up a bunch of archive tables.

[jdcoats]

When it runs it get this now.

2021/04/10 12:40:17 - BOOST WARNING: Boost Poller terminated by user
--
2021/04/10 12:40:17 - BOOST WARNING: Killing Boost Child PID 29648 due to it exceeding 3x its maximum allowed runtime
2021/04/10 12:40:17 - BOOST WARNING: Boost Poller terminated by user
2021/04/10 12:40:17 - POLLER: Poller[Main Poller] PID[2116] ERROR: Process being killed due to timeout! (boost, master, 1, 29638)

No actual boost stats are displayed. Maxim run time is set to 40 minutes and it only runs for 40 seconds

[TheWitness]

So, I'm wondering who is starting poller_boost.php? Did you manually run?

[jdcoats]

not a manual run. The cacti schedule every 2 hours or 7000000 records