Plugins may not work correctly with Multi-Poller setups

[roninsakos]

Describe the bug

We have cacti 1.2.16 on linux env with a multipoller setup (6 remote +1 main ) and we decide to upgrade to 1.2.22.The main plugins that we use is reportit and weathermap. it seems that after the upgrade we face a long/lot of delays on every click that we do on the new environment. After reading the issues
#4611
#4611
#4615
#4615

from the above i understood that until version 1.2.20 the hook tables of the plugins from the main poller didn't sync on the remote pollers so the plugins always run only on the main.after 1.2.20 every poller run the plugins.it seems that on the above 2 plugins that we use (reportit ,weathermap) create unexpectedly long delays and they can not work correctly any more on a multi poller environment.

To Reproduce

we tested it with one main poller and one remote. when we install the plugins and we go to the graphs tab , the graphs take a long time to appear.also clicking on any cacti choice from "console" it seems that we have 2-3 sec delay. if we uninstall the plugins or disable the remote pollers the delays disappear . there are no error logs even on a debug log level.

Expected behavior

option for the remote pollers not to copy or not act like they are the main poller and run plugins.

Desktop (please complete the following information)

• OS centos 7

• chrome,firefox

Additional context

the plugin options on the INFO file
capabilities = online_view:0, online_mgmt:0, offline_view:0, offline_mgmt:0, remote_collect:0
seems not to have any effect

[thurban]

The sync/copy of files for plugins is configured using the "nosync" setting in the INFO file.

Here's an example to skip most of the file/directory copy to the remote pollers for the CereusReporting plugin:
nosync = archive,backup,tmp,*.pdf,setup.php,vendor,server,templates,translations,ReportEngines,queueScripts,modules,libs,include,images,dsstats_reports,docs,assets,*.php

[roninsakos]

How this action can help our case?isn't a file problem but a db/cacti problem and it has to do with the way cacti handles the plugins in a multipoller env after the 1.2.20 changes.To mention that we have tried the above options but we have also the cacti option resource sync disable.we tried also to delete he plugin folder from the remote poller but we can not avoid the sync between the main and the remote poller and whatever we do the tables are copied in the remote poller.(plugin_*).i am confident that it has to do with the sync because when we disable the remote poller with the plugins activated and installed everything is ok. Also it works fine in a standalone poller and it wotks fine in all versions prior 1.2.20

[thurban]

Well, check if the plugins exists on the remote poller and if there is a setup.php file on there. If yes, then it copied over the plugin.
You can prevent this behaviour with the nosync entry in the INFO file.

Obviously setting "resource sync" to disable should also have the same effect, just checking if the setting is being ignored or the sync is always done even if the "nosync" is set.

[roninsakos]

The nosync option has to do with files/folders.And yes it works.but have in mind that even with no plugin folder in the renote poller the plugin tables are sync and i can see the active/inactive plugins from the remote poller gui using the cacti plugin option with/or without a plugin folder( on the remote poller)

[thurban]

Yes, but it does not call the plugin hook if the files are not present, the cacti code prevents it and does not cause any performance issues.

Basically, it's a UI issue on the remote pollers, where the plugin page should hide any plugins that are not existing/working on the remote poller. But it does not cause any performance issues.

I had this behaviour with a Cacti setup previously, but that was mostly caused by hostname/dns resolution issues. Can you check that DNS resolution is ok or switch to IP addresses for the remote pollers ?

[roninsakos]

for being more precise i uploaded a video with the problem.in the browser i have two tabs .one with the main poller and one with the remote. in the remote the plugin folder is always empty.i have disable the resource sync.there is only one device on the main poller.(localhost) . when the plugin is active/installed when i go to the graphs tabs there is a delay of 3-4 sec on every graph.in a live env this delay is multiplied by the graphs that the device has. if i remove the remote poller or disable the plugin everything works fine and there is no delay. In the remote poller the plugin is visible without the plugin files to be present on the /cacti/plugin folder.one more thing that i saw is that the apache service take long time to restart when the problem occurs (without errors on cacti logs).from dns perspective everything looks fine.

cacti.mp4

[roninsakos]

While testing things i came across on this piece of code on /cacti/lib/poller.php

if ($class == 'all' || $class == 'plugins') {
       $data = db_fetch_assoc('SELECT * FROM plugin_config');
       replicate_out_table($rcnn_id, $data, 'plugin_config', $remote_poller_id);
       $data = db_fetch_assoc('SELECT * FROM plugin_hooks');
       replicate_out_table($rcnn_id, $data, 'plugin_hooks', $remote_poller_id);
       $data = db_fetch_assoc('SELECT * FROM plugin_realms');
       replicate_out_table($rcnn_id, $data, 'plugin_realms', $remote_poller_id);
}

if i comment it out ,it fix the delay/performance issue. Perhaps a check/flag code could be introduced here for which plugins we/must need to be replicated to the remote pollers.

[TheWitness]

Interesting that reportit shows up, but nothing else. Is that due to it being the only plugin replicated?

[roninsakos]

The plugins on the main poller are not installed/disabled except from reportit.if i activate/install them they will show up also on the remote without the cacti/plugin folder to be present on the remote poller.the performance issue appears with reportit,weathermap.i havent test the others.the above code that i post replicates all the plugin table.so i assume it will replicate all the installed plugins.

[TheWitness]

So, you have file replication disabled?

[roninsakos]

resource replication is disabled, yes.

[roninsakos]

hello, any update on that? do you have tested it to verify the performance issue? thnk you.

[TheWitness]

I've updated Intropage to work a bit better, but you have to show the log's for the other plugin errors.

[TheWitness]

Update to the latest lib/plugins.php and see if it makes a difference.

[roninsakos]

hello there are no errors on the plugins/logs. i will try the new file .

[roninsakos]

i just push the change to the test env but still there is delay when going to see graphs. ( i unistall/install again the plugins)

[TheWitness]

Just getting back from vacation.

[roninsakos]

Hello , any update?

[TheWitness]

Busy with the day job. Sorry. Weekends mostly and when I get free time at the moment.

[roninsakos]

reminder

[TheWitness]

Can you enable the file replication between your master Cacti server and the remotes?

[roninsakos]

Yes.no difference.same problem

[TheWitness]

@roninsakos, I'm back at it today. By any chance, are you available to go through the issue online? Do you have a lab setup with the problems? Sorry, stuff like this has to happen on the weekends. You can reach me at thewitness at cacti dot net.