Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When poller overruns the script server can throw errors upon shutdown #3908

Closed
xmacan opened this issue Nov 6, 2020 · 18 comments
Closed

When poller overruns the script server can throw errors upon shutdown #3908

xmacan opened this issue Nov 6, 2020 · 18 comments
Labels
bug Undesired behaviour resolved A fixed issue
Milestone

Comments

@xmacan
Copy link
Member

xmacan commented Nov 6, 2020

Poller method is cmd. Selective Device Debug with one host id. ss_esxi_vhost return correct value and graph is ok.
In log is:
2020-11-06 16:10:54 - POLLER: Poller[1] Device[220] DS[3213] SERVER: /usr/local/share/cacti/scripts/ss_esxi_vhosts.php ss_esxi_vhosts '192.168.250.97' 'kaktus' '161' '1000', output: vh:9 vh_state:8 tools_run:7 tools_nrun:1 tools_ninst:1
2020-11-06 16:11:00 - ERROR PHP NOTICE: fputs(): write of 53 bytes failed with errno=32 Broken pipe in file: /usr/local/share/cacti/script_server.php on line: 175
2020-11-06 16:11:00 - CMDPHP PHP ERROR NOTICE Backtrace: (/script_server.php[175]:fputs(), CactiErrorHandler())
2020-11-06 16:11:02 - SYSTEM STATS: Time:61.6975 Method:cmd.php Processes:1 Threads:0 Hosts:88 HostsPerProcess:88 DataSources:5028 RRDsProcessed:2057
2020-11-06 16:11:02 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyDeviceFailedPoll (CACTI-MIB), severity: medium

Cacti 1.2.14
PHP 7.4.11
FreeBSD 12.1

@xmacan xmacan added bug Undesired behaviour unverified Some days we don't have a clue labels Nov 6, 2020
@xmacan
Copy link
Member Author

xmacan commented Nov 6, 2020

The same error without selective debug, sorry for inaccuracy

@netniV
Copy link
Member

netniV commented Nov 6, 2020

This may be related to your other issue #3909 and seems like something is killing the connection or interfering with it?

@xmacan xmacan changed the title cmd poller script_server error if selective device debug enabled cmd poller script_server error ~~if selective device debug enabled~~ Nov 6, 2020
@xmacan xmacan changed the title cmd poller script_server error ~~if selective device debug enabled~~ cmd poller script_server error Nov 6, 2020
@xmacan
Copy link
Member Author

xmacan commented Nov 6, 2020

yes, it is the same installation. I cannot find what is wrong. Manual run is without problem, no warning or errors:
[root@cacti /usr/local/share/cacti]# php script_server.php
PHP Script Server has Started - Parent is cmd
/usr/local/share/cacti/scripts/ss_esxi_vhosts.php ss_esxi_vhosts '172.16.2.238' 'kaktus' '161' '2000'
vh:0 vh_state:0 tools_run:0 tools_nrun:0 tools_ninst:0
quit
PHP Script Server Shutdown request received, exiting

@xmacan
Copy link
Member Author

xmacan commented Nov 6, 2020

I made another error in another php script (division by 0). The same result
2020-11-06 20:21:01 - ERROR PHP NOTICE: fputs(): write of 53 bytes failed with errno=32 Broken pipe in file: /usr/local/share/cacti/script_server.php on line: 175
2020-11-06 20:21:01 - CMDPHP PHP ERROR NOTICE Backtrace: (/script_server.php[175]:fputs(), CactiErrorHandler())

@netniV You are right, it is connected with #3909 . It damage different data but the behavior is the same.

@TheWitness
Copy link
Member

Did you figure this one out too, or was it really the same issue?

@TheWitness
Copy link
Member

So, that fputs is spine basically going away. Could it be that it's segfaulting? Check the Cacti Standard Error log for segfaults.

@xmacan
Copy link
Member Author

xmacan commented Nov 8, 2020

I will try it tomorrow. I want to try disable all my own scripts. It could still be anz problem in my scripts. I'll let you know as soon as possible

@TheWitness
Copy link
Member

Any update?

@TheWitness
Copy link
Member

Not that I'm inpatient, but I'm pretty sure I can close this one.

@xmacan
Copy link
Member Author

xmacan commented Nov 23, 2020

Sorry, I was busy. I tried this now and it is strange - I cannot reproduce this!

@mcncacti
Copy link

mcncacti commented Dec 4, 2020

So, if you have an error in your script will script_server.php throw error messages in the log?
I ask cause I have a script that if I run from command line it works - if I try to run it in script_server.php it just hangs forever.

@xmacan
Copy link
Member Author

xmacan commented Dec 4, 2020

I cannot say it. I had errors in log but now I cannot reproduce it. I tried to make the same mistake and the error did not occur

@mcncacti
Copy link

mcncacti commented Dec 4, 2020

I guess I was trying to figure out if script_server.php has error reporting in the cacti.log or if there is a problem with it / or my script.

@TheWitness
Copy link
Member

Forums...

@mcncacti
Copy link

mcncacti commented Dec 4, 2020

I looked - searching led me here.
If I find something that applies I will post back, so others can use the fix (if I find)

@click0
Copy link
Contributor

click0 commented Feb 6, 2021

Up.

@TheWitness
Copy link
Member

This can happen when the pipe is broken, in other words, the script_server.php crashed. Therefore, I'm going to make a change to output that when called from the command line for debugging.

@TheWitness TheWitness changed the title cmd poller script_server error When poller overruns the script server can throw errors upon shutdown Feb 7, 2021
@TheWitness TheWitness added resolved A fixed issue and removed unverified Some days we don't have a clue labels Feb 7, 2021
@TheWitness TheWitness added this to the v1.2.17 milestone Feb 7, 2021
@TheWitness
Copy link
Member

Script server has been updated. This will be in 1.2.17. I suggest moving to spine in any case.

@github-actions github-actions bot locked and limited conversation to collaborators May 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Undesired behaviour resolved A fixed issue
Projects
None yet
Development

No branches or pull requests

5 participants