Ubuntu 16.04 multiple instaces issue

[pierluigiconti]

Hello
we have been using check_vmware_api on ubuntu 14 without any problem but, we recently upgraded to ubuntu 16.04, and we're having a concurrency problem.
Infact all nagios checks aren't working correctly anymore.
After doing some tests we noticed that the problem is maybe related to a concurrency execution.
To do this tests we just executed three simultaneous instances of check_vmware_api and we got the issue: the script hangs.

We have runned this tests with vsphere SDK version 5.1 6.0 and 6.5.

Ubuntu's perl version is 5.22.1

Thank you,

[rhagman]

Hi!
Which other nagios checks aren't working? When the script hangs does it time out or how does the hang show? It works if you execute two simultaneous instances of check_vmware_api but hangs with three? The check is quite resource hungry so if you are testing this in a small vm it might give you some problems.
I'll see if I can get an ubuntu environment up and running to see if I can reproduce the problem.

[pierluigiconti]

Hi,

no other nagios aren't working, only the check_vmware_api.
When the script hangs and it is runned by nagios we got always a timeout (for the 30s default)
When I manually run the script I got sometimes a timeout and sometimes it just hangs and never turn back the prompt.
If I manually run two or more instances for this testing purpose the script hangs, but resources are good.
And if I run this tests on ubuntu 14.04 I don't have the issue.

Maybe it could be related to some library, please note that all the perl library version are changed between the two version of ubuntu.

Tell me if I can help you with more information,

Thanks

[pierluigiconti]

Hi,

if you need a 16.04 environment to test with I can provide to you.
We can schedule for a webex and I can show you the issue.

[fmikker]

@pierluigiconti If you run the check manually from the shell, do you get any error message?

[pierluigiconti]

no, if I run a single instance it works well, if I run at least two concurrent instances it hangs (via nagios and also via shell)

[fmikker]

Had some issues under CentOS 7 as well that gave the result that the check timed out, and the solution was to compile the required Perl libraries for VMware Perl SDK from cpan, which mitigated the issues.
You can have a look at the bug report and see if anything in that can mitigate your problems as well: https://jira.op5.com/browse/MON-10348

[pierluigiconti]

I also have tryied to install the perl modules from src via CPAN but no luck. The issue is still here

[eberkut]

I have the very same issue. I've tried various combination of vSphere SDK and PERL libs to no avail. Would appreciate any pointers to troubleshoot this issue.

[mikerosile]

I had similar issues using CentOS7, with version 6.0 of the vSphere Perl SDK. The patch provided by HapcJr , in the following article solved the issue for me.

https://communities.vmware.com/message/2298661

Also when installing the Perl SDK, do NOT use the pre-built perl modules, ensure that you have the "Development tools" group of packages installed (i.e. gcc and make) and allow the SDK installer to build the Perl modules (it will fetch them from cpan).

[underscoredje]

Hi!
I know this thread is inactive for a long time, but i had the same issue in my new job, on ubuntu 16.04.
In my case, it was related to /dev/random and entropy generator.

with command:
strace -e open,read perl -w check_vmware_api -H ...
I could see that last opened file is /dev/random, and the process hangs when reading it.

The cause was the entropy queue was too low. you can see it with
cat /proc/sys/kernel/random/entropy_avail

Normally, it must be around 3000. In my case, it was below 50.

The solution was to install rng-tools and start the rngd daemon.

You can find more information here:
https://wiki.archlinux.org/index.php/Rng-tools
http://manpages.ubuntu.com/manpages/xenial/fr/man4/random.4.html

[chukaman]

Thank you thank you thank you thank you a million times!!!! That has solved the issue for me too :)