SyncVmDataStoreUsage fails on invalid data #23

Thomas-Gelf · 2018-11-09T10:31:49Z

/usr/bin/icingacli vspheredb daemon run
ErrorException in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Sync/SyncVmDatastoreUsage.php:52 with message: Undefined property: stdClass::$VirtualMachineUsageOnDatastore
PDOException in /usr/share/icingaweb2/library/vendor/Zend/Db/Adapter/Pdo/Abstract.php:302 with message: There is already an active transaction
Server for vCenterID=1 failed (PID 23163), will try again in 30 seconds

And when starting the daemon with systemctl start icinga-vspheredb.service, it fails with

● icinga-vspheredb.service - Icinga vSphereDB Daemon
Loaded: loaded (/etc/systemd/system/icinga-vspheredb.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2018-11-09 10:56:58 CET; 17s ago
Docs: https://icinga.com/docs/icinga-vsphere/latest/
Process: 50916 ExecStart=/usr/bin/icingacli vspheredb daemon run (code=exited, status=217/USER)
Main PID: 50916 (code=exited, status=217/USER)

Nov 09 10:56:58 Hostname systemd[1]: icinga-vspheredb.service: Main process exited, code=exited, status=217/USER

Originally posted by @robertkrenn in https://github.com/_render_node/MDU6SXNzdWUzNzkwOTU4NTk=/issues/unread_timeline#issuecomment-437317562

Thomas-Gelf · 2018-11-09T10:35:19Z

This consists of two issues:

we do not get the expected data. I'll double-check the specs and come back with a workaround
this Sync operation is catching Zend_Db_Exceptions and rolling back transactions cleanly. However, a PDOException is thrown here, this one slips through - and the failed transaction stays open. Will be fixed immediately.

refs #23

Thomas-Gelf · 2018-11-09T10:39:47Z

It should now no longer fail after reaching this point. So, when you pull and restart the other tasks should continue to run. I'll now tackle the original problem, we get wrong data in a place where it is not expected.

@robertkrenn: what vCenter/ESXi version are you using?

Thomas-Gelf · 2018-11-09T10:45:55Z

@robertkrenn: one more question, are you running VMs with no attached Storage Device?

Thomas-Gelf · 2018-11-09T10:53:41Z

I think the fix I pushed should resolve your problem. @robertkrenn: I'd still love to see above questions answered:

what vCenter/ESXi version are you using?
are you running VMs with no attached Storage Device?
This would help me to make sure that my assumptions where correct.

Thanks once again for reporting this!

Thomas.

robertkrenn · 2018-11-09T11:40:03Z

hi,
we have a vcsa 6.5 running.
what do you mean with "vms with no attached Storage Device"? We have about 50 SAN LUNs on an central strage cluster atteched to the ESXi hosts, where the vmdks of the VMs are lying on.

robertkrenn · 2018-11-09T12:01:06Z

ok, when running

/usr/bin/icingacli vspheredb daemon run
everything works just fine, all datastores, snapshots, events and alarms are visible, really great :-)
But I still don't get the daemon flying with systemctl
systemctl start icinga-vspheredb
about 30 Seconds nothing happens and then

Job for icinga-vspheredb.service failed because the control process exited with error code.
See "systemctl status icinga-vspheredb.service" and "journalctl -xe" for details.

● icinga-vspheredb.service - Icinga vSphereDB Daemon
Loaded: loaded (/etc/systemd/system/icinga-vspheredb.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: protocol) since Fri 2018-11-09 12:59:16 CET; 16s ago
Docs: https://icinga.com/docs/icinga-vsphere/latest/
Process: 9938 ExecStart=/usr/bin/icingacli vspheredb daemon run (code=exited, status=217/USER)
Main PID: 9938 (code=exited, status=217/USER)

it's as if systemd is waiting for something and ends in timeout

robertkrenn · 2018-11-09T12:20:37Z

in messages I find this

2018-11-09T13:15:16.582776+01:00 hostname systemd[1]: Starting Icinga vSphereDB Daemon...
2018-11-09T13:15:16.582964+01:00 hostname systemd[38406]: icinga-vspheredb.service: Failed to determine user credentials: No such process
2018-11-09T13:15:16.583176+01:00 hostname systemd[1]: icinga-vspheredb.service: Main process exited, code=exited, status=217/USER
2018-11-09T13:15:16.583364+01:00 hostname systemd[1]: Failed to start Icinga vSphereDB Daemon.

Thomas-Gelf · 2018-11-09T13:49:55Z

@robertkrenn: Is there an icingaweb2 User on your system? You can otherwise use apache, httpd or any member of your icingaweb2 group. If this is the case you can find a related solution here:

#14 (comment)

Please run systemctl daemon-reload when changing your Unit-File.

lokidaibel · 2018-11-13T14:30:20Z

Hey, wanted to try this aswell. But run into the same problem (or a similar):

ENV: vSphere Client Version 6.5.0.20000
RHEL: 7.6 (Maipo)
icinga2 / Icingaweb2:
icinga-rpm-release.noarch 7-2.el7.centos
icinga2.x86_64 2.10.1-1.el7.icinga
icinga2-bin.x86_64 2.10.1-1.el7.icinga
icinga2-common.x86_64 2.10.1-1.el7.icinga
icinga2-ido-mysql.x86_64 2.10.1-1.el7.icinga
icingacli.noarch 2.6.1-1.el7.icinga
icingaweb2.noarch 2.6.1-1.el7.icinga
icingaweb2-common.noarch 2.6.1-1.el7.icinga
icingaweb2-vendor-HTMLPurifier.noarch 1:2.6.1-1.el7.icinga
icingaweb2-vendor-JShrink.noarch 2.6.1-1.el7.icinga
icingaweb2-vendor-Parsedown.noarch 2.6.1-1.el7.icinga
icingaweb2-vendor-dompdf.noarch 2.6.1-1.el7.icinga
icingaweb2-vendor-lessphp.noarch 2.6.1-1.el7.icinga
icingaweb2-vendor-zf1.noarch 2.6.1-1.el7.icinga
php-Icinga.noarch 2.6.1-1.el7.icinga

I followed all steps in /doc/install:
[root@lim-vt21 vspheredb]# /usr/bin/icingacli vspheredb daemon run
PHP Fatal error: Uncaught Error: Class 'React\EventLoop\Factory' not found in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php:75
Stack trace:
#0 /usr/share/icingaweb2/modules/vspheredb/application/clicommands/DaemonCommand.php(26): Icinga\Module\Vspheredb\Daemon\Daemon->run()
#1 /usr/share/php/Icinga/Cli/Loader.php(265): Icinga\Module\Vspheredb\Clicommands\DaemonCommand->runAction()
#2 /usr/share/php/Icinga/Application/Cli.php(149): Icinga\Cli\Loader->dispatch()
#3 /usr/share/php/Icinga/Application/Cli.php(139): Icinga\Application\Cli->dispatchOnce()
#4 /usr/bin/icingacli(7): Icinga\Application\Cli->dispatch()
#5 {main}
thrown in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php on line 75

Fatal error: Uncaught Error: Class 'React\EventLoop\Factory' not found in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php:75
Stack trace:
#0 /usr/share/icingaweb2/modules/vspheredb/application/clicommands/DaemonCommand.php(26): Icinga\Module\Vspheredb\Daemon\Daemon->run()
#1 /usr/share/php/Icinga/Cli/Loader.php(265): Icinga\Module\Vspheredb\Clicommands\DaemonCommand->runAction()
#2 /usr/share/php/Icinga/Application/Cli.php(149): Icinga\Cli\Loader->dispatch()
#3 /usr/share/php/Icinga/Application/Cli.php(139): Icinga\Application\Cli->dispatchOnce()
#4 /usr/bin/icingacli(7): Icinga\Application\Cli->dispatch()
#5 {main}
thrown in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php on line 75
[root@lim-vt21 vspheredb]# php --ini
Configuration File (php.ini) Path: /etc/opt/rh/rh-php71
Loaded Configuration File: /etc/opt/rh/rh-php71/php.ini
Scan for additional .ini files in: /etc/opt/rh/rh-php71/php.d
Additional .ini files parsed: /etc/opt/rh/rh-php71/php.d/20-bz2.ini,
/etc/opt/rh/rh-php71/php.d/20-calendar.ini,
/etc/opt/rh/rh-php71/php.d/20-ctype.ini,
/etc/opt/rh/rh-php71/php.d/20-curl.ini,
/etc/opt/rh/rh-php71/php.d/20-dom.ini,
/etc/opt/rh/rh-php71/php.d/20-exif.ini,
/etc/opt/rh/rh-php71/php.d/20-fileinfo.ini,
/etc/opt/rh/rh-php71/php.d/20-ftp.ini,
/etc/opt/rh/rh-php71/php.d/20-gd.ini,
/etc/opt/rh/rh-php71/php.d/20-gettext.ini,
/etc/opt/rh/rh-php71/php.d/20-iconv.ini,
/etc/opt/rh/rh-php71/php.d/20-intl.ini,
/etc/opt/rh/rh-php71/php.d/20-json.ini,
/etc/opt/rh/rh-php71/php.d/20-ldap.ini,
/etc/opt/rh/rh-php71/php.d/20-mbstring.ini,
/etc/opt/rh/rh-php71/php.d/20-mysqlnd.ini,
/etc/opt/rh/rh-php71/php.d/20-pdo.ini,
/etc/opt/rh/rh-php71/php.d/20-pgsql.ini,
/etc/opt/rh/rh-php71/php.d/20-phar.ini,
/etc/opt/rh/rh-php71/php.d/20-posix.ini,
/etc/opt/rh/rh-php71/php.d/20-shmop.ini,
/etc/opt/rh/rh-php71/php.d/20-simplexml.ini,
/etc/opt/rh/rh-php71/php.d/20-soap.ini,
/etc/opt/rh/rh-php71/php.d/20-sockets.ini,
/etc/opt/rh/rh-php71/php.d/20-sqlite3.ini,
/etc/opt/rh/rh-php71/php.d/20-sysvmsg.ini,
/etc/opt/rh/rh-php71/php.d/20-sysvsem.ini,
/etc/opt/rh/rh-php71/php.d/20-sysvshm.ini,
/etc/opt/rh/rh-php71/php.d/20-tokenizer.ini,
/etc/opt/rh/rh-php71/php.d/20-xml.ini,
/etc/opt/rh/rh-php71/php.d/20-xmlwriter.ini,
/etc/opt/rh/rh-php71/php.d/20-xsl.ini,
/etc/opt/rh/rh-php71/php.d/20-zip.ini,
/etc/opt/rh/rh-php71/php.d/30-mysqli.ini,
/etc/opt/rh/rh-php71/php.d/30-pdo_mysql.ini,
/etc/opt/rh/rh-php71/php.d/30-pdo_pgsql.ini,
/etc/opt/rh/rh-php71/php.d/30-pdo_sqlite.ini,
/etc/opt/rh/rh-php71/php.d/30-wddx.ini,
/etc/opt/rh/rh-php71/php.d/30-xmlreader.ini

Thomas-Gelf · 2018-11-13T16:35:02Z

@lokidaibel: you probably didn't install a release version of the reactbundle - please re-check all requirements.

lokidaibel · 2018-11-14T10:34:06Z

Nope.

https://gyazo.com/9a941bc8581ad94e96f0b9244cb10dd5

Any other suggestions ?

Greetings

Lukas

Thomas-Gelf · 2018-11-14T10:38:54Z

This does not fit the shown error message:

Fatal error: Uncaught Error: Class 'React\EventLoop\Factory' not found in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php:75

Is this still the shown error?

lokidaibel · 2018-11-14T12:08:46Z

I have dig some more arround:

[root@lim-vt21 reactbundle]# /usr/bin/icingacli vspheredb daemon run --trace --debug
Ready to run
DB configuration loaded
Reconnecting to DB
Connected to the database
vCenter ID=1 is now starting
Constructing ServerRunner
Starting for vCenterID=
Initializing vCenter for vc6-vp01
Loading sdk/vimService.wsdl
CURL ERROR: Received HTTP code 502 from proxy after CONNECT
Pid 20000 stopped

Proxy related error. The proxy Settings i set in the konfiguration of VSphere seams to be ignored ?!

greetings

Lukas

Thomas-Gelf · 2018-11-14T12:45:53Z

This looks better now. What did you change? Proxy SHOULD work fine, but honestly I only tested with SOCKS proxies. What kind of proxy did you configure, and are you using it with or without authentication?

Thanks,
Thomas

NB: Could you please open a dedicated issue? This is way off-topic here.

lokidaibel · 2018-11-14T15:12:40Z

That are my Settings (for test): https://gyazo.com/66655195a119d21de7af172103f7f79a

…

On Wed, Nov 14, 2018 at 1:45 PM Thomas Gelf ***@***.***> wrote: This looks better now. What did you change? Proxy SHOULD work fine, but honestly I only tested with SOCKS proxies. What kind of proxy did you configure, and are you using it with or without authentication? Thanks, Thomas NB: Could you please open a dedicated issue? This is way off-topic here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMleu7HTUjx8olWevao6-hdoQa1ieOMHks5uvBCBgaJpZM4YWVVO> .

Thomas-Gelf · 2018-11-14T16:15:17Z

Please continue in #41.

Bassysta · 2018-12-07T15:14:58Z

Hi. i've similar problem runnind daemon. here the log:
PHP Fatal error: Uncaught Error: Call to undefined function Icinga\Module\Vspheredb\Daemon\posix_getpid() in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php:63
Stack trace:
#0 /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php(56): Icinga\Module\Vspheredb\Daemon\Daemon->detectProcessInfo()
#1 /usr/share/icingaweb2/modules/vspheredb/application/clicommands/DaemonCommand.php(25): Icinga\Module\Vspheredb\Daemon\Daemon->__construct()
#2 /usr/share/php/Icinga/Cli/Loader.php(265): Icinga\Module\Vspheredb\Clicommands\DaemonCommand->runAction()
#3 /usr/share/php/Icinga/Application/Cli.php(149): Icinga\Cli\Loader->dispatch()
#4 /usr/share/php/Icinga/Application/Cli.php(139): Icinga\Application\Cli->dispatchOnce()
#5 /usr/bin/icingacli(7): Icinga\Application\Cli->dispatch()
#6 {main}
thrown in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php on line 63

Fatal error: Uncaught Error: Call to undefined function Icinga\Module\Vspheredb\Daemon\posix_getpid() in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php:63
Stack trace:
#0 /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php(56): Icinga\Module\Vspheredb\Daemon\Daemon->detectProcessInfo()
#1 /usr/share/icingaweb2/modules/vspheredb/application/clicommands/DaemonCommand.php(25): Icinga\Module\Vspheredb\Daemon\Daemon->__construct()
#2 /usr/share/php/Icinga/Cli/Loader.php(265): Icinga\Module\Vspheredb\Clicommands\DaemonCommand->runAction()
#3 /usr/share/php/Icinga/Application/Cli.php(149): Icinga\Cli\Loader->dispatch()
#4 /usr/share/php/Icinga/Application/Cli.php(139): Icinga\Application\Cli->dispatchOnce()
#5 /usr/bin/icingacli(7): Icinga\Application\Cli->dispatch()
#6 {main}
thrown in /usr/share/icingaweb2/modules/vspheredb/library/Vspheredb/Daemon/Daemon.php on line 63

Thomas-Gelf · 2018-12-09T07:57:01Z

@Bassysta: this (closed) issue reads SyncVmDataStoreUsage fails on invalid data, I do not see why your problem should be related. Your system complains that posix_getpid() hasn't been found, and php-posix is part of our requirements. When running on RHEL/CentOS you can find it in the php-process RPM, see here. I'll add a related hint to the documentation to make it easier to figure out.

refs #23

Thomas-Gelf mentioned this issue Nov 9, 2018

daemon problems #21

Closed

Thomas-Gelf self-assigned this Nov 9, 2018

Thomas-Gelf added a commit that referenced this issue Nov 9, 2018

SyncVmDatastoreUsage: roll back on any Exception

c1a3de6

refs #23

Thomas-Gelf added the bug label Nov 9, 2018

Thomas-Gelf added this to the v1.0.2 milestone Nov 9, 2018

Thomas-Gelf mentioned this issue Nov 9, 2018

All transactions should be rolled back on any Exception #24

Closed

Thomas-Gelf closed this as completed in 7c1b09b Nov 9, 2018

Thomas-Gelf mentioned this issue Nov 14, 2018

502 using a Proxy Connection #41

Closed

Thomas-Gelf added a commit that referenced this issue Dec 9, 2018

doc/install: mention php-process for RH

6c36add

refs #23

lokidaibel mentioned this issue Feb 11, 2019

Division by zero (DatastoreUsage.php:196) when viewing Datastores page #75

Closed

ruslanss-bob mentioned this issue Feb 8, 2021

Monitoring Integration #236

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SyncVmDataStoreUsage fails on invalid data #23

SyncVmDataStoreUsage fails on invalid data #23

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

robertkrenn commented Nov 9, 2018

robertkrenn commented Nov 9, 2018

robertkrenn commented Nov 9, 2018 •

edited

Loading

Thomas-Gelf commented Nov 9, 2018

lokidaibel commented Nov 13, 2018

Thomas-Gelf commented Nov 13, 2018

lokidaibel commented Nov 14, 2018

Thomas-Gelf commented Nov 14, 2018

lokidaibel commented Nov 14, 2018 •

edited

Loading

Thomas-Gelf commented Nov 14, 2018

lokidaibel commented Nov 14, 2018 via email

Thomas-Gelf commented Nov 14, 2018

Bassysta commented Dec 7, 2018

Thomas-Gelf commented Dec 9, 2018

SyncVmDataStoreUsage fails on invalid data #23

SyncVmDataStoreUsage fails on invalid data #23

Comments

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

Thomas-Gelf commented Nov 9, 2018

robertkrenn commented Nov 9, 2018

robertkrenn commented Nov 9, 2018

robertkrenn commented Nov 9, 2018 • edited Loading

Thomas-Gelf commented Nov 9, 2018

lokidaibel commented Nov 13, 2018

Thomas-Gelf commented Nov 13, 2018

lokidaibel commented Nov 14, 2018

Thomas-Gelf commented Nov 14, 2018

lokidaibel commented Nov 14, 2018 • edited Loading

Thomas-Gelf commented Nov 14, 2018

lokidaibel commented Nov 14, 2018 via email

Thomas-Gelf commented Nov 14, 2018

Bassysta commented Dec 7, 2018

Thomas-Gelf commented Dec 9, 2018

robertkrenn commented Nov 9, 2018 •

edited

Loading

lokidaibel commented Nov 14, 2018 •

edited

Loading