Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #9570] Multi select causes commands to fail #3130

Closed
icinga-migration opened this issue Jul 7, 2015 · 15 comments

Comments

Projects
None yet
1 participant
@icinga-migration
Copy link
Member

commented Jul 7, 2015

This issue has been migrated from Redmine: https://dev.icinga.com/issues/9570

Created by smadmin on 2015-07-07 07:34:12 +00:00

Assignee: (none)
Status: Closed (closed on 2016-02-24 23:55:42 +00:00)
Target Version: (none)
Last Update: 2016-02-24 23:55:42 +00:00 (in Redmine)

Icinga Version: v2.3.8

Using multi select on hosts or service causes commands to fail with message

Can't send external Icinga command to the local command file "/usr/local/nagios/icinga2.cmd": No such device or address


#0 /usr/share/icingaweb2/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php(68): Icinga\Module\Monitoring\Command\Transport\LocalCommandFile->send(Object(Icinga\Module\Monitoring\Command\Object\ScheduleServiceCheckCommand))
#1 /usr/share/php/Icinga/Web/Form.php(909): Icinga\Module\Monitoring\Forms\Command\Object\CheckNowCommandForm->onSuccess()
#2 /usr/share/icingaweb2/modules/monitoring/application/controllers/ServicesController.php(98): Icinga\Web\Form->handleRequest()
#3 /usr/share/icingaweb2/library/vendor/Zend/Controller/Action.php(507): Monitoring_ServicesController->showAction()
#4 /usr/share/icingaweb2/library/vendor/Zend/Controller/Dispatcher/Standard.php(303): Zend_Controller_Action->dispatch('showAction')
#5 /usr/share/icingaweb2/library/vendor/Zend/Controller/Front.php(937): Zend_Controller_Dispatcher_Standard->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#6 /usr/share/php/Icinga/Application/Web.php(154): Zend_Controller_Front->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#7 /usr/share/php/Icinga/Application/webrouter.php(111): Icinga\Application\Web->dispatch()
#8 /usr/share/icingaweb2/public/index.php(4): require_once('/usr/share/php/...')
#9 {main}
while executed on a single object it works.

Changesets

2015-08-10 09:31:27 +00:00 by elippmann cb0b3c8754b0e8b1dc96b240f5f40a084b0503c2

monitoring: Let PHP flush the writer buffer to the command file

refs #9570

Relations:

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by jmeyer on 2015-07-07 07:53:35 +00:00

  • Status changed from New to Feedback
  • Assigned to set to smadmin

Hi,

which version of Icinga Web 2 are you using?
This should have been fixed long ago: #8815 (I guess you're using a remote instance)

Best regards,
Johannes

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by dgoetz on 2015-07-07 08:20:00 +00:00

  • Assigned to deleted smadmin

Using the release candidate and a local command pipe

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by jmeyer on 2015-07-07 08:37:55 +00:00

I cannot reproduce this, neither with rc1 nor the latest master. Please make sure you've upgraded properly. (all files are up2date etc)

If the issue persists:

  • Enable debug log
  • Submit a single command
  • Submit multiple commands
  • Copy&Paste the respective log lines as well as your instances.ini (/etc/icingaweb2/modules/monitoring) here
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by smadmin on 2015-07-07 09:39:55 +00:00

We see this in the log:

2015-07-07T11:21:49+02:00 - DEBUG - Sending external Icinga command "[1436260909] SCHEDULE_FORCED_SVC_CHECK;SERVER1;linux_daemon_nrpe;1436260909" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - DEBUG - Sending external Icinga command "[1436260917] SCHEDULE_FORCED_SVC_CHECK;SERVER1;linux_daemon_nrpe;1436260917" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - DEBUG - Sending external Icinga command "[1436260917] SCHEDULE_FORCED_SVC_CHECK;SERVER2;linux_daemon_nrpe;1436260917" to the local command file "/usr/local/nagios/icinga2.cmd"
2015-07-07T11:21:57+02:00 - ERROR - Icinga\Module\Monitoring\Command\Exception\TransportException in /usr/share/icingaweb2/modules/monitoring/library/Monitoring/Command/Transport/LocalCommandFile.php:132 with message: Can't send external Icinga command to the local command file "/usr/local/nagios/icinga2.cmd":  No such device or address
2015-07-07T11:21:57+02:00 - ERROR - Stacktrace: #0 /usr/share/icingaweb2/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php(68): Icinga\Module\Monitoring\Command\Transport\LocalCommandFile->send(Object(Icinga\Module\Monitoring\Command\Object\ScheduleServiceCheckCommand))

#1 /usr/share/php/Icinga/Web/Form.php(909): Icinga\Module\Monitoring\Forms\Command\Object\CheckNowCommandForm->onSuccess()
#2 /usr/share/icingaweb2/modules/monitoring/application/controllers/ServicesController.php(98): Icinga\Web\Form->handleRequest()
#3 /usr/share/icingaweb2/library/vendor/Zend/Controller/Action.php(507): Monitoring_ServicesController->showAction()
#4 /usr/share/icingaweb2/library/vendor/Zend/Controller/Dispatcher/Standard.php(303): Zend_Controller_Action->dispatch('showAction')
#5 /usr/share/icingaweb2/library/vendor/Zend/Controller/Front.php(937): Zend_Controller_Dispatcher_Standard->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#6 /usr/share/php/Icinga/Application/Web.php(154): Zend_Controller_Front->dispatch(Object(Icinga\Web\Request), Object(Icinga\Web\Response))
#7 /usr/share/php/Icinga/Application/webrouter.php(111): Icinga\Application\Web->dispatch()
#8 /usr/share/icingaweb2/public/index.php(4): require_once('/usr/share/php/...')
#9 {main}
First line is from the single command, afterwards from selecting 5 services. It seems to fail from the second service on, so perhaps the pipe is blocked. Directly inserting command in a while loop does not block it and all commands are issued.
# cat /etc/icingaweb2/modules/monitoring/instances.ini 
[icinga]
transport           = "local"
path                = "/usr/local/nagios/icinga2.cmd"
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by jmeyer on 2015-07-07 11:08:03 +00:00

Please apply the following patch and report whether it fixes the issue:

diff --git a/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php b/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
index 850443e..10c06e9 100644
--- a/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
+++ b/modules/monitoring/application/forms/Command/Object/CheckNowCommandForm.php
@@ -65,7 +65,11 @@ class CheckNowCommandForm extends ObjectsCommandForm
                 ->setObject($object)
                 ->setForced()
                 ->setCheckTime(time());
-            $this->getTransport($this->request)->send($check);
+            $transport = $this->getTransport($this->request);
+            if ($transport::TRANSPORT === 'local') {
+                $transport->setOpenMode('w');
+            }
+            $transport->send($check);
         }
         Notification::success(mtp(
             'monitoring',
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by smadmin on 2015-07-07 13:00:55 +00:00

This removes the error message, but let it fail silently while sending if pipe is blocked. (I can only see one or two external commands in icinga2.log out of five executed in the webinterface).

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by jmeyer on 2015-07-07 14:30:48 +00:00

Then there is something else wrong in your environment. Using fopen(, "w") is a blocking call, which will not succeed until something (Icinga 2) actually reads from the pipe. The patch I've proposed behaves the same as your while loop and issues one command at a time and continues after Icinga 2 retrieved the command.

I've even tried it myself to be really sure:

  • Stopped Icinga 2
  • Scheduled five host checks (Web page loads infinitely as the first write attempt blocks the process)
  • Started Icinga 2
  • All five commands were received and processed by Icinga 2

How did you install Icinga 2?
Which operating system are you running?

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2015

Updated by vvv on 2015-07-07 22:29:30 +00:00

I was seeing similar issue in 3.beta3. 4.rc1 seems to have fixed it for me.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 16, 2015

Updated by elippmann on 2015-07-16 14:11:27 +00:00

  • Status changed from Feedback to New
  • Target Version set to 271
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jul 17, 2015

Updated by elippmann on 2015-07-17 14:23:08 +00:00

  • Estimated Hours set to 0.5
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 10, 2015

Updated by icinga-kanban on 2015-08-10 09:34:47 +00:00

Build !#939 triggered by commit cb0b3c8 passed successfully.

Branch: origin/master
Author: Eric Lippmann

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 18, 2015

Updated by jmeyer on 2015-08-18 06:38:29 +00:00

  • Duplicated set to 9938
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 18, 2015

Updated by jmeyer on 2015-08-18 09:12:59 +00:00

  • Project changed from Icinga Web 2 to Icinga 2
  • Category deleted 148
  • Priority changed from Normal to High
  • Target Version deleted 271
  • Estimated Hours deleted 0.5
  • Icinga Version set to v2.3.8

Hi Icinga2!

We suspect that this issue is caused by your commandpipe listener implementation. Can you please make your devs have a look at it if there is something to improve when processing multiple commands?

Thanks,
Johannes

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 18, 2015

Updated by mfriedrich on 2015-08-18 09:19:13 +00:00

  • Category set to libicinga
  • Priority changed from High to Normal
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Feb 24, 2016

Updated by mfriedrich on 2016-02-24 23:55:42 +00:00

  • Status changed from New to Closed

Haven't seen the issue after the release of 2.4.0. Please retest and open a new issue if it still occurs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.