Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
[dev.icinga.com #10363] Notification times w/ empty begin/end specifications prevent sending notifications #3486
This issue has been migrated from Redmine: https://dev.icinga.com/issues/10363
Created by nichols356 on 2015-10-14 18:32:15 +00:00
I apologize in advance if I am doing this the wrong way. I believe I have run into a bug the way I have implemented notifications and wanted to share it with the icinga2 dev team in hopes that this may improve the product.
My customers need several tiers of notifications with different people notified at different tiers and "apply Notification" got a bit out of hand with the many variances from one host to another. My solution was to define all required notification variables within the host itself, using templates for frequently used notification tiers and doing one-offs for the outliers that needed something special. This has the added benefit of being able to see what notification variables are being applied to a specific host from within icingaweb2. It could be that I'm just doing this completely wrong, but I would think the way I'm doing it should work.
My problem is that it appears to be applying all the variables as I want them to, but notifications aren't happening when they are expected. Using a test host I have established that the following is what happens:
I would think that the CUSTOM notification wouldn't work if notifications were completely broken, so I don't know why PROBLEM never sends, and RECOVERY will only send if you send a CUSTOM before it.
It looks like this in my configuration files:
I think I got everything pertaining to the .conf files...
Here is what the output of "icinga2 object list" looks like
icinga2.log (please also look at the attached screenshot with notations of the events)
I hope this provides enough information to you for debugging purposes. If there is anything else I should provide, please let me know.
2016-08-04 15:27:35 +00:00 by mfriedrich 43a177f
Updated by nichols356 on 2015-10-26 20:17:56 +00:00
I just some time to recreate this on a fresh install. It appears I left out a few parts of my config. Oops. I disabled the hosts.conf file and added the following chain of configs into a "test.conf" file to simplify debugging.
again, the same happens while recreating this issue:
I did get a debug log and wrote down the specific times of interest. I trimmed it to start just before I pulled the plug and stop right after i received the recovery notification, so it should contain all relevant information.
Here is the timeline:
16:00:50 unplugged - no notification
16:02:58 unplugged - no notification
Updated by nichols356 on 2015-10-27 18:20:23 +00:00
More progress on this. It appears that if I comment out the line
in my config above, it resolves the issue. Also, this can be resolved by setting host.vars.notify_infra.cutoff to something like 60m.
If times.end doesn't exist, notification works.
I would expect that if times.end is null, it should behave as if times.end doesn't exist.
Updated by mfriedrich on 2016-08-04 15:27:22 +00:00
While at it, also fixed printing the values in the debug log messages.