Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #12529] Icinga 2 sends SOFT recovery notifications #4550

Closed
icinga-migration opened this issue Aug 24, 2016 · 21 comments

Comments

Projects
None yet
1 participant
@icinga-migration
Copy link
Member

commented Aug 24, 2016

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12529

Created by phil_fry on 2016-08-24 07:02:44 +00:00

Assignee: gbeutner
Status: Resolved (closed on 2016-08-24 09:15:12 +00:00)
Target Version: 2.5.2
Last Update: 2016-08-30 07:18:32 +00:00 (in Redmine)

Icinga Version: 2.5.1
Backport?: Not yet backported
Include in Changelog: 1

Hi,

sorry that I have to reopen it (#12518), but the problem got worse. Now I'm getting LOTS of recovery notifications for soft {critical, warn, unknown} alerts.

Kind regards

Philippe

Changesets

2016-08-24 09:13:19 +00:00 by gbeutner 1a47a7c

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 09:50:29 +00:00 by gbeutner 220abc7

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 10:13:18 +00:00 by gbeutner 614e0ac

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 10:22:08 +00:00 by gbeutner deb938d

Fix incorrect notifications for soft recoveries

fixes #12529

2016-08-24 12:01:10 +00:00 by gbeutner 8c1fc81

Fix incorrect notifications for soft recoveries

fixes #12529

Relations:

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by phenomic on 2016-08-24 07:22:09 +00:00

Jep, same here. Getting notifications after soft state.
But i'm not getting recovery notifications - i'm getting problem notifications.

PROBLEM - vmware-esx9 - CPU_USAGE is OK

[2016-08-24 09:14:09 +0200] information/Checkable: Checking for configured notifications for object 'vmware-esx9!CPU_USAGE'
[2016-08-24 09:14:09 +0200] information/Notification: Sending 'Problem' notification 'vmware-esx9!CPU_USAGE!mail-icingaadmin for user 'xxxxxxx'
[2016-08-24 09:14:09 +0200] information/Notification: Completed sending 'Problem' notification 'vmware-esx9!CPU_USAGE!mail-icingaadmin' for checkable 'vmware-esx9!CPU_USAGE' and user 'xxxxxx'.
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by Clasko on 2016-08-24 07:45:24 +00:00

Same here and also problem notifications but with state "OK".

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by ios77 on 2016-08-24 08:12:31 +00:00

Same problem for me,
receiving a lot of recovery notifications for SOFT NOT-OK states.
Please fix it thanks.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by gbeutner on 2016-08-24 08:13:21 +00:00

  • Status changed from New to Assigned
  • Assigned to set to gbeutner
  • Target Version set to 2.5.2
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by gbeutner on 2016-08-24 09:13:38 +00:00

  • Relates set to 12530
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by gbeutner on 2016-08-24 09:15:12 +00:00

  • Status changed from Assigned to Resolved
  • Done % changed from 0 to 100

Applied in changeset 1a47a7c.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by mfriedrich on 2016-08-24 09:49:21 +00:00

  • Priority changed from Normal to Urgent
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by mfriedrich on 2016-08-24 10:02:19 +00:00

  • Subject changed from getting spurious recovery notifications (2) to Icinga 2 sends SOFT recovery notifications
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by mnardin on 2016-08-24 12:17:42 +00:00

Hi,
just to clarify: is this problem https://dev.icinga.org/issues/12293#change-53944 still fixed? I don't know anything about the source code, but I've noticed that part of the code regarding the previous fix was changed.

Best regards
Mirko

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by mfriedrich on 2016-08-24 12:22:33 +00:00

  • Relates set to 12293
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 24, 2016

Updated by mfriedrich on 2016-08-24 12:23:53 +00:00

Nope that has been reverted. Broken fix slipped through the notification QA tests.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 25, 2016

Updated by mnardin on 2016-08-25 12:19:19 +00:00

Thank you, understood. What is the planned target version to fix this behavior (issue 12293) again?

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by ios77 on 2016-08-26 10:01:41 +00:00

Dear, have you really fix this bug?
With the last update I have both notification for soft error anf for soft recovery!!!

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by gbeutner on 2016-08-26 10:02:19 +00:00

ios77: icinga2 --version

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by ios77 on 2016-08-26 10:04:22 +00:00

icinga2 - The Icinga 2 network monitoring daemon (version: v2.5.3)

Copyright © 2012-2016 Icinga Development Team (https://www.icinga.org/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Application information:
Installation root: /usr
Sysconf directory: /etc
Run directory: /run
Local state directory: /var
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

System information:
Platform: Red Hat Enterprise Linux Workstation
Platform version: 7.2 (Maipo)
Kernel: Linux
Kernel version: 3.10.0-327.28.2.el7.x86_64
Architecture: x86_64

Build information:
Compiler: GNU 4.8.5
Build host: unknown

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by gbeutner on 2016-08-26 10:05:24 +00:00

No, this should definitely work with 2.5.3.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by ios77 on 2016-08-26 10:13:19 +00:00

Service using this template start to send notification for soft problem state and for soft recovery in the last two icinga version (2.5.1-3)

template Service "fbk-critical-service" {

/* Maximum number of checks which are executed before changing to a hard state. */
max_check_attempts = 2
check_interval = 1m
retry_interval = 30s
}

no configuration changes or others. Only version update...
Maybe I am wrong, but in my opinion is not fixed.
Could you please double check it?

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 26, 2016

Updated by gbeutner on 2016-08-26 10:37:12 +00:00

Are you able to reproduce this with max_check_attempts other than 2?

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 29, 2016

Updated by ios77 on 2016-08-29 09:03:13 +00:00

Dear,
yes same problem with max_check_attempts = 3.
The notifications are sended before reaching the hard state.

[2016-08-29 10:49:54 +0200] information/Notification: Completed sending 'Problem' notification 'XXX.itmail-itid2' for checkable 'XXX.it!httpInaz' and user 'XXX'.
[2016-08-29 10:50:54 +0200] information/Notification: Completed sending 'Recovery' notification 'XXX.itmail-itid2' for checkable 'XXX.it!httpInaz' and user 'XXX'.

Thanks

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Aug 30, 2016

Updated by gbeutner on 2016-08-30 07:18:32 +00:00

There does appear to be a problem with max_check_attempts = 2 (#12530), however I am unable to reproduce this with any other value.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2016

Updated by mfriedrich on 2016-09-27 14:11:59 +00:00

  • Relates set to 12694

@icinga-migration icinga-migration added this to the 2.5.2 milestone Jan 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.