Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #11654] Flapping detection enabled but notifications sent #4150

Closed
icinga-migration opened this issue Apr 21, 2016 · 17 comments
Closed

Comments

@icinga-migration
Copy link
Member

@icinga-migration icinga-migration commented Apr 21, 2016

This issue has been migrated from Redmine: https://dev.icinga.com/issues/11654

Created by jamacias on 2016-04-21 23:20:09 +00:00

Assignee: (none)
Status: New
Target Version: Backlog
Last Update: 2016-12-05 09:32:40 +00:00 (in Redmine)

Icinga Version: 2.4.4
Backport?: Not yet backported
Include in Changelog: 1

I upgraded Icinga to version 2.4.4 and learned that bug #9969 is still present. I used the rpm repository to upgrade. I get PROBLEM
emails after the flapping has been turned on. However, I did stop receiving RECOVERY messages. This is my configuration and tools I used:

max_check_attempts = 3
check_interval = 1m
retry_interval = 30s

flapping_threshold = 3

this is the simple script that I used at the host in order to trigger the alert:

#!/bin/sh
while [ true ]
do
stress -c 2 -t 300
sleep 120
done

I'm also attaching the script used at the server

Attachments

Changesets

2016-05-31 15:51:45 +00:00 by (unknown) b2e7747

WIP: Make change to OK always a hard state

refs #11654

2016-06-09 12:28:46 +00:00 by (unknown) 754638e

WIP: Make change to OK always a hard state

refs #11654

2016-06-13 08:43:57 +00:00 by (unknown) 8808e70

Make change to OK always a hard state

refs #11654

Relations:

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Apr 22, 2016

Updated by mfriedrich on 2016-04-22 07:58:47 +00:00

  • Status changed from New to Feedback
  • Assigned to set to jamacias

Please verify that flapping is active (e.g. by querying the API for that object in a defined interval). In addition to that check the debug logs for suppressing notifications due to flapping detection.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Apr 22, 2016

Updated by mfriedrich on 2016-04-22 08:05:01 +00:00

  • Subject changed from Icinga version 2.4.4 still sends notification while flapping is active to Flapping detection enabled but notifications sent
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Apr 23, 2016

Updated by jamacias on 2016-04-23 00:05:17 +00:00

  • File added API-OUTPUT
  • File added clean-debug.log
  • File added debug.log

I am sending you the information requested.
1.) file: API-OUTPUT - query of object in question every 15 seconds
2.) clean-debug.log

*** indicates FLAPPINGSTART for the object in question. This occurs at 016-04-22 16:12:08
*** indicates SERVICE NOTIFICATION for the same object. This occurs at 2016-04-22 16:16:13 and again at 2016-04-22 16:23:12

3.) debug.log - this is the complete debug.log

I noticed a very important observation: Even though the Icinga2 Web GUI already indicates "Currently flapping with a 13.00% state change rate" for the object, and the debug.log also indicates FLAPPINGSTART for the same object, the same API object being queried still indicates a parameter flapping false

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented May 31, 2016

Updated by mfrosch on 2016-05-31 14:50:19 +00:00

  • Status changed from Feedback to New
  • Assigned to changed from jamacias to mfrosch

It seems like there is an odd problem with hard/soft states when returning to okay.

Which basically blocks any flapping detection if the max_check_attempts != 1

In addition, we need to further test and document flapping.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented May 31, 2016

Updated by mfrosch on 2016-05-31 14:51:34 +00:00

  • Relates set to 11861
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented May 31, 2016

Updated by mfrosch on 2016-05-31 15:03:31 +00:00

  • Relates set to 11862
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Jul 5, 2016

Updated by mfrosch on 2016-07-05 09:33:32 +00:00

  • Assigned to deleted mfrosch
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Jul 28, 2016

Updated by mfriedrich on 2016-07-28 16:20:59 +00:00

  • Target Version set to Backlog
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Jul 28, 2016

Updated by mfriedrich on 2016-07-28 16:21:15 +00:00

  • Priority changed from Normal to High
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Aug 23, 2016

Updated by mfriedrich on 2016-08-23 12:39:43 +00:00

  • Relates set to 12517
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 2, 2016

Updated by jamacias on 2016-12-02 22:08:45 +00:00

Hi,
The last update was done by mfriedrich 3 months ago. He makes a reference to bug 12517 which is closed. Does that mean this bug is also fixed? Please advise so that I can test it.
Best regards,
Antonio macias

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 5, 2016

Updated by mfriedrich on 2016-12-05 09:32:40 +00:00

It is still an open issue. The linked issue fixes a different problem but is related in terms of tests.

@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 5, 2016

Updated by mfriedrich on 2016-12-05 10:42:46 +00:00

  • Blocks set to 13399
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Dec 16, 2016

Updated by elippmann on 2016-12-16 11:43:15 +00:00

  • Blocks deleted 13399
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Jan 9, 2017

Updated by mfriedrich on 2017-01-09 17:01:24 +00:00

  • Relates set to 13847
@icinga-migration
Copy link
Member Author

@icinga-migration icinga-migration commented Jan 9, 2017

Updated by mfriedrich on 2017-01-09 17:01:48 +00:00

  • Relates set to 13827
@dnsmichi
Copy link
Contributor

@dnsmichi dnsmichi commented Feb 7, 2017

Will be continued in #4982.

@dnsmichi dnsmichi closed this Feb 7, 2017
@dnsmichi dnsmichi removed this from the Backlog milestone Feb 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.