Alert triggered but message not included #2382
Comments
This looks to be a problem with the message count alert condition code. It first retrieves the number of matching messages and then runs a second query to retrieve the actual messages. For the second query it uses a All other alert conditions behave correctly. |
Thanks for the update Kay. |
This condition works by first running a count in Elasticsearch and then, if the condition triggers, a search to fetch the messages that will be included in the check result. Both queries use a RelativeRange object which returns a new time for every getFrom() and getTo() call that is made. This can result in different messages being included in the check result or no messages at all given the count query takes a while. The RelativeRange is now converted to an AbsoluteRange object which is then used to run the count and search query. This makes sure the exact same time range is used no matter how much time is in between the calls. Refs #1704 Fixes #2382
A fix for this will be included in the upcoming Graylog 2.1. #2546 |
This condition works by first running a count in Elasticsearch and then, if the condition triggers, a search to fetch the messages that will be included in the check result. Both queries use a RelativeRange object which returns a new time for every getFrom() and getTo() call that is made. This can result in different messages being included in the check result or no messages at all given the count query takes a while. The RelativeRange is now converted to an AbsoluteRange object which is then used to run the count and search query. This makes sure the exact same time range is used no matter how much time is in between the calls. Refs #1704 Fixes #2382
Can this be reopened, or should a new ticket be logged. I setup multiple notification plugins (Jira, slack) Thanks |
@123dev Please open a new issue and add details on how to reproduce this. Thank you! |
Problem description
We use graylog-jira-alarmcallback to log jira tickets when an alert on a stream is raised.
Every now and then we notice that an alert is raised without the message being passed to the plugin.
We have configured the alert condition as such.
More than 0 messages in the last 1 minute and wait at least 0 minutes until triggering a new alert.
When sending an alert, include the last 1 messages ...
Checking the graylog logs around the problem time we see the following message.
2016-06-17_16:52:04.15041 2016-06-17 16:52:04,149 WARN : com.bidorbuy.graylog.alarmcallbacks.jira.JiraAlarmCallback - Skipping JIRA-issue MD5 generation, alarmcallback did not provide a message
We initailly logged an issue with the plugin author who after investigating the issue believed this is possibly a graylog bug.
You can see the discussion here
It is worth to note that we are running more than one graylog servers in a clustered environment, in case that has any relevance.
Steps to reproduce the problem
Not easy, as it happens rarely
WARN : com.bidorbuy.graylog.alarmcallbacks.jira.JiraAlarmCallback - Skipping JIRA-issue MD5 generation, alarmcallback did not provide a message
Environment
AWS Image
Thanks
The text was updated successfully, but these errors were encountered: