[Discover] UI for alert creation

[lukeelmers]

As the last step of a Discover alerting MVP, we need to add an alert creation UI that is used to create the alert executor.

Here are some wireframe concepts from @AlonaNadler for further discussion:

Clicking on create alert opens a flyout. The flyout takes into account the search bar query and filters. Whatever is in blue is configurable:

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[shaunmcgough]

The initial mocks here were fluid, and are being revised. The create alert button in the first image can be global, and let's have design's @andreadelrio take a look at where this is most aesthetic. The second image of the flyout need some refinement. We have the ability to take what has been done by other teams and leverage it.

Here is a new flyout mock, which is based on the Metrics alerting flyout

Please note, any information that can be automatically filled in, such as the parameters of a search that was done in Discover, or filters that were applied, should be auto-populated in the 'Create alert' form.

[AlonaNadler]

Few things im not im following:

• The image you posted seems like a screenshot from the metric app, what's the goal?
• The alert in metric is a bit different than the one in Discover not sure all the fields here are valid or if it is the right comparison

[shaunmcgough]

Hi @AlonaNadler thanks for pointing this out. The image is a screenshot from the metric app. The goal here is to just put some more detail around what this should look like. Of course, these aren't metrics alerts, but the sections; Create alert, Alert trigger, filter, Create alert per (Grouping), and Actions are relevant. We can reword things if needed, but I see all of these sections being relevant. Perhaps, some more relevant in a phase II of the UI. The desired audience for this right now is @andreadelrio so let's see what questions come up from her end.

[andreadelrio]

Figma file - Currently shows first iteration of design:
https://www.figma.com/proto/AzfJmKtiQyM8WjnFRQnY5H/Alerting?node-id=1%3A1705&viewport=-537%2C442%2C0.4641232490539551&scaling=min-zoom

Things coming to next iteration (discussed in our meeting):

• No need to have KQL + filters + Update button inside flyout, users should go to main screen to make changes to those fields.
• The KQL query and filters should be mirrored into the flyout but their editing happens outside the flyout.
• Add threshold to Alert Trigger section

[shaunmcgough]

@andreadelrio one other thing that was mentioned in our meeting was to make it clear through simple text to the user, that this alert will, "Create an alert to be notified when new results match your search" or some similar micro-copy.

Also, after speaking with Aris, the Create alert per functionality can be accomplished with the Grouped functionality taken from the screenshot of a generic alert. In our case, we could set it to over one document to alert anytime that field has a new document that matches, or maybe the user wants to limit that to only when new documents are over five or 100.

Would you mind incorporating this into the Alert trigger section as well, and removing the Create alert per section?

More reading on how this is accomplished on the backend is in the alert instances section of this alerting doc - https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html#alerting-concepts-alert-instances

[AlonaNadler]

Alerts in Discover in my perspective (and the way users express that to me) are similar to a subscribe capability, based on the query users are currently viewing they will get notifications.
I was looking at the mocks and I had a few questions:

1. there are a few use case in my mind and I wasn't sure how to do them based on the alert mocks:

• a user is looking at error logs and wants to get notified in case there are more than 5 alerts in the last 15 minutes
• a user is looking at error logs and wants to get notify on every new record

2. I'm sure how the alert per category works and whether it's useful in the Discover context
3. from UI perspective there are 3 time ranges: check every, notify every, range. It might all be necessary just raising since to me they create confusion.

[shaunmcgough]

Hi @AlonaNadler thanks for the questions. Let me try to answer them. 1 and 2 are related. For 1, we need the group function of the alerting framework in order to accomplish. 2 is being changed from category to group since the functionality already exists. For 3, I somewhat agree. We may be able to combine the check for and alert for to the same field, but we definitely need the range. I will consider the options after researching further. The initial though is that if you have a large dataset, and are pulling all the records from the last day if you only want a days worth of alerts, it would tax the server -vs- if you poll for the last five minutes, consolidate those alerts and then send every day. Regardless, the helper text should clear up what the alert does, which your perspective on from above is correct.

[shaunmcgough]

Ok, so after a bit more research, three time fields are critical for most use cases. Imagine an alert that checks 12 hours, but notifies every five minutes. This means that five minutes after the 12 hour check, you would get an alert. If there is only one field, you would only get an alert every 12 hours. Range is also important so as to enable users to go back 24 hours on a 12 hour alert, or perhaps if they are ingesting historical data or consolidating, perhaps you would need to look back one years and alert every 12 hours and check every 12 hours. The other alerting solutions in Kibana have these three fields, and we are keeping them as part of the design.

[alexfrancoeur]

Heya @shaunmcgough similar to your ask around visualization alerts, I wanted to summarize our quick discussion from our 1:1. Myself and @arisonl have been in a number of discussions with customers and community members around the types of alerts they'd like in Elastic. The search alert comes up frequently with users that have used Splunk at some point, and generally as the most useful alert our base is interested in. Both the Logs application and our threat detection engine have been called out with functionality that users of stack features such as Discover, would like to see generically as part of our alerting capabilities.

The use case being I have a query, and want to be able to receive an alert when more than X results are returned over Y timeframe. There seemed to be confusion during our discussion across the fields in the latest mockups, so I wanted to provide some clarification. @arisonl, please correct me if I'm missing anything.

• Names, tags, check every and re-notify are common across alerts. Check every is the interval at which the alert is running. The mockup you showed had "notify", which I think was a large reason for confusion. Re-notify is our notification throttling. If an alert stays in the same "state", we will notify you again within the interval defined here vs. sending a notification every time the criteria is met.
• Query / filters - this is the core portion of the alert type and we may be able to borrow from prior art (logs, threat engine)
• Threshold - related to @AlonaNadler 's comments and the Logs implementation, this is essentially determining if you want to be alerted every time that criteria is matched, or if a certain number of results are returned within the given timeframe
• Group by - this would define the alert instances. For example, if I run wanted to create an alert where log.message:*error* I could group by host and create a separate alert instance (and notification) for each host that meets that criteria

Hope this helps!

[shaunmcgough]

Thanks, @alexfrancoeur I was just previewing a response to this with our updated notes, so your timing was perfect! @andreadelrio it's safe to say that we should add in the threshold field again, and change notify to re-notify. I believe a simple button should be added to create alert for any new documents would be a good shortcut for folks since this will be a heavy use case. In addition to what Alex states, the Group by will only provide results that meet the group by criteria, so e.g. if a user picks Group by, TOP, X, Fields only the TOP X Fields would be reported. @arisonl I'd lean on you for any clarity as well.

[arisonl]

Alex and Alona cover it but let me add a couple of notes just in case they help clarify certain points in the dicussion:

• The alerting framework involves two time components: the scheduling one (how often the alerting condition is checked) and the throttling one (how often an action such as a notification is produced). A third time component is defined on the alert type level: the time window on which the alerting condition is evaluated (e.g. a statistic or a metric evaluated over a period of time). The first two are determined on the framework level and are exposed on the top section of the flyout (which comes from the framework). The time-picker in Discover or in a visualisation is a fourth time-component and in the current implementations (e.g. Metrics) it does not interact with the evaluation window (afaik).
• The group-over functionality allows you to evaluate the condition per group of documents that share the same value inside a selected field (e.g. same host, same os, same server etc. -conceptually similar to buckets etc.). The way this is done for the generic threshold alert is currently through the top X functionality: It picks the top X such groups, i.e. the X groups that present the highest statistic/metric and then create a separate alert instance for each such group. However the logic with which alert instances are created is entirely up to the alert type, you do not need to follow the top X pattern. For example with the Metrics alert type (see the screenshot pasted above), you can group and create instances using filters. More generally, the logic with which instances are created is up to the alert type developer, in this case the Discover alert type dev. The top X-based groups is just the logic of the generic threshold alert and it is not binding. If it helps, you can mimic and tweak it.
• The main requirement that I have come across with regards to a search alert is the ability to be alerted when a field matches a term and/or phrase in more than X documents (the threshold) within a given period of time (the evaluation window). By extension, you may want to support matching logic against a combination of fields. The current Logs alert implementation covers a number of such requirements, for example see thread. A somewhat extreme flavour involves alerting even based on matching terms in a single document, for the purposes of brand and sentiment monitoring on social media. In this use case the customer needs to be alerted whenever a certain term appears in posts, comments etc. in combination with some type of a brand reference.

[timroes]

Closing in favor of #117532