Support multiple queries while define monitor

[seraphjiang]

Scenario:

I want to setup an alerting, the trigger expression like below

ctx.results[0].aggregations.issueCount.value > ctx.results[1].aggregations.averageCountInPast7Days.value

the first aggregation is just count in current internval (now-interval ~ now)
the second aggregation is average of count(not average of value) in (now-7Days ~ now)

i'm not able to come up a ES query which could achieve my goal .

with current Alerting, i could only given a hard code fix number, it is not very useful in real case.
ctx.results[0].aggregations.issueCount.value > 1800

support multiple queries while define monitor could meet our need. Also this will enable a lot of possibility and use cases.

we are also open for suggestion if this could be done in one query.

[dbbaughe]

This is possible, just not currently allowed.

There is an arbitrary limit set to 1 from these two places:

https://github.com/opendistro-for-elasticsearch/alerting/blob/master/alerting/src/main/kotlin/com/amazon/opendistroforelasticsearch/alerting/settings/AlertingSettings.kt#L29

alerting/alerting/src/main/kotlin/com/amazon/opendistroforelasticsearch/alerting/model/Monitor.kt

Line 70 in b0e2913

require(inputs.size <= MONITOR_MAX_INPUTS) { "Monitors can only have $MONITOR_MAX_INPUTS search input." }

Unfortunately it's hard coded right now so you can't change it for a live domain. But, I did test locally and it worked by querying two indices and having the two different responses available on the ctx variable. Before this can be changed we'd need to get a Kibana UI to support multiple queries though.

[seraphjiang]

That's awesome.

Could we release a version with backend support. I assume we could use API to create multiple queries monitor we want.

We are good without UI ^_^

[dbbaughe]

Yes, it would work directly through API. Unfortunately I don't believe we can release it without the Kibana UI as multiple search inputs might break the UI in the worst case or severely limit user experience in the best case. We'll have to work with UI/UX team to get the needed designs to implement the multiple input support on Kibana first.

[seraphjiang]

Got it, thanks for explanation.

[manuasir]

Waiting for this to be released ASAP!

[qreshi]

Closing in favor of opensearch-project/alerting#80