"Large shard size" Stack Monitoring rule is missing "Look at the average over X minutes" option

[ravikesarwani]

• Add "Look at the average over X minutes" option to "Large shard size" rule with default value of 15 minutes.
• Change the default value for the "Large shard size" rule from 55gb to 75gb

The documentation of stack monitoring alert for Large shard size mentions "The condition is met if an index’s average shard size is 55gb or higher in the last 5 minutes" but the parameter to specify the time period is somehow missing on the rule definition.

We don't want a single spike over 55gb for primary shard size to cause an alert.
Force merges can cause the shard to grow much more than 50 GB (in some cases may double) for a short while and potentially trigger an alert that would be considered false positive.
We want the alert to fire only when size in last X minutes (default 15 minutes) averages over 75gb.
This provides additional control point for the users and avoids unneeded noise at time.

This would be similar to "Disk usage" rule.

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[elasticmachine]

Pinging @elastic/stack-monitoring (Team:Monitoring)

[ravikesarwani]

Force merges can cause the shard to grow much more than 50 GB for a short while and potentially trigger an alert that would be considered false positive. The change in this issue (where we check for average over last X minute) will help with this temporary condition.

[elasticmachine]

Pinging @elastic/infra-monitoring-ui (Team:Monitoring)