set monitor alert for all services in an azure resource group (vm's, storage accounts..)

[mikehughesACN]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Currently when using Terraform to create a azurerm_monitor_metric_alert or azurerm_monitor_metric_alertrule it only supports creating alerts for individual resources and not all resources in a resource_group - i.e. I have to create an individual alert rule for every individual virtual machine within a resource_group, I can't create a rule and apply it to all virtual machines within the resource group.

New or Affected Resource(s)

azurerm_metric_alertrule

Potential Terraform Configuration

resource "azurerm_monitor_metric_alert" "test" {
  name                = "example-metricalert"
  resource_group_name = "${azurerm_resource_group.main.name}"
  scopes              = ["/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/resource_group_name/"]
  description         = "Action will be triggered when Transactions count is greater than 50 on all storage accounts with in resource group."

  criteria {
    odata_type = String
    metric_namespace = "Microsoft.Storage/storageAccounts"
    metric_name = "Transactions"
    aggregation = "Total"
    operator = "GreaterThan"
    threshold = 50
  }
}

odata_type = "MultipleResourceMultipleMetricCriteria", "SingleResourceMultipleMetricCriteria"

References

Apparently this is supported in ARM templates using the odata.type metric and configuring it to SingleResourceMultipleMetricCriteria.

"properties": {...
    "criteria": {...
        "odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria"
        ...
    }
}

Documents for the ARM templates can be found here - Azure - static-threshold-alert-on-all-virtual-machines-in-one-or-more-resource-groups. It should also be noted that in this linked ARM template, under the key "targetResourceType" there is a comment in "metadata" a couple lines down that declares "Resource type of target resources to be monitored. Currently only supported resource type is Microsoft.Compute/virtualMachines". This obviously leads me to assume that currently Azure doesn't even support this functionality in ARM templates outside of monitoring all virtual machines within a resource group which may render this request superfluous for the time being.

• set monitor alert for all services in an azure resource group (vm's, storage accounts..) #4337

[gukeqin]

I tried following code and failed as well:

resource "azurerm_monitor_metric_alert" "main" {
name = "${azurerm_virtual_machine.main.name}-alert-cpu"
resource_group_name = "${azurerm_resource_group.main.name}"
scopes = ["${azurerm_resource_group.main.id}"]

severity = 3
enabled = true
frequency = "PT1M"
window_size = "PT1M"

criteria {
metric_namespace = "Microsoft.Compute/virtualMachines"
metric_name = "Percentage CPU"
aggregation = "Average"
operator = "GreaterThan"
threshold = 50
}

action {
action_group_id = "${azurerm_monitor_action_group.main.id}"
}
}

Error:

Error: Error creating or updating metric alert "xxxxx-vm-alert-cpu" (resource group "xxxxx-resources"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Alert scope is invalid."

But I can do the same thing from Portal !!!

[Bobbicus]

It should also be possible to set the scope to subscription level as well.

This is possible via ARM and the Portal and is a valid configuration for example, to set the CPU alert for All VMs in the subscription, it will then apply to any newly created VM resources. The below should be a valid configuration;

resource "azurerm_monitor_metric_alert" "vm-cpu" {
name ="vm-cpu-metric-alert"
resource_group_name = azurerm_resource_group.rg.name
scopes = ["/subscriptions/00000000-0000-0000-0000-000000000000"]
description = "Action will be triggered when Transactions count is greater than 90."
severity = "1"
frequency = "PT5M"
window_size = "PT15M"

criteria {
metric_namespace = "Microsoft.Compute/virtualMachines"
metric_name = "Percentage CPU"
aggregation = "Average"
operator = "GreaterThan"
threshold = 90

}

action {
action_group_id = "${azurerm_monitor_action_group.main.id}"
}
}

Error: Error creating or updating metric alert "vm-cpu-metric-alert" (resource group "rg-plat-uks-dev"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request:
StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Alert scope is invalid."

[estensen]

Found a list of supported newer metrics

Microsoft.Storage/storageAccounts should be supported

[rapster83]

If I try to set the metric "Percentage CPU" (Microsoft.Compute/virtualMachines) for VMs on scopes of subscription level (e.g. /subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e).

Unfortunately, I always get the following error:
Error: Error creating or updating metric alert "alrule-company-metric-vm-cpu-01" (resource group "rg-company-mon-alert-infra"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Scopes property is invalid. Only single resource is allowed for criteria type SingleResourceMultipleMetricCriteria. If you want to create an alert on multiple resources, use MultipleResourceMultipleMetricCriteria odata.type."

But if I use the RG as scopes like:
"/subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e/resourceGroups/rg-company-vm-ad/providers/Microsoft.Compute/virtualMachines/vm-company-advm01"

...everything is working fine.

I need to make sure, if a new VM will be added to the subscription, it must be monitored as well. How can I make sure, that an assignment to a subscription works, like the following screenshot illustrates?

Any recommendations? Thx!

[j2020v]

Is there a way to disable alert for a certain unhandled exception in AppInsights:

System.Threading.Tasks.TaskCanceledException
System.OperationCanceledException

I.e. if exception is one of those we don't need to get alerted?

[mckennapl]

For anyone wondering if they can get around this by deploying an ARM template with Terraform, the resource group variable in the ARM template for resource group level alerting only accepts arrays, which cannot be passed in by Terraform as pointed out in this issue.

Hopefully if anyone is thinking of going down that road I can save them some time and headaches.

[jliang888]

Does anyone have a solution or workaround about it?? I have the same problem, I can't create a rule and apply it to all virtual machines within the resource group. Thanks.

[jliang888]

If I try to set the metric "Percentage CPU" (Microsoft.Compute/virtualMachines) for VMs on scopes of subscription level (e.g. /subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e).

Unfortunately, I always get the following error:
Error: Error creating or updating metric alert "alrule-company-metric-vm-cpu-01" (resource group "rg-company-mon-alert-infra"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Scopes property is invalid. Only single resource is allowed for criteria type SingleResourceMultipleMetricCriteria. If you want to create an alert on multiple resources, use MultipleResourceMultipleMetricCriteria odata.type."

But if I use the RG as scopes like:
"/subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e/resourceGroups/rg-company-vm-ad/providers/Microsoft.Compute/virtualMachines/vm-company-advm01"

...everything is working fine.

I need to make sure, if a new VM will be added to the subscription, it must be monitored as well. How can I make sure, that an assignment to a subscription works, like the following screenshot illustrates?

Any recommendations? Thx!

I tested the same issue, I can't create a rule and apply it to all virtual machines within the resource group.

[jliang888]

If I try to set the metric "Percentage CPU" (Microsoft.Compute/virtualMachines) for VMs on scopes of subscription level (e.g. /subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e).

Unfortunately, I always get the following error:
Error: Error creating or updating metric alert "alrule-company-metric-vm-cpu-01" (resource group "rg-company-mon-alert-infra"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Scopes property is invalid. Only single resource is allowed for criteria type SingleResourceMultipleMetricCriteria. If you want to create an alert on multiple resources, use MultipleResourceMultipleMetricCriteria odata.type."

But if I use the RG as scopes like:
"/subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e/resourceGroups/rg-company-vm-ad/providers/Microsoft.Compute/virtualMachines/vm-company-advm01"

...everything is working fine.

I need to make sure, if a new VM will be added to the subscription, it must be monitored as well. How can I make sure, that an assignment to a subscription works, like the following screenshot illustrates?

Any recommendations? Thx!

Same issue, any workaround?

[jliang888]

I tried following code and failed as well:

resource "azurerm_monitor_metric_alert" "main" {
name = "${azurerm_virtual_machine.main.name}-alert-cpu"
resource_group_name = "${azurerm_resource_group.main.name}"
scopes = ["${azurerm_resource_group.main.id}"]

severity = 3
enabled = true
frequency = "PT1M"
window_size = "PT1M"

criteria {
metric_namespace = "Microsoft.Compute/virtualMachines"
metric_name = "Percentage CPU"
aggregation = "Average"
operator = "GreaterThan"
threshold = 50
}

action {
action_group_id = "${azurerm_monitor_action_group.main.id}"
}
}

Error:

Error: Error creating or updating metric alert "xxxxx-vm-alert-cpu" (resource group "xxxxx-resources"): insights.MetricAlertsClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="Alert scope is invalid."

But I can do the same thing from Portal !!!

Same issue.

[rapster83]

Hi @jliang888 , unfortunately I still do not have a solution to use the subscription level as scope.

[jliang888]

Hi @jliang888 , unfortunately I still do not have a solution to use the subscription level as scope.

Hi, according to the doc, the scopes is supposed to a list, so I tried to put a multiple resource id in the list, like what you did here:
scopes = ["/subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e/resourceGroups/rg-company-vm-ad/providers/Microsoft.Compute/virtualMachines/vm-company-advm01" , "/subscriptions/5f74bd4f-d470-4b4f-983b-72a0fee8ff4e/resourceGroups/rg-company-vm-ad/providers/Microsoft.Compute/virtualMachines/vm-company-advm02"]
It will return you the same error, that's surprised me, have you ever tried successfully?
Thanks.