Deployments failing on App GW NSG rule validation after initial deployment

[biggles007]

This is autogenerated. Please review and update as needed.

Describe the bug

After initial greenfield deployment, future deployments fail on Application Gateway NSG validation, previously reported in #21256 and was seemingly working again in a previous version.

Command Name
az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup\\\",\\r\\n    \\\"message\\\": \\\"Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.\\\",\\r\\n    \\\"details\\\": []\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behaviour. Note that argument values have been redacted, as they may contain sensitive information.

Run the deployment a second time

Expected Behaviour

Deployment should succeed

Environment Summary

Windows-10-10.0.19043-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.36.0

Additional Context

[yonzhan]

ARM

[zhoxing-ms]

Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.

@bigglesuk69 Could you see if this error message is helpful to you?

[biggles007]

@zhoxing-ms the rules are on the NSG, I have validated they are correct, so the error message is wrong. The issue is somewhere in the validation. To test I did the following.

Deleted the App GW, ran the deployment, successful
Run the deployment a second time, error is shown.
At all times the NSG has the correct rule.

NSG is deployed with no rules, then we inject in the pre-requisite rules using the Microsoft.Network/networkSecurityGroups/securityRules resource. If the validation only checking the Microsoft.Network/networkSecurityGroups resource for the rules? If so, that is a bug.

[biggles007]

Some further information on this issue. I have a master bicep template that calls child modules for the deployments. So the vnet/NSG is deployed in one deployment, and the app GW in another. So what is happening is:

• vnet deployment runs first time and succeeds (no AppGW deployed)
• AppGW deployment runs successfully once the vnet deployment has completed
• vnet deployment fails on all further runs

[zhoxing-ms]

This error message is returned by the REST service, so this is a service issue

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @josephkwchan, @jennyhunter-msft.

Issue Details

---

This is autogenerated. Please review and update as needed.

Describe the bug

After initial greenfield deployment, future deployments fail on Application Gateway NSG validation, previously reported in #21256 and was seemingly working again in a previous version.

Command Name
az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup\\\",\\r\\n    \\\"message\\\": \\\"Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.\\\",\\r\\n    \\\"details\\\": []\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behaviour. Note that argument values have been redacted, as they may contain sensitive information.

Run the deployment a second time

Expected Behaviour

Deployment should succeed

Environment Summary

Windows-10-10.0.19043-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.36.0

Additional Context

Author:	bigglesuk69
Assignees:	zhoxing-ms
Labels:	Service Attention, ARM, customer-reported, Auto-Assign
Milestone:	Backlog