Help with the "right" way to build idempotent Bicep files for resources with network intents... #5839
-
Recently I've been trying to add Azure Data Explorer into my deployment. It requires an existing virtual network, a network security group, and a route table. Subnet delegation is enabled on the subnet which uses network intents to apply the appropriate NSG rules and add the appropriate routes to the route table. This is all works great on the first deployment, but I'm really struggling with how I can make this idempotent. It always fails on the second deployment with conflicts for the NSG and route table. I'm just trying to define an empty route table and let subnet delegation do its thing: resource routeTable 'Microsoft.Network/routeTables@2021-05-01' = { I'm not trying to define any rules, but on the second deployment, I get dozens of these errors: Route Table doesn't have supporting Route for Network Intent Policy Route: Name: 52_224_146_56_32, Id: /subscriptions//resourceGroups//providers/Microsoft.Network/networkIntentPolicies/Microsoft_Kusto_clusters-adx-subnet/routes/52_224_146_56_32, AddressPrefix: 52.224.146.56/32, NextHopType: Internet, NextHopIpAddress: \\\\r\\\\n ----\\\\r\\\\n I've seen similar discussions (regarding Databricks) advising people to just add all of those rules into their templates. But that seems to me to defeat the whole purpose of delegation. The ADX team could change their NSG rules or any of the IP addresses in the route table at any time. I don't think I want to be adding all that into my template. The issue with the NSG is essentially the same. I'm trying to define an empty NSG and then define my own rules (about a dozen proxy IP addresses that I need to allow) as separate Microsoft.Network/networkSecurityGroups/securityRules resources. resource adxNsg 'Microsoft.Network/networkSecurityGroups@2021-05-01' = { Errors: I'm sure I could work around this by adding a flag to the deployment if it's the first deployment, and then reference existing resources if it's not. I'm just curious how other people are getting this to work. Is there a better way to handle the subnet delegation / network intents? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
I am not familiar with this scenario. It seems like this is something that will not work well with Bicep declarative deployments.
A similar scenario is with Just In Time (JIT) policies and NSG. The way that I manage it is as following.
By doing this it's easy to manage and Automate. All prefered NSG rules are on the NSG that is attached to the Subnet, so the dedicated NIC NSG is just for the policy. By doing this can you continue to deploy without any conflict. E.g. // NSG without any rules
param Deployment string
param VM object
resource vmJMP 'Microsoft.Network/networkSecurityGroups@2021-05-01' = {
name: '${Deployment}-vm${VM.Name}-JITNSG'
location: resourceGroup().location
} Not related to your issue, however a later module that runs to create the JIT policy on the NSG. https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/x.vmJIT.bicep In this scenario both of these run from the VM provisioning module e.g. |
Beta Was this translation helpful? Give feedback.
-
Just in case it helps anyone in future:
I totally agree. I managed to solve this (to a degree) by:
This has worked - deployments are now successful, and any changes I make get realized. For the route table, for example: // Network.bicep
// Get reference to the existing route table
resource ref_existingRouteTable 'Microsoft.Network/routeTables@2022-11-01' existing = {
name: routeTableName
scope: resourceGroup(resourceGroupName)
}
// Filter existing routes to get only the routes added by SQL MI subnet delegation.
var sqlDelegatedRoutes = filter(ref_existingRouteTable.properties.routes, route => contains(route.name, 'Microsoft.Sql-'))
// Define custom routes
var customRoutes = [
{
// Set Default route to go through virtual appliance
// Overrides Azure system default routes that sends default traffic to nextHopType: Internet
name: 'DefaultRoute'
properties: {
addressPrefix: '0.0.0.0/0'
nextHopType: 'VirtualAppliance'
nextHopIpAddress: VirtualApplianceIpAddress
}
}
]
// Combine custom routes with existing sql routes, to get the routes to deploy
var routesToDeploy = union(sqlDelegatedRoutes, customRoutes)
// Create Route Table using library resource module
module mod_routeTable '..\library\resources\network\routeTable.resource.bicep' = {
name: 'mod_RouteTable'
scope: resourceGroup(resourceGroupName)
params: {
name: routeTableName
location: location
routes: routesToDeploy
tags: tags
}
}
// routeTable.resource.bicep
resource res_routeTable 'Microsoft.Network/routeTables@2022-07-01' = {
name: name
location: location
properties: {
disableBgpRoutePropagation: disableBgpRoutePropagation
routes: routes
}
tags: tags
}
Notes/Limitations
|
Beta Was this translation helpful? Give feedback.
I am not familiar with this scenario. It seems like this is something that will not work well with Bicep declarative deployments.
A similar scenario is with Just In Time (JIT) policies and NSG.
The way that I manage it is as following.
By doing this it's easy to manage and Automate.
All prefered NSG rules are on the NSG that is attached to the Subnet, so the dedicated NIC NSG is just for the policy. By doing this can you continue to deploy without any conflict.
E.g.
htt…