Bug: ALZ Landing Zone Accelerator Hub/Spoke with VPN Gateway P2S & DNS Issues

[capitanou]

Is there an existing issue for this?

• I have searched the existing issues

Infrastructure as Code Type? (Required)

Terraform

PowerShell Module Version (Optional)

No response

Bootstrap Module Version (Optional)

No response

Starter Module? (Required)

terraform - platform_landing_zone

Starter Module Version (Optional)

singe region hub-spoke

Input arguments of the ALZ-PowerShell-Module (Optional)

No response

Debug Output/Panic Output (Optional)

Expected Behaviour (Required)

When configuring VPN P2S connectivity the VPN client profile generated from the vpn gateway does not include dns_server settings. The result is the macOS & Windows client is not able to resolve host names to azure resources such as storage using nslookup .blob.core.windows.net. to the private ip.

The Client profile can be manually edited to overcome the issue as a temporary workaround by specifying pointing to the Private DNS Resolver on the hub as documented here: https://learn.microsoft.com/en-us/azure/vpn-gateway/azure-vpn-client-optional-configurations

Attempting to push these settings by specifying in the Terraform deployment does not update the VPN Profile. Expected behavior would be for the VPN profile downloaded from the VPN Gateway P2S configuration to include the dns_servers suggested and applied during the terraform apply.

Here's the configuration used below.

platform-landing-zone.tfvars  VPN configuration
vpn = {
    name = "$${primary_virtual_network_gateway_vpn_name}"
    sku  = "VpnGw1AZ"
    #removing this option because it requires Express Route Gateway.
    #vpn_dns_forwarding_enabled = true
    ip_configurations = {
      active_active_1 = {
        # name = "vnetGatewayConfigactive_active_1"  # For backwards compatibility with previous naming, uncomment this line
        public_ip = {
          name = "$${primary_virtual_network_gateway_vpn_public_ip_name_1}"
        }
      }
      active_active_2 = {
        # name = "vnetGatewayConfigactive_active_2"  # For backwards compatibility with previous naming, uncomment this line
        public_ip = {
          name = "$${primary_virtual_network_gateway_vpn_public_ip_name_2}"
        }
      }
      p2s_ip_config = {
        public_ip = {
          name = "$${primary_virtual_network_gateway_vpn_public_ip_name_p2s}"
        }
      }
    }
    vpn_point_to_site = {
      address_space        = ["172.16.0.0/24"]
      vpn_auth_types       = ["AAD"]
      aad_audience         = "41b23e61-6c1e-4545-b367-cd054e0ed4b4"
      aad_issuer           = "https://sts.windows.net/<tenantid>/"
      aad_tenant           = "https://login.microsoftonline.com/<tenantid>/"
      vpn_client_protocols = ["OpenVPN"]
      dns_servers          = ["10.0.0.X"] 
    }
  }
}

Actual Behaviour (Required)

VPN Profile does not contain the dns_server settings provided in the Terraform configuration.

Steps to Reproduce (Optional)

No response

Important Factoids (Optional)

No response

References (Optional)

No response

[jaredfholgate]

Hi @capitanou. I'm not clear if you are saying this is a bug with the accelerator or the Terraform provider is not behaving as expected? Can you provide more concise information on what you feel isn't working and what the expected behaviour should be?

[capitanou]

Hey @jaredfholgate,

The Terraform section in platform-landing-zone.auto.tfvars has been customized to specify dns_servers for the VPN Gateway shown below.

vpn_point_to_site = {
dns_servers = ["10.0.0.X"]

In my case I hard coded this value to the private ip of the Private DNS Resolver which is 10.0.0.164. I was expecting this to push the dns_server to the P2S VPN profile so that VPN clients will use the private resolver to resolve naming for private resources like blob storage attached to the hub via private endpoints.

When I download the VPN profile from the portal and import into the VPN client the client profile azurevpnconfig.xml does not have the dns_servers and the client is not able to resolve to the privatelink resource and instead is resolving the storage to public IP.

Here's a snippet of the azurevpnconfig.xml generated.

I was expecting the dns_server to be in the client profile like this.

10.110.0.164

We can manually edit the profile and this will resolve our dns issue; however, we would prefer to push the setting via terraform.

The Terraform provider shows an optional list of DNS Servers for the Gateway.
dns_servers - (Optional) A list of IP Addresses of DNS Servers for the Point-to-Site VPN Gateway.

I'm very new to terraform and the accelerator. It may be that it would require customization to the accelerator because this is not implemented.

Hoping you can help me to understand why this is occurring? How would we need to customize the repo to achieve this?

[jaredfholgate]

Hey @jaredfholgate,

The Terraform section in platform-landing-zone.auto.tfvars has been customized to specify dns_servers for the VPN Gateway shown below.

vpn_point_to_site = {
dns_servers = ["10.0.0.X"]

In my case I hard coded this value to the private ip of the Private DNS Resolver which is 10.0.0.164. I was expecting this to push the dns_server to the P2S VPN profile so that VPN clients will use the private resolver to resolve naming for private resources like blob storage attached to the hub via private endpoints.

When I download the VPN profile from the portal and import into the VPN client the client profile azurevpnconfig.xml does not have the dns_servers and the client is not able to resolve to the privatelink resource and instead is resolving the storage to public IP.

Here's a snippet of the azurevpnconfig.xml generated.

I was expecting the dns_server to be in the client profile like this.

10.110.0.164

We can manually edit the profile and this will resolve our dns issue; however, we would prefer to push the setting via terraform.

The Terraform provider shows an optional list of DNS Servers for the Gateway.
dns_servers - (Optional) A list of IP Addresses of DNS Servers for the Point-to-Site VPN Gateway.

I'm very new to terraform and the accelerator. It may be that it would require customization to the accelerator because this is not implemented.

Hoping you can help me to understand why this is occurring? How would we need to customize the repo to achieve this?

Hi. I understand now. This is the module we use, it seems to be missing a setting for DNS servers: https://github.com/Azure/terraform-azurerm-avm-ptn-alz-connectivity-hub-and-spoke-vnet/blob/848bcb7baa8fb13d658d19eba030781e1846fd1b/variables.tf#L563

I don't see it in the api docs either: https://learn.microsoft.com/en-us/azure/templates/microsoft.network/virtualnetworkgateways?pivots=deployment-language-terraform

[jaredfholgate]

I don't see it in the azurerm provider resource you shared either? https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/vpn_gateway#dns_servers-1

[capitanou]

Here's the link that I was looking at

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/point_to_site_vpn_gateway

[jaredfholgate]

Here's the link that I was looking at

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/point_to_site_vpn_gateway

For clarity, are you using virtual wan? I thought it was hub and spoke virtual network?

For vwan, I can see it is supported here: https://github.com/Azure/terraform-azurerm-avm-ptn-alz-connectivity-virtual-wan/blob/35d925703302c2f6e944bf828b5595da580d84ee/variables.tf#L169

[capitanou]

We are using hub and spoke currently though I'm starting to think we may need to move to VWAN. Our goal is to utilize the accelerator to support multiple customer deployments. So far I'm impressed with the value it has brought and thank you and the team very much!

Perhaps more requirements and context would help our discussion and determine whether this is a bug or if we may be headed in the wrong direction. We are working with a small company that is a cloud first organization with no on-premise presence. Requirements are for remote employees to connect via P2S VPN to resources in Azure such as azure virtual desktop, storage, etc. There is also a requirement for the remote employees (15 today) to connect to customer environments that require a public ip whitelist for network security into the customer environment. We were hoping this could be done using the P2S gateway and forced tunneling through SNAT on the firewall. In this deployment we hit issues trying to attach a route table to the Gateway subnet. It does not seem supported and looks like internet is black-holed at the Gateway.

That said, I'm still learning and will be humble. My experience comes from the data platform side and I'm growing into the Cloud Infrastructure. It is starting to sound as if this issue is not a bug but is by design to support most common architectures. This may be why the dns_servers and additional routes are not made available in the hub and spoke template. Can you confirm?

For our requirements, is the VWAN the only choice that is possible using native azure components? Is forced tunneling not supported with a traditional VNET gateway combined with Azure Firewall? They are a small company and feel like the VWAN expense is quite high and were hoping the architecture could be achieved via the hub and spoke template combined with the firewall because pricing calculator indicates significantly lower cost. I'm starting to become convinced that VWAN may be their only option.

Finally, is there a good feedback mechanism to submit to gain support for forced tunneling from VNET gateway for those smaller customers.

[jaredfholgate]

We are using hub and spoke currently though I'm starting to think we may need to move to VWAN. Our goal is to utilize the accelerator to support multiple customer deployments. So far I'm impressed with the value it has brought and thank you and the team very much!

Perhaps more requirements and context would help our discussion and determine whether this is a bug or if we may be headed in the wrong direction. We are working with a small company that is a cloud first organization with no on-premise presence. Requirements are for remote employees to connect via P2S VPN to resources in Azure such as azure virtual desktop, storage, etc. There is also a requirement for the remote employees (15 today) to connect to customer environments that require a public ip whitelist for network security into the customer environment. We were hoping this could be done using the P2S gateway and forced tunneling through SNAT on the firewall. In this deployment we hit issues trying to attach a route table to the Gateway subnet. It does not seem supported and looks like internet is black-holed at the Gateway.

That said, I'm still learning and will be humble. My experience comes from the data platform side and I'm growing into the Cloud Infrastructure. It is starting to sound as if this issue is not a bug but is by design to support most common architectures. This may be why the dns_servers and additional routes are not made available in the hub and spoke template. Can you confirm?

For our requirements, is the VWAN the only choice that is possible using native azure components? Is forced tunneling not supported with a traditional VNET gateway combined with Azure Firewall? They are a small company and feel like the VWAN expense is quite high and were hoping the architecture could be achieved via the hub and spoke template combined with the firewall because pricing calculator indicates significantly lower cost. I'm starting to become convinced that VWAN may be their only option.

Finally, is there a good feedback mechanism to submit to gain support for forced tunneling from VNET gateway for those smaller customers.

Having spoken to my more experienced colleagues, it may be the case that this setting is missing from our docs, but it does exist in the ARM API. I would need to do a deployment and testing to validate and find out what it is called in order to add it into the module(s). I am unsure when I'll have time to do this, but will try to combine it with some other fixes to the connectivity module.

As such, I don't believe you are limited for vwan for your use case, but need to confirm.