Network security groups for TRE core subnets

docs/architecture.md

@@ -94,39 +94,6 @@ The flow to provision a Workspace is as follows (the flow is the same for all ki
 1. The status of a Porter bundle execution is received.
 1. The status of a Porter bundle execution is updated in the Configuration Store.
 
-## Network Architecture
+## Network architecture
 
-The network topology is based on [hub-spoke](https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/hybrid-networking/hub-spoke). The TRE Management VNET ([Azure Virtual Network](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview)) is the central hub and each Workspace are spokes.
-
-> **Note:** TRE Management is referred to as Core in scripts and code.
-
-![Network Architecture](./assets/network-architecture.png)
-
-Azure TRE VNETs are segregated allowing limited traffic between the TRE Management VNET and Workspace VNETs. The rules are managed in the `nsg-ws` Network Security Group (NSG):
-
-- Inbound traffic from TRE Management VNET to Workspace allowed for [Azure Bastion](https://docs.microsoft.com/en-us/azure/bastion/bastion-overview) (22, 3389) - All other inbound traffic from Core to Workspace denied.
-- Outbound traffic to `SharedSubnet` from Workspace allowed.
-- Outbound traffic to Internet allowed on HTTPS port 443 (next hop Azure Firewall).
-- All other outbound traffic denied.
-
-> In Azure traffic between subnets are allowed except explicitly denied.
-
-Each of these rules can be managed per Workspace.
-
-Each Workspace has a default route routing all egress traffic through the Azure Firewall, to ensure only explicitly allowed destinations on the Internet to be accessed. It is planned that all other subnet will use the same pattern (Issue [#421](https://github.com/microsoft/AzureTRE/issues/421))
-
-The Azure Firewall rules are:
-
-- No default inbound rules – block all.
-- No default outbound rules – block all.
-
-Inbound traffic from the Internet is only allowed through the Application Gateway, which forwards HTTPS (port 443) call to the TRE API in the `WebAppSubnet`.
-
-| Subnet | Description |
-| -------| ----------- |
-| `AzureBastionSubnet` | A dedicated subnet for Azure Bastion hosts. |
-| `AppGwSubnet` | Subnet for Azure Application Gateway controlling ingress traffic. |
-| `AzureFirewallSubnet` | Subnet for Azure Firewall controlling egress traffic. |
-| `ResourceProcessorSubnet` | Subnet for VMSS used by the Composition Service to host Docker containers to execute Porter bundles that deploys Workspaces. |
-| `WebAppSubnet` | Subnet for TRE API. |
-| `SharedSubnet` | Shared Services subnet for all things shared by TRE Management and Workspaces. Future Shared Services are Firewall Shared Service, Source Mirror Shared Service and Package Mirror Shared Service. |
+See [networking](./networking.md).

docs/index.md

@@ -4,6 +4,7 @@
   * [Concepts](./concepts.md)
   * [User roles](./user-roles.md)
   * [Architecture](./architecture.md)
+  * [Networking](./networking.md)
   * [Logical data model](./logical-data-model.md)
 * Getting started
   * [Dev environment](./dev-environment.md)

docs/networking.md

@@ -0,0 +1,52 @@
+# Networking
+
+The Trusted Research Environment (TRE) network topology is based on [hub-spoke](https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/hybrid-networking/hub-spoke). The TRE Management VNET ([Azure Virtual Network](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview)) is the central hub and each workspace is a spoke.
+
+> Note: TRE Management is referred to as **core** in scripts and code.
+
+![Network architecture](./assets/network-architecture.png)
+
+Azure TRE VNETs are segregated allowing limited traffic between the TRE Management VNET and Workspace VNETs. The security rules are managed by `nsg-ws` network security group. See [workspace network security groups (NSG)](#workspaces) further down.
+
+Each workspace has a default route routing all egress traffic through the Azure Firewall, to ensure only explicitly allowed destinations on the Internet to be accessed. It is planned that all other subnet will use the same pattern (Issue [#421](https://github.com/microsoft/AzureTRE/issues/421))
+
+The Azure Firewall rules are:
+
+- No default inbound rules – block all.
+- No default outbound rules – block all.
+
+Inbound traffic from the Internet is only allowed through the Application Gateway, which forwards HTTPS (port 443) call to the TRE API in the `WebAppSubnet`.
+
+| Subnet | Description |
+| -------| ----------- |
+| `AzureBastionSubnet` | A dedicated subnet for Azure Bastion hosts. |
+| `AppGwSubnet` | Subnet for Azure Application Gateway controlling ingress traffic. |
+| `AzureFirewallSubnet` | Subnet for Azure Firewall controlling egress traffic. |
+| `ResourceProcessorSubnet` | Subnet for VMSS used by the Composition Service to host Docker containers to execute Porter bundles that deploys Workspaces. |
+| `WebAppSubnet` | Subnet for TRE API. |
+| `SharedSubnet` | Shared Services subnet for all things shared by TRE Management and Workspaces. Future Shared Services are Firewall Shared Service, Source Mirror Shared Service and Package Mirror Shared Service. |
+
+## Network security groups
+
+### TRE Management/core
+
+Network security groups (NSG) and their security rules for TRE core resources are defined in [`/templates/core/terraform/network/network_security_groups.tf`](../templates/core/terraform/network/network_security_groups.tf).
+
+| Network security group | Associated subnet(s) |
+| ---------------------- | -------------------- |
+| `nsg-bastion-subnet` | `AzureBastionSubnet` |
+| `nsg-app-gw` | `AppGwSubnet` |
+| `nsg-default-rules` | `ResourceProcessorSubnet`, `SharedSubnet`, `WebAppSubnet` |
+
+### Workspaces
+
+Azure TRE VNETs are segregated allowing limited traffic between the TRE Management VNET and Workspace VNETs. The rules to manage and limit the traffic between the TRE Management VNET and Workspace VNETs are defined by the `nsg-ws` network security group:
+
+- Inbound traffic from TRE Management VNET to workspace allowed for [Azure Bastion](https://docs.microsoft.com/en-us/azure/bastion/bastion-overview) (22, 3389) - All other inbound traffic from Core to workspace denied.
+- Outbound traffic to `SharedSubnet` from Workspace allowed.
+- Outbound traffic to Internet allowed on HTTPS port 443 (next hop Azure Firewall).
+- All other outbound traffic denied.
+
+> In Azure, traffic between subnets are allowed except explicitly denied.
+
+Each of these rules can be managed per workspace.

templates/core/terraform/main.tf

@@ -63,17 +63,17 @@ module "storage" {
   tre_id              = var.tre_id
   location            = var.location
   resource_group_name = azurerm_resource_group.core.name
-  shared_subnet       = module.network.shared
-  core_vnet           = module.network.core
+  shared_subnet       = module.network.shared_subnet_id
+  core_vnet           = module.network.core_vnet_id
 }
 
 module "appgateway" {
   source                 = "./appgateway"
   tre_id                 = var.tre_id
   location               = var.location
   resource_group_name    = azurerm_resource_group.core.name
-  app_gw_subnet          = module.network.app_gw
-  shared_subnet          = module.network.shared
+  app_gw_subnet          = module.network.app_gw_subnet_id
+  shared_subnet          = module.network.shared_subnet_id
   api_fqdn               = module.api-webapp.api_fqdn
   keyvault_id            = module.keyvault.keyvault_id
   static_web_dns_zone_id = module.network.static_web_dns_zone_id
@@ -85,10 +85,10 @@ module "api-webapp" {
   tre_id                                     = var.tre_id
   location                                   = var.location
   resource_group_name                        = azurerm_resource_group.core.name
-  web_app_subnet                             = module.network.web_app
-  shared_subnet                              = module.network.shared
-  app_gw_subnet                              = module.network.app_gw
-  core_vnet                                  = module.network.core
+  web_app_subnet                             = module.network.web_app_subnet_id
+  shared_subnet                              = module.network.shared_subnet_id
+  app_gw_subnet                              = module.network.app_gw_subnet_id
+  core_vnet                                  = module.network.core_vnet_id
   app_insights_connection_string             = azurerm_application_insights.core.connection_string
   app_insights_instrumentation_key           = azurerm_application_insights.core.instrumentation_key
   log_analytics_workspace_id                 = azurerm_log_analytics_workspace.core.id
@@ -125,7 +125,7 @@ module "resource_processor_vmss_porter" {
   resource_group_name                             = azurerm_resource_group.core.name
   acr_id                                          = data.azurerm_container_registry.mgmt_acr.id
   app_insights_connection_string                  = azurerm_application_insights.core.connection_string
-  resource_processor_subnet_id                    = module.network.resource_processor
+  resource_processor_subnet_id                    = module.network.resource_processor_subnet_id
   docker_registry_server                          = var.docker_registry_server
   resource_processor_vmss_porter_image_repository = var.resource_processor_vmss_porter_image_repository
   resource_processor_vmss_porter_image_tag        = var.resource_processor_vmss_porter_image_tag
@@ -148,17 +148,17 @@ module "servicebus" {
   tre_id                       = var.tre_id
   location                     = var.location
   resource_group_name          = azurerm_resource_group.core.name
-  core_vnet                    = module.network.core
-  resource_processor_subnet_id = module.network.resource_processor
+  core_vnet                    = module.network.core_vnet_id
+  resource_processor_subnet_id = module.network.resource_processor_subnet_id
 }
 
 module "keyvault" {
   source                     = "./keyvault"
   tre_id                     = var.tre_id
   location                   = var.location
   resource_group_name        = azurerm_resource_group.core.name
-  shared_subnet              = module.network.shared
-  core_vnet                  = module.network.core
+  shared_subnet              = module.network.shared_subnet_id
+  core_vnet                  = module.network.core_vnet_id
   tenant_id                  = data.azurerm_client_config.current.tenant_id
   managed_identity_tenant_id = module.identity.managed_identity.tenant_id
   managed_identity_object_id = module.identity.managed_identity.principal_id
@@ -186,8 +186,8 @@ module "routetable" {
   tre_id                       = var.tre_id
   location                     = var.location
   resource_group_name          = azurerm_resource_group.core.name
-  shared_subnet_id             = module.network.shared
-  resource_processor_subnet_id = module.network.resource_processor
+  shared_subnet_id             = module.network.shared_subnet_id
+  resource_processor_subnet_id = module.network.resource_processor_subnet_id
   firewall_private_ip_address  = module.firewall.firewall_private_ip_address
 }
 
@@ -196,16 +196,16 @@ module "state-store" {
   tre_id              = var.tre_id
   location            = var.location
   resource_group_name = azurerm_resource_group.core.name
-  shared_subnet       = module.network.shared
-  core_vnet           = module.network.core
+  shared_subnet       = module.network.shared_subnet_id
+  core_vnet           = module.network.core_vnet_id
 }
 
 module "bastion" {
   source              = "./bastion"
   tre_id              = var.tre_id
   location            = var.location
   resource_group_name = azurerm_resource_group.core.name
-  bastion_subnet      = module.network.bastion
+  bastion_subnet      = module.network.bastion_subnet_id
 }
 
 module "gitea" {

templates/core/terraform/network/network.tf

@@ -48,41 +48,6 @@ resource "azurerm_subnet" "web_app" {
   }
 }
 
-resource "azurerm_subnet" "aci" {
-  name                                           = "AciSubnet"
-  virtual_network_name                           = azurerm_virtual_network.core.name
-  resource_group_name                            = var.resource_group_name
-  address_prefixes                               = [local.aci_subnet_address_prefix]
-  enforce_private_link_endpoint_network_policies = true
-  enforce_private_link_service_network_policies  = true
-
-  delegation {
-    name = "acidelegationservice"
-
-    service_delegation {
-      name    = "Microsoft.ContainerInstance/containerGroups"
-      actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
-    }
-  }
-}
-
-resource "azurerm_network_profile" "aciprofile" {
-  name                = "aciprofile"
-  location            = var.location
-  resource_group_name = var.resource_group_name
-
-  container_network_interface {
-    name = "acr-frontal-nic"
-
-    ip_configuration {
-      name      = "acrfrontal"
-      subnet_id = azurerm_subnet.aci.id
-    }
-  }
-
-  lifecycle { ignore_changes = [tags] }
-}
-
 resource "azurerm_subnet" "shared" {
   name                 = "SharedSubnet"
   virtual_network_name = azurerm_virtual_network.core.name