Jchomarat/purview terraform recipe

.github/workflows/purview-recipe-ci.yml

@@ -0,0 +1,51 @@
+name: Microsoft Purview Recipe CI
+
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - src/az-purview/deploy/**
+      - .github/workflows/purview-recipe-ci.yml
+  pull_request:
+    branches: [main]
+
+permissions:
+  id-token: write
+  contents: read
+
+env:
+  AZURE_REGION: eastus
+
+jobs:
+  validate-bicep:
+    runs-on: ubuntu-latest
+    name: Validate Terraform
+    environment: ${{ github.event.repository.name}}
+    steps:
+      - uses: actions/checkout@v3
+      - uses: ./.github/actions/validate-terraform
+        with:
+          azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+          azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
+          azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
+          azure-resource-group-name: "rg-purview-gh-bicep"
+          azure-resource-group-location: ${{ env.AZURE_REGION }}
+          bicep-path: ./src/az-purview/deploy/bicep/main.bicep
+          bicep-parameters: "location=${{ env.AZURE_REGION }} vnetAddressPrefix=10.11.0.0/16 privateEndpointSubnetAddressPrefix=10.11.0.0/24 bastionSubnetAddressPrefix=10.11.1.0/24 containerSubnetAddressPrefix=10.11.2.0/24 hostSubnetAddressPrefix=10.11.3.0/24 dnsZoneResourceGroupName=rg-scenario-operations newOrExistingDnsZones=existing"
+
+  validate-terraform:
+    runs-on: ubuntu-latest
+    name: Validate Terraform configuration
+    environment: ${{ github.event.repository.name}}
+    steps:
+      - uses: actions/checkout@v3
+      - uses: ./.github/actions/validate-terraform
+        env:
+          RESOURCE_GROUP_NAME: rg-purview-gh-tf
+        with:
+          azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+          azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
+          azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
+          terraform_version: "1.3.2"
+          terraform_variables: -var "resource_group_name=${{ env.RESOURCE_GROUP_NAME }}" -var "dns_zone_resource_group_name=${{ env.RESOURCE_GROUP_NAME }}" -var "new_or_existing_dns_zones=new" -var "location=${{ env.AZURE_REGION }}" -var "tags={environment=\"Azure-Samples\"}" -var "dns_zone_resource_group_subscription_id=\"\"" -var "vnet_address_prefix=\"10.14.0.0/16\"" -var "bastion_subnet_address_prefix=\"10.14.1.0/24\"" -var "private_endpoint_subnet_address_prefix=\"10.14.0.0/24\""
+          working_directory: ./src/az-purview/deploy/terraform/

.gitignore

@@ -404,4 +404,10 @@ src/*/deploy/.certs
 src/*/deploy/bicep/azuredeploy.parameters.json
 
 # Ignore .vscode settings
-.vscode/settings.json
+.vscode/settings.json
+
+# Ignore personal folders
+me/
+
+# Ignore OSX files
+.DS_Store

src/az-purview/README.md

@@ -46,6 +46,7 @@ The following pre-requisites should be in place in order to successfully use thi
 
 - [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli)
 - [Bicep](https://docs.microsoft.com/azure/azure-resource-manager/bicep/install) (Only if using Azure Bicep)
+- [Terraform](https://developer.hashicorp.com/terraform/downloads?product_intent=terraform) (Only if using Azure Bicep)
 - [Azure PowerShell](https://docs.microsoft.com/powershell/azure/install-az-ps) (Only if using Azure PowerShell to deploy via Azure Bicep)
 
 ### Deployment
@@ -54,11 +55,15 @@ There are three parts in this deployment:
 
 1. Deploy main Azure Purview recipe.
 2. Create Self-Hosted Integration Runtime (SHIR) in Azure Purview.
-3. Deploy VM scaleset to act as the Purview SHIR.
+3. Deploy VM scale set to act as the Purview SHIR.
 
 #### 1. Deploy main Azure Purview Recipe
 
-To deploy this recipe, please perform the following actions:
+This recipe is available in Bicep and Terraform.
+
+#### Using Bicep
+
+To deploy this recipe using Bicep, please perform the following actions:
 
 - Create a new Azure resource group to deploy the Bicep template, passing in a location and name.
 
@@ -80,6 +85,30 @@ az deployment group what-if --resource-group <RESOURCE_GROUP_NAME> --template-fi
 az deployment group create --resource-group <RESOURCE_GROUP_NAME> --template-file .\main.bicep --parameters @.\azuredeploy.parameters.json --verbose
 ```
 
+#### Using Terraform
+
+To deploy this recipe using Terraform, please perform the following actions:
+
+- The [terraform.tfvars.sample](./deploy/terraform/terraform.tfvars.sample) file contains the necessary variables to deploy the Terraform project. Rename the file to **terraform.tfvars** and update the file with appropriate values. Descriptions for each parameter can be found in the [variables.tf](./deploy/terraform/variables.tf) file.
+
+- Initialize the working directory containing Terraform configuration files.
+
+```bash
+terraform init
+```
+
+- Optionally, verify what Terraform will deploy, passing the necessary parameters.
+
+```bash
+terraform plan -var-file=terraform.tfvars
+```
+
+- Deploy the resources with the necessary parameters.
+
+```bash
+terraform apply -var-file=terraform.tfvars
+```
+
 #### 2. Create Jumpbox to enabled deployment of Self-Hosted Integration Runtime (SHIR) in Azure Purview
 
 Because the Azure Purview portal has public access disabled, these steps needs to be executed from a Virtual Machine (VM) in a VNet which has a network-line-of-site to Azure Purview. For simplicity, you can deploy this VM in the same application VNet which has been created as part of Bicep deployment as it has the required networking setup. Please follow the [Azure Documentation](https://docs.microsoft.com/azure/virtual-machines/linux/quick-create-portal) for detailed instructions.
@@ -149,4 +178,4 @@ Describe the change history for this recipe. For example:
 ## Next Steps
 
 You can read the official documentation on Network Architecture best practices for Azure Purview [here](https://docs.microsoft.com/azure/purview/concept-best-practices-network)
-<!-- Provide description and links to what a user of this recipe could do next.  Include suggestions for how the recipe could be enhanced or built upon. -->
+<!-- Provide description and links to what a user of this recipe could do next.  Include suggestions for how the recipe could be enhanced or built upon. -->

src/az-purview/deploy/terraform/.tflint.hcl

@@ -0,0 +1,9 @@
+config {
+    module = true
+}
+
+plugin "azurerm" {
+    enabled = true
+    version = "0.10.1"
+    source  = "github.com/terraform-linters/tflint-ruleset-azurerm"
+}

src/az-purview/deploy/terraform/deploy.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+set -o errexit
+set -o pipefail
+set -o nounset
+
+# The name of the resource group to be created for the deployment.
+RESOURCE_GROUP_NAME=[YOUR-RESOURCE-GROUP-NAME]
+
+# The name of the operations resource group (used in a hub/spoke vnet model).
+OPERATIONS_RESOURCE_GROUP_NAME=[YOUR-OPERATIONS-RESOURCE-GROUP-NAME]
+
+# The name of the existing virtual network within the OPERATIONS_RESOURCE_GROUP_NAME resource group; used to establish a peering relationship.
+OPERATIONS_VNET_NAME=[YOUR-OPERATIONS-VIRUTAL-NETWORK-NAME]
+
+# Flag to indicate if virtual network peering should be established. Set to "true" if peering to the OPERATIONS_VNET_NAME virtual network.
+PEER_VNET="false"
+
+# !!!! NOTE !!!! 
+# Overriding the resource group values in terraform.tfvars file to ensure resoruce group values set in script are used
+# since the resource group values are used in the Azure CLI commands below to peer virtual network.
+
+terraform init -input=false
+terraform apply -var resource_group_name="$RESOURCE_GROUP_NAME" -var dns_zone_resource_group_name="$OPERATIONS_RESOURCE_GROUP_NAME" -input=false -auto-approve
+
+if [[ $PEER_VNET == "true" ]]; then
+    echo "Retrieving virtual network name created in Function Private HTTP recipe deployment . . ."
+
+    RECIPE_VNET_NAME="$(terraform output -json | jq -r '.vnet_name.value')"
+
+    echo "Retrieved virtual network name: '$RECIPE_VNET_NAME'."
+
+    # Hub vnet to the recipe vnet.
+    echo "Establishing peering relationship: '$OPERATIONS_VNET_NAME' to '$RECIPE_VNET_NAME' . . ."
+    az deployment group create \
+        --resource-group "$OPERATIONS_RESOURCE_GROUP_NAME" \
+        --template-file '../../../common/infrastructure/bicep/vnet-peering.bicep' \
+        --parameters sourceVnetName="$OPERATIONS_VNET_NAME" targetVnetName="$RECIPE_VNET_NAME" targetResourceGroup="$RESOURCE_GROUP_NAME"
+
+    # Recipe vnet to the hub vnet
+    echo "Establishing peering relationship: '$RECIPE_VNET_NAME' to '$OPERATIONS_VNET_NAME' . . ."
+    az deployment group create \
+        --resource-group "$RESOURCE_GROUP_NAME" \
+        --template-file '../../../common/infrastructure/bicep/vnet-peering.bicep' \
+        --parameters sourceVnetName="$RECIPE_VNET_NAME" targetVnetName="$OPERATIONS_VNET_NAME" targetResourceGroup="$OPERATIONS_RESOURCE_GROUP_NAME"
+fi

src/az-purview/deploy/terraform/locals.tf

@@ -0,0 +1,58 @@
+locals {
+  // Generating unique name to be added to the resource names
+  base_name           = random_string.base_name.result
+  pe_base_name_suffix = local.base_name
+
+  purview_account_name = "pview-${local.base_name}"
+
+  // Private DNS Zones Map
+  private_dns_names_map = {
+    "ServiceBus"    = "privatelink.servicebus.windows.net"
+    "PurviewStudio" = "privatelink.purviewstudio.azure.com"
+    "Purview"       = "privatelink.purview.azure.com"
+    "Vault"         = "privatelink.vaultcore.azure.net"
+    "Blob"          = "privatelink.blob.core.windows.net"
+    "Dfs"           = "privatelink.dfs.core.windows.net"
+    "Queue"         = "privatelink.queue.core.windows.net"
+  }
+
+  purview_private_dns_zone_id_map = {
+    "namespace" = {
+      "dns_zone_id" = module.dns.purview_private_dns_zone_id
+    }
+
+    "portal" = {
+      "dns_zone_id" = module.dns.purview_private_dns_zone_id
+    }
+
+    "account" = {
+      "dns_zone_id" = module.dns.purview_studio_private_dns_zone_id
+    }
+
+    "blob" = {
+      "dns_zone_id" = module.dns.blob_private_dns_zone_id
+    }
+
+    "queue" = {
+      "dns_zone_id" = module.dns.queue_private_dns_zone_id
+    }
+  }
+
+  // VNet variables
+  virtual_network_name         = "vnet-${local.base_name}"
+  private_endpoint_subnet_name = "snet-${local.base_name}-pe"
+  azure_bastion_subnet_name    = "AzureBastionSubnet"
+
+  // Azure Storage variables
+  storage_account_name   = "st${local.base_name}"
+  default_container_name = "container001"
+  private_dns_names_map_for_storage = {
+    "blob" = "privatelink.blob.core.windows.net"
+    "dfs"  = "privatelink.dfs.core.windows.net"
+  }
+
+  // Azure Key Vault variables
+  key_vault_name     = "kv-${local.base_name}"
+  keyvault_sku_name  = "standard"
+  azurerm_pe_kv_name = "pe-kv-vault-${local.base_name}"
+}