#35 - Add remote backend to track deployment state

shared/README.md

@@ -4,28 +4,68 @@
 ## Requirements
 
 - Azure Subscription
-- Service Principal
 - [Terraform](https://www.terraform.io/downloads.html)
 
+Fill the values required below in backend.tfvars to initialize Azure backend for Terraform, using the information [below](###Configure-Terraform-to-Store-State-Data-in-Azure)
+
+- Storage Account Name
+- Storage Account Access Key
+- Storage Account Container Name
+- Key Name to store for Terraform State for Test Environment
+
+
 ## Resources
 
-The following resources will be deployed
+
+The following resources will be deployed:
 - Azure Resource Group
 - Azure KeyVault 
 
 ## Deployment
 
-1. Authenticate using your Azure Principal or an Azure account with privileges to deploy resource groups.
+### Authenticate using your Azure Principal or an Azure account with privileges to deploy resources.
 
 ``` bash
 $ az login
 ```
 
-2. Execute the following commands:
+### Configure Terraform to Store State Data in Azure
+
+Terraform records the information about what is created in a [Terraform state file](https://www.terraform.io/docs/state/) after it finishes applying.  By default, Terraform will create a file named `terraform.tfstate` in the directory where Terraform is applied.  Terraform needs this information so that it can be loaded when we need to know the state of the infrastructure for future modifications.
+
+In production scenarios, storing the state file on a local file system is not desired because typically you want to share the state between operators of the system.  Instead, we configure Terraform to store state remotely, and in Cobalt we use Azure Blob Store for this.  This is defined using a `backend` block.  The basic block looks like:
+
+```bash
+terraform {
+   backend “azure” {
+   }
+}
+```
+
+In order to setup an Azure backend, one needs an Azure Storage account.  If one must be provisioned, use [this](https://docs.microsoft.com/en-us/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-create) link to provision, where `name` is the name of the storage account to store the Terraform state, `location` is the Azure region the storage account should be created in, and `resource group` is the name of the resource group to create the storage account in.  
+
+If there is already a pre-existing storage account, then retrieve the equivalent information for the existing account.
+
+Once the storage account details are known, we need to fetch the storage account key so we can configure Terraform with it:
+
+```bash
+>  az storage account keys list --account-name <storage account name>
+```
+
+Once the storage account is created, one needs to create a container needed to store the Terraform state using [this](https://docs.microsoft.com/en-us/cli/azure/storage/container?view=azure-cli-latest#az-storage-container-create) link. Use the above storage account details required to create the container. 
+
+Once the container is created, update `backend.tfvars` file in your shared infrastructure environment directory with these values. 
+
+### Execute following commands to setup using the Azure backend:
 
 ``` bash
 $ cd ./shared
-$ terraform init
+$ terraform init -backend-config=./backend.tfvars
+```
+
+### Execute the following commands to deploy resources:
+
+``` bash
 $ terraform apply
 ```
 
@@ -67,9 +107,10 @@ variable "app_name" {
 }
 
 ```
-Alternatively, use the cluster.tfvars file to set parameter values as shown below:
+
+Alternatively, use the shared.tfvars file to set parameter values as shown below:
 
 ``` 
-location="esatus"
+location="eastus"
 company="myCompany"
 ```

shared/backend.tfvars

@@ -0,0 +1,4 @@
+storage_account_name="storage-account-name"
+access_key="storage-account-access-key"
+container_name="storage-account-container"
+key="cobalt-backend-terraform.tfstate"

shared/main.tf

@@ -4,6 +4,8 @@ provider "azurerm" {
 
 terraform {
   required_version = "~> 0.11.11"
+  backend "azurerm" {
+  }
 }
 
 locals {

shared/variables.tf

@@ -13,3 +13,15 @@ variable "keyvault_sku" {
   description = "SKU of the keyvault to create"
   default     = "standard"
 }
+
+variable "storage_account_name" {
+  description = "Storage Account Name for backend"
+}
+
+variable "container_name" {
+  description = "Storage Container Name for backend blob"
+}
+
+variable "access_key" {
+  description = "Access Key for backend blob"
+}

tests/backend_test/README.md

@@ -0,0 +1,37 @@
+# Backend Dev Testing
+
+## Description
+
+- [main.sh](../main.sh) is the parent file to execute, that contains all the tests.
+- [assert.sh](../assert.sh) is the test framework used for the base functions required to test the backend.
+
+## Requirements:
+Fill the values required below in test_backend.tfvars using [this](../../shared/README.md###Configure-Terraform-to-Store-State-Data-in-Azure) link.
+
+- Storage Account Name
+- Storage Account Access Key
+- Storage Account Container Name
+- Key Name to store for Terraform State for Test Environment
+
+#### Folder backend_create_multiple_resource_test1 contains backend_create_multiple_resource_test1.tf
+1. Deploy Resource Group
+2. Deploy KeyVault1
+- Should deploy above resources
+
+#### Folder backend_create_multiple_resource_test2 contains backend_create_multiple_resource_test2.tf
+1. Deploy Resource Group
+2. Deploy KeyVault1
+3. Deploy KeyVault2 
+- Should not change/deploy resource 1 and 2. Only deploy resource 3.
+
+#### Folder backend_create_multiple_resource_test3 contains backend_create_multiple_resource_test3.tf
+1. Deploy Resource Group
+2. Deploy KeyVault1
+3. Deploy KeyVault2
+- Should not deploy any resource since above 3 resources are already deployed.
+
+## Usage
+``` bash
+cd ../../tests
+./main.sh
+```

tests/backend_test/backend_create_multiple_resource_test1/backend_create_multiple_resource_test1.tf

@@ -0,0 +1,28 @@
+provider "azurerm" {
+    version = "~>1.21.0"
+}
+
+terraform {
+  required_version = "~> 0.11.11"
+  backend "azurerm" {
+  }
+}
+
+resource "azurerm_resource_group" "rg_core" {
+  name = "core-usea-rg-test"
+  location = "eastus"
+}
+
+data "azurerm_client_config" "current" {}
+
+resource "azurerm_key_vault" "keyvault1" {
+  name                = "core-usea-kv1-test"
+  location            = "eastus"
+  resource_group_name = "core-usea-rg-test"
+  tenant_id           = "${data.azurerm_client_config.current.tenant_id}"
+  depends_on          = ["azurerm_resource_group.rg_core"]
+
+  sku {
+    name = "standard"
+  }
+}

tests/backend_test/backend_create_multiple_resource_test2/backend_create_multiple_resource_test2.tf

@@ -0,0 +1,40 @@
+provider "azurerm" {
+    version = "~>1.21.0"
+}
+
+terraform {
+  required_version = "~> 0.11.11"
+  backend "azurerm" {
+  }
+}
+
+resource "azurerm_resource_group" "rg_core" {
+  name = "core-usea-rg-test"
+  location = "eastus"
+}
+
+data "azurerm_client_config" "current" {}
+
+resource "azurerm_key_vault" "keyvault1" {
+  name                = "core-usea-kv1-test"
+  location            = "eastus"
+  resource_group_name = "core-usea-rg-test"
+  tenant_id           = "${data.azurerm_client_config.current.tenant_id}"
+  depends_on          = ["azurerm_resource_group.rg_core"]
+
+  sku {
+    name = "standard"
+  }
+}
+
+resource "azurerm_key_vault" "keyvault2" {
+  name                = "core-usea-kv2-test"
+  location            = "eastus"
+  resource_group_name = "core-usea-rg-test"
+  tenant_id           = "${data.azurerm_client_config.current.tenant_id}"
+  depends_on          = ["azurerm_resource_group.rg_core"]
+
+  sku {
+    name = "standard"
+  }
+}

tests/backend_test/backend_create_multiple_resource_test3/backend_create_multiple_resource_test3.tf

@@ -0,0 +1,40 @@
+provider "azurerm" {
+    version = "~>1.21.0"
+}
+
+terraform {
+  required_version = "~> 0.11.11"
+  backend "azurerm" {
+  }
+}
+
+resource "azurerm_resource_group" "rg_core" {
+  name = "core-usea-rg-test"
+  location = "eastus"
+}
+
+data "azurerm_client_config" "current" {}
+
+resource "azurerm_key_vault" "keyvault1" {
+  name                = "core-usea-kv1-test"
+  location            = "eastus"
+  resource_group_name = "core-usea-rg-test"
+  tenant_id           = "${data.azurerm_client_config.current.tenant_id}"
+  depends_on          = ["azurerm_resource_group.rg_core"]
+
+  sku {
+    name = "standard"
+  }
+}
+
+resource "azurerm_key_vault" "keyvault2" {
+  name                = "core-usea-kv2-test"
+  location            = "eastus"
+  resource_group_name = "core-usea-rg-test"
+  tenant_id           = "${data.azurerm_client_config.current.tenant_id}"
+  depends_on          = ["azurerm_resource_group.rg_core"]
+
+  sku {
+    name = "standard"
+  }
+}

tests/backend_test/backend_test.sh

@@ -0,0 +1,47 @@
+#!/bin/bash
+set -euo pipefail
+
+#Cleanup
+terraform init -backend-config=./test_backend.tfvars backend_create_multiple_resource_test3
+terraform destroy -auto-approve backend_create_multiple_resource_test3
+
+#Check if resource group exists
+#Resource group does not exists
+assertRG core-usea-rg-test not
+
+#Create Resource group and Keyvault 1
+terraform init -backend-config=./test_backend.tfvars backend_create_multiple_resource_test1
+terraform apply -auto-approve backend_create_multiple_resource_test1
+
+#Check if resource group exists
+assertRG core-usea-rg-test
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv1-test
+
+#Key Vault 2 does not exists
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv2-test not
+
+terraform init -backend-config=./test_backend.tfvars backend_create_multiple_resource_test2
+#Deploy Key Vault 2 only
+terraform apply -auto-approve backend_create_multiple_resource_test2
+
+assertRG core-usea-rg-test
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv1-test
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv2-test
+before=`az resource list -g core-usea-rg-test --resource-type Microsoft.KeyVault/vaults --query '[].name' --o tsv | wc -l`
+
+#Deploy Nothing since resources are already deployed
+terraform init -backend-config=./test_backend.tfvars backend_create_multiple_resource_test3
+terraform apply -auto-approve backend_create_multiple_resource_test3
+after=`az resource list -g core-usea-rg-test --resource-type Microsoft.KeyVault/vaults --query '[].name' --o tsv | wc -l`
+assertRG core-usea-rg-test
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv1-test
+assertResource core-usea-rg-test "Microsoft.KeyVault/vaults" core-usea-kv2-test
+
+if [ $before -eq $after ]
+then
+    printf "\e[0;32mTest Passed: Number of resources in resource group are the same before and after the current test \n"
+else
+    printf "\e[0;31mTest Failed... \n"
+fi
+
+terraform destroy -auto-approve backend_create_multiple_resource_test3

tests/backend_test/test_backend.tfvars

@@ -0,0 +1,4 @@
+storage_account_name="storage-account-name"
+access_key="storage-account-access-key"
+container_name="storage-account-container"
+key="cobalt-test-terraform.tfstate"

tests/main.sh

@@ -1,6 +1,7 @@
 #!/bin/bash
 
-. ./assert.sh continue_on_error #  load test functions with continue_on_error/stop_on_error
+. ./assert.sh continue_on_error     # Load test functions with continue_on_error/stop_on_error
 
-. ./sampleTest1.sh  # sample1
-. ./sampleTest2.sh  # sample2
+. ./sampleTest1.sh                  # sample1
+. ./sampleTest2.sh                  # sample2
+. ./backend_test/backend_test.sh            # Tests for terraform backend