Skip to content
Scott Cate Scott Cate
Scott Cate and Scott Cate updates
Latest commit 10694bf Feb 24, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
demos updates Feb 24, 2019
setup updates Feb 24, 2019
.gitignore init Dec 6, 2018
README.md updates Feb 24, 2019
cleanup.sh updates Feb 24, 2019
reset.sh updates Feb 24, 2019
setup.sh updates Feb 24, 2019

README.md

SRE10 - Modernizing your infrastructure: moving to Infrastructure as Code

This is a work-in-progress. Please feel free to update to clarify issues that you observe.

Flavor text around the context for this set of demos

Deploying applications to the cloud can be as simple as clicking the mouse a few times and running "git push". The applications running at Tailwind Traders, however, are quite a bit more complex and, correspondingly, so are our deployments. The only way that we can reliably deploy complex applications (such as our sales and fulfillment system) is to automate it.

In this module, you'll learn how Tailwind Traders uses automation with Azure Resource Management (ARM) templates to provision infrastructure, reducing the chances of errors and inconsistency caused by manual point and click. Once in place, we move on to deploying our applications using continuous integration and continuous delivery, powered by Azure DevOps.

The demo environment

In Azure CloudShell

Note: the git commands below require some auth setup, see Appendix A at the end of this document.

Get the code

We're going to make a working directory called source and then clone the setup repo into

mkdir ~/source
pushd ~/source

# This repo has all the setup scripts for SRE10 and application code
git clone https://dev.azure.com/ignite-tour-lp5/_git/SRE10-Setup

# Setup scripts are also available publicly but may not be up to date

git clone https://github.com/Microsoft/IgniteTheTour

# This repo has the database schema scripts
git clone https://github.com/Azure-Samples/tailwind-traders

If you are going to use the public repo:

pushd ~/source/IgniteTheTour/SRE\ -\ Operating\ applications\ and\ infrastructure\ in\ the\ cloud/SRE10

If you are going to use the private repo:

pushd ~/source/SRE10-Setup

Set up the demo environment

The setup script depends on a few variables that can be modified in 0-params.sh.

CITY="berlin"
LEARNING_PATH="SRE"
SESSION_NUMBER="10"
SUBSCRIPTION="Ignite the Tour"
LOCATION="westeurope"

You should modify CITY and (maybe) LOCATION as appropriate. APP_ENVIRONMENT will get set during the run of setup.sh and doesn't need to be modified.

By default, the script will set up 3 resource groups in the form of SRE10-${CITY}-${APP_ENVIRONMENT} so each person will have an individual standalone environment.

# edit the CITY (and potentially) LOCATION parameters.
code ./setup/0-params.sh

# run the setup
./setup.sh

Output from each of the commands in the scripts can be found in a corresponding log file in ./setup/log (e.g. for ./2-database.sh there will be a ./2-database.log).

Create the Azure DevOps Project

In this next step you are going to create the project that will be running your demo with pipelines. You'll create your own throwaway organization and the SRE10 project along with 4 repos. In the following screenshots, we'll use modernops as a placeholder to illustrate the process.

Where we use {YOURORG}, you'll replace with the organization that you create.

New Organization

First Project

  • Create the first project - name it SRE10

Authorize Azure Subscription

  • Add a service connection for your Azure Subscription New service connection New ARM service connection New ARM service connection detail

Create KeyVault Mapping for Keys and Connection Strings

  • Navigate to Azure Pipelines Library Azure Pipelines library

  • Create a new variable group Create new variable group

  • Configure the variable group

    • Name
    • Link secrets to keyvault as variables
    • Subscription - use the subscription we've configured above
    • Pick the keyvault that was created in the setup script
    • Click Authorize Configure variable group
  • Add the variables from keyvault Add variable Add variable

  • Save the variable group.

Set up source code repos

  • Create the repositories

    • web-app-infra
    • frontend
    • product-service
    • inventory-service
  • You should already have the setup repository cloned to your workstation. If not, go ahead and follow the instructions above for the setup repo.

  • Clone the empty repositories that we just set up to your local workstation

pushd ~/source
git clone https://dev.azure.com/{YOURORG}/SRE10/_git/inventory-service
git clone https://dev.azure.com/{YOURORG}/SRE10/_git/product-service
git clone https://dev.azure.com/{YOURORG}/SRE10/_git/frontend
git clone https://dev.azure.com/{YOURORG}/SRE10/_git/web-app-infra

Checkpoint:

Do a directory listing. Minimally should have your 4 empty reposistories and the setup repository SRE10-Setup in the ~/source directory.

  • Stage starting content to the empty repos.

    • PowerShell
copy-item -recurse ./SRE10-Setup/demos/src/web-app-infra/* -destination ./web-app-infra/
copy-item -recurse ./SRE10-Setup/demos/src/frontend/* -destination ./frontend/
copy-item -recurse ./SRE10-Setup/demos/src/product-service/* -destination ./product-service/
copy-item -recurse ./SRE10-Setup/demos/src/inventory-service/* -destination ./inventory-service/
  • Bash
cp -r ./SRE10-Setup/demos/src/web-app-infra/* ./web-app-infra/
cp -r ./SRE10-Setup/demos/src/frontend/* ./frontend/
cp -r ./SRE10-Setup/demos/src/product-service/* -destination ./product-sevice/
cp -r ./SRE10-Setup/demos/src/inventory-service/* ./inventory-service/
  • Change into each directory (web-app-infra, frontend, product-service, and inventory-service), commit, and push each of the repos up to Azure DevOps
git add .
git commit -m 'initial commit'
git push origin master

Now each of your repos should have the first commit.

Create the build pipelines

  • Open the Builds tab New build
  • Create the first build New build
  • Set Azure Repos as the source location for the build New build location
  • Pick web-app-infra New build location
  • It should detect the azure-pipelines.yml in the repo Existing build definition
  • Run the build Run build
  • Navigate back to the builds page via the breadcrumbs and add the other builds. Add more builds

Create the release pipelines

  • Navigate to Azure Pipelines Releases
  • Create the first release New Release Pipeline
  • Use an empty template Empty template
  • Name the first stage Dev Name task
  • Rename the release as App Environment Release Name release
  • Add an artifact to the release Add artifact
  • Configure the source as web-app-infra
  • Configure the Default version as Latest Add artifact
  • Click "Add"
  • Link the KeyVault mapped variables we created above to the release. Link the KeyVault mapped variables Link the KeyVault mapped variables Link the KeyVault mapped variables
  • Open the "Dev" stage Open task detail
  • Add a task Add a task
  • Use the search to find the Azure Resource Group Deployment task Add a task
  • Configure the release task
    • Subscription can use the mapped service connection
    • Resource group name - SRE10-app-berlin-dev
    • Location - West US 2 Configure the task
    • Pick the template template.json from the published artifact location Select from artifact Select from artifact
    • Pick the parameters file parameters.json from the published artifact location
    • Select the Complete deployment mode Select from artifact
    • Select Override Template Parameters Select override template parameters
      • appServicePlanName: "tw-svc-SRE10-dev"
      • frontendAppName: "tw-frontend-SRE10-dev"
      • inventoryServiceName: "tw-inventory-SRE10-dev"
      • sqlConnectionString: "$(InventoryContextSQL-dev)"
      • productServiceName: "tw-product-SRE10-dev"
      • cosmosConnectionString: "$(MongoConnectionString-dev)" Select override template parameters detail
  • Save the release
  • Navigate to the pipeline view and clone the Dev stage. Clone the dev task
  • Rename the stage UAT and update any references from -dev to -uat.
  • Check the following for needed replacements
    • Resource group
    • All of the template parameters
  • Navigate to the pipeline view and clone the UAT stage.
  • Rename the stage Prod and update any references from -uat to -prod.
  • Check the following for needed replacements
    • Resource group
    • All of the template parameters
  • Navigate to the pipeline view and configure continuous deployment Configure continuous delivery
  • Add approval to the Prod stage Configure continuous approval Configure continuous approval
  • Create the next release pipeline for the Inventory Service Next release pipeline
  • Configure the artifact to pull from the Inventory Service build
  • Configure continuous deployment
  • Configure a dev stage to use the Azure CLI task
    • Use Inline Script
    • Use the below CLI
az webapp deployment source config-zip --resource-group SRE10-app-berlin-dev --name tw-inventory-SRE10-dev --src ./_inventory-service/drop/InventoryService.Api.zip
  • Copy the Dev stage and create a UAT stage.
  • Update the task script to use -uat instead of -dev
  • Copy the UAT stage and create a Prod stage.
  • Update the task script to use -prod instead of -uat
  • Save the release
  • Create the next release pipeline for the Product Service
  • Configure the artifact to pull from the Product Service build
  • Configure continuous deployment
  • Configure a dev stage to use the Azure CLI task
    • Use Inline Script
    • Use the below CLI
az webapp deployment source config-zip --resource-group SRE10-app-berlin-dev --name tw-product-SRE10-dev --src ./_product-service/drop/ProductService.zip
  • Copy the Dev stage and create a UAT stage.
  • Update the task script to use -uat instead of -dev
  • Copy the UAT stage and create a Prod stage.
  • Update the task script to use -prod instead of -uat
  • Save the release
  • Create the next release pipeline for the Frontend App
  • Configure the artifact to pull from the Frontend build
  • Configure continuous deployment
  • Configure a dev stage to use the Azure CLI task
    • Use Inline Script
    • Use the below CLI
az webapp deployment source config-zip --resource-group SRE10-app-berlin-dev  --name tw-frontend-SRE10-dev  --src ./_frontend/drop/Frontend-dev.zip
  • Copy the Dev stage and create a UAT stage.
  • Update the task script to use -uat instead of -dev
  • Copy the UAT stage and create a Prod stage.
  • Update the task script to use -prod instead of -uat
  • Save the release

Running the Demos

Provision Infra from the Portal

  • Sign in to the Azure Portal - https://portal.azure.com Create a resource
  • Click "Create a resource" Create a web app
  • Click "Web App" Web app details
    • 1 Add an app name SRE10-berlin-manualdeploy
    • Ensure the subscription is Ignite the Tour
    • The resource group name should match the app name
    • 2 Change the OS to Linux
    • 3 Ensure publish is set to Code
    • 4 Set the runtime stack to Node.js 8.11
  • Change the App Service plan to Create New App service plan App service plan
  • Name the app service plan SRE10-berlin-manualdeploy
  • Click OK Web app details
  • Click Create
  • From the portal, select the resource group SRE10-{YOURCITY}-manualdeploy (navigate to it from the top-level Resource Group menu)
  • Show the created or creating web application Web app deployment
  • Leave the portal open to this location for the next demo

Provision Infra with an ARM Template

  • In the resource group, click "Deployments" Web app deployment
  • Click on the deployment Deployment list
  • Click on Template Deployment detail
  • This can give us a starting place to create our infrastructure as code.
  • Open cloud shell ( https://shell.azure.com ) CloudShell
  • In CloudShell, the below snippet will duplicate the deployment we created through the portal.
source ~/source/IgniteTheTour/SRE - Operating applications and infrastructure in the cloud/SRE10/setup/0-params.sh
cd ~/source/IgniteTheTour/SRE - Operating applications and infrastructure in the cloud/SRE10/demos/2-webapp_from_template
code .
sub=$(az account show --query id -o tsv)
./deploy.sh -i $sub -g SRE10-${CITY}-templatedeploy -n demo -l ${LOCATION}
  • In the portal, show the deployed app service and plan Deployed template

CI/CD for infrastructure

  • In Visual Studio Code, open the git repository for the web app deployment ./SRE10/web-app-infra Edit template in VS Code
  • Create a branch in VSCode Create branch in VS Code
  • Edit template to add sku and skuCode parameters
"sku":{
    "type": "string",
    "defaultValue" : "PremiumV2",
    "metadata": {
        "description": "The SKU of App Service Plan "
    }
},
"skuCode":{
    "type": "string",
    "defaultValue" : "P1v2",
    "metadata": {
        "description": "The SKU of App Service Plan "
    }
},
  • Delete the sku and skuCode variables.
    "variables": {
      "location": "[resourceGroup().location]",
      "sku": "PremiumV2",
      "skuCode": "P1v2"
    },

to

    "variables": {
      "location": "[resourceGroup().location]"
    },
  • Edit the variable references for sku and skuCode to be parameter references
        "sku": {
          "Tier": "[variables('sku')]",
          "Name": "[variables('skuCode')]"
        },

to

        "sku": {
          "Tier": "[parameters('sku')]",
          "Name": "[parameters('skuCode')]"
        },
  • Sync changes to origin Stage change Comment and commit Sync changes

  • Create PR Create pull request

  • Review and merge Approve pull request Complete pull request

  • Build will validate the ARM template

  • Release will deploy the template

  • Talk about ARM deployment modes, incremental vs complete

CI/CD for applications

  • In Visual Studio Code, open the git repository for the web app deployment ./SRE10/frontend

  • Edit header on src/index.html to add the city to the header Edit index.html

  • Commit and push changes to master (for time). We'd ideally follow a pull request process with code review for these changes as well.

Cleaning up

cd ~
source ~/source/IgniteTheTour/SRE - Operating applications and infrastructure in the cloud/SRE10/setup/0-params.sh
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-app-${CITY}-dev" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-app-${CITY}-uat" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-app-${CITY}-prod" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-db-${CITY}-dev" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-db-${CITY}-uat" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-db-${CITY}-prod" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-${CITY}-manualdeploy" --yes --no-wait
az group delete -n "${LEARNING_PATH}${SESSION_NUMBER}-${CITY}-templatedeploy" --yes --no-wait

rm ~/source/IgniteTheTour/SRE - Operating applications and infrastructure in the cloud/SRE10/setup/.dbpass
rm -rf ~/source/tailwind-traders
rm -rf ~/source/IgniteTheTour

Appendix A: git auth

During the period of time where the code for this demo env lives in private repos, there are two separate sets of git auth that have to be set up a single time:

  1. auth for github.com: create a personal access token by choosing Settings->Developer settings->personal access tokens from the drop down menu under your picture on github.com (when logged in). When you create a person access token, be sure to choose the "[ ] repo Full control of private repositories" scope box. Note: you must also be a member of the Azure-Samples Organization for the repo to be accessible. For more infomation on personal auth tokens see https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/. Your github username and the personal access token you created will be used for the git clone prompts.
You can’t perform that action at this time.