Skip to content

Latest commit

 

History

History
168 lines (113 loc) · 10.7 KB

Day3-DevOps-ML-IotEdge.md

File metadata and controls

168 lines (113 loc) · 10.7 KB
Raw

AMLWorkshop-IotEdge-DevOps

ML/IoT/DevOps Hands on workshop

Agenda

Day 3

ML+IoT Edge+DevOps Track

  • 09:30-10:00 Day 1, 2 reflection, Day 3 expectations

    1. Reflect what we've learn in Day 1 and Day 2

    2. Check out basic concept: MLOps

    3. Also check out this MLADS tutorial

    4. There's a great guideline LearnAI_Azure_ML (a GitHub repo), but we will leave this as further exercise for you.

    5. Note the coverage for today:

      • We'll use template to start quickly, instead of developing DevOps pipeline from scratch.
      • We'll make it work, test CI/CD, then tweak a bit to use IoT Edge for deployment target.
      • At the end of the day, initiate the CI/CD pipeline with git commit, and track the pipeline run.

  • 10:00-11:50 Dev environment setup: Use GitHub Desktop, Azure DevOps(create DevOps account, Organization), create from Azure ML template, customize Build Pipeline

    1. Create an Azure DevOps account from DevOps start page - Start Free What you also create is an organization. Note the organization name created.

    2. We will use this quick starter - Demo Generator

      1. Alternative way to do this is LearnAI_Azure_ML, which helps you with step-by-step approach to create the pipeline from scratch. We will not use this in this workshop.

    3. From the Demo Generator, choose the template, Azure Machine Learning. It's under DevOps Labs tab. Choose your organization, and specify the project name to create.

    4. Explore Repos

      1. Edit config.json under aml_config. You can obtain the content for this file from AML service Workspace Overview from Azure Portal.
      2. Notice that editing this file lets to commit it to master branch, which will initiate the Build Pipeline. It will fail and we'll fix the issue in the following steps.
        • You can alternatively commit it to another branch, and merge it later into the master branch. A general git practice.

      Note: Instead of keeping sensitive files in Repo you could use Secure File feature from Azure Pipelines. A sample guidance is here. More details on Secure Files here.

    5. Explore Pipelines

    6. Edit Build Pipeline DevOps-for-AI-CI

      1. Starting from Create or Get Workspace, specify the Azure subscription to use, and authorize.
      2. Save and Queue.
      3. Monitor the run, and fix any outstanding issues.

      Note: We are using Azure CLI Authentication now. Check out other ways to authenticate.

  • 13:00-14:50 Customize Release Pipeline, Git clone using personal token, test CI build

    1. Open Deploy Webservice Release Pipeline. Notice that releases were automatically initiated but failed.

    2. Click Edit for the Release Pipeline. Check out Pre-deployment conditions, and Post-deployment conditions for each stage.

      1. In the Prod - Deploy on AKS stage, check out Gates. See what deployment gates can be added.

      2. Click 1 job, 4 tasks under QA - Deploy on ACI stage.

      3. Specify Azure subscriptions to use for deployment and test.

      4. Continue to the Prod - Deploy on AKS stage and do the same.

    3. The release pipeline is ready. Now let's clone the repo locally, and try committing it to initiate the whole process.

      1. Create a Personal access credential from Azure DevOps portal. To do that:

        1. Click your account icon from top right. Click Security. Click Personal access tokens from left pane.

        2. Click New token. Specify scope for this token. In this workshop, allow read and write for Code.

        3. When you click Create, a token is created. Copy this to somewhere.

      2. Click Repos from Azure DevOps. Copy the URL of your repo. It looks like this: https://dev.azure.com/<org name>/<project name>/_git/<repo name>

      3. Git clone using GitHub Desktop. To do that:

        1. From the local computer, open GitHub Desktop.
        2. Open File menu, click Clone repository.
        3. Click URL tab, paste the URL copied above.
        4. Make sure the local path does not contain any files.
        5. It will ask you to authenticate. Use your Azure account and personal access token you created above.

      4. Use VS Code or any other editor to open the local repo.

      5. Let's edit AKS cluster definition to use Standard_D3_V2 instead of Standard_F2 (unless you have already increased quota in your region). To do that:

        1. Open 51-deployOnAks.py, click Edit, and change the vm_size parameter of AksCompute.provisioning_configuration() to Standard_D3_v2 (be careful - case sensitive).

          except:
    aks_name = "aks" + datetime.datetime.now().strftime("%m%d%H")
    aks_service_name = "akswebservice" + datetime.datetime.now().strftime("%m%d%H")
    prov_config = AksCompute.provisioning_configuration(
        agent_count=6, vm_size="Standard_D3_v2", location="eastus"
    )
    print(
        "No AKS found in aks_webservice.json. Creating new Aks: {} and AKS Webservice: {}".format(
            aks_name, aks_service_name
        )
    )

      6. Git commit and git push (using command line or GitHub Desktop). Monitor Pipeline initiation.

        1. If needed, delete existing Image and/or Model from Azure ML service Workspace before testing.
        2. If the first stage (QA - Deploy on ACI) in the Release Pipeline was succesful, it will wait for your Approval for the second stage (Prod - Deploy on AKS)
        3. Approve and monitor the final stage. Note: Current 61-AksWebserviceTest.py deletes the web service (deployment) at the end. Change if needed.

      7. If you want to re-test CI/CD, you may make any arbitrary change (for example README.md), and do git commit and git push.

  • 15:00-16:50 Integrate with IoT Edge deployment

    1. We'll create a DevOps Project from Azure Portal this time, leveraging this and this, and an additional sample code (shared below).

    2. Create new DevOps Project.

      • IoT Hub is also created.

    3. You can now go to dev.azure.com and continue browsing.

    4. First disable CI for a moment. To do that:

      1. Go to Pipelines - Build Pipelines.
      2. Click Triggers tab, uncheck Enable continuous integration, save.

    5. Git clone its repo. Create a personal token if needed.

    6. Copy these files(TBD)(VisionSampleModule) into /modules/VisionSampleModule of that local repo, git commit, and git push. Now the additional module for our Edge deployment is added.

    7. Rename /deployment.template.json into /deployment.template.basic.json

    8. Copy this file(TBD) to / as /deployment.template.visionsample.json of local repo.

    9. Then git commit and git push.

    10. Modify Build Pipelines to work on our VisionSampleModule, not the original sample. To do that:

      1. Go to Build Pipeline, edit the Build Pipeline.

        • deployment template: deployment.template.visionsample.json
        • platform: arm32v7

      2. Click Variable tab, add variable REGISTRYNAME

      3. Save and Queue to test the build.

    11. Go to Azure Portal, open the IotHub created above, choose IoT Edge from left pane. Click the edge device myEdgeDevice, copy connection string and register it on AI Camera.

    12. Test locally.

    13. Enable CI. To do that:

      1. Go to Pipelines - Build Pipelines.
      2. Click Triggers tab, check Enable continuous integration, save.

    14. Test with git commit.

  • 17:00-17:50 Questions and answers